Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
11 out of 60 is failed (threshold=3) [.....X.......X...X................X...........XX...X.XXX...X] duration:
1m0.002089262s
FAIL (2025-07-09 00:42:59 UTC)
operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1m0.002117612s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [................................................X....X......] duration:
833.335147ms
FAIL (2025-07-09 00:42:58 UTC)
orbiter warning:
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
tion; conn closed: op; piecestore: piecestore close: context canceled
Jul 9 00:37:52.848582652 (375h26m19.774877332s) [seg_pos={0 0}, upload=300672]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 00:37:52.848656023 (375h26m19.774950766s) [upload=300672]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 00:37:52.848661061 (375h26m19.774955738s) [upload=300672]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 00:37:53.034475703 (375h26m19.960770383s) [upload=300672]: Done uploading segments...
Jul 9 00:37:53.034504712 (375h26m19.960799388s) [upload=300672]: Upload finished. err: <nil>
Jul 9 00:38:14.795156836 (375h26m41.721451522s) [upload=300678]: Starting upload
Jul 9 00:38:14.795233303 (375h26m41.721527988s) [upload=300678]: Done starting upload
Jul 9 00:38:14.795335298 (375h26m41.721629977s) [upload=300678]: Uploading segments...
Jul 9 00:38:14.795594071 (375h26m41.721888765s) [upload=300678]: Got next segment. Inline: false
Jul 9 00:38:14.795674023 (375h26m41.721968717s) [upload=300678]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 00:38:14.795689186 (375h26m41.721983879s) [upload=300678]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 00:38:14.823801892 (375h26m41.750096591s) [seg_pos={0 0}, upload=300678]: Begin upload segment...
Jul 9 00:38:14.823863472 (375h26m41.75015817s) [seg_pos={0 0}, upload=300678]: Done begin upload segment.
Jul 9 00:38:14.823896674 (375h26m41.750191376s) [noise=true , address=104.36.82.8:4969 , piece_id=ONQRKW57EBGOMVH32QVYYAYHVD6HTLQYII2JKSKGM6SAAYMQ6JPQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.823907419 (375h26m41.750202111s) [noise=true , address=77.90.31.11:4553 , piece_id=V77ZVDEM2MDH53UYS6PSMMYD2SNLDD4BEA3R3DSG6AFYPMDMG2BQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.824421129 (375h26m41.750715827s) [noise=true , address=45.91.116.8:4045 , piece_id=GZPXFXFCJMYI3RIZF5KWZI5R3VRIYGUDXC6PS4XZI5WB27JZKVXQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.824849933 (375h26m41.751144628s) [noise=true , address=45.91.116.6:4975 , piece_id=CPPBPWYULER3ZOZRD2JXYJOKLAULE3YCV6ZEONQLDUHOXRUPTIAQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.824923234 (375h26m41.751217926s) [noise=true , address=77.90.31.16:4733 , piece_id=C2VQHR4WVBP7XSRLADN3NN6OCTO5ROEDC6ZM2XTDNNDMKMACR6DA, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.825248920 (375h26m41.751543617s) [noise=true , address=45.91.116.12:4226 , piece_id=QWG3A45EEBSNR2D4XYIBE3Y32SRK3AWS5QEFZMOVAWBZSC6EASQA, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.825280427 (375h26m41.751575123s) [noise=true , address=104.36.82.16:4293 , piece_id=WY5S2ADFULP3ZIF5LIKSTVJCPEM3CTG3DQQRLXHBMDXB634Y2THA, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.825569855 (375h26m41.751864541s) [noise=true , address=104.36.82.11:4113 , piece_id=75XEEPUONKKHCRZJQLRRL3RJMSILZBWJ6HRVZBNJAHWCC4PQ3YVA, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.825642542 (375h26m41.751937239s) [noise=true , address=104.36.82.6:4874 , piece_id=J5ZN2VIWDNZY4SZ3GGZXJCHFEEMLPEDVT6443SEITLPHMHRWKG4A, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.825906508 (375h26m41.752201197s) [noise=true , address=45.91.116.14:4191 , piece_id=FTHEMV3GWZNLUMSIIMMQ5IRQZPFJ6F4RVGXM2XVSVRH2KWLTMZZA, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.826326502 (375h26m41.752621198s) [noise=true , address=45.91.116.8:4047 , piece_id=TDXBCU3XF32U6LCCRHR4YAKYWOOY4NMTLULNTLABNTRIBT2PBL6Q, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.826792611 (375h26m41.753087303s) [noise=true , address=45.91.116.10:4083 , piece_id=HFVDYNX5CJHVR743LMTVASKPKCEESO6GRAV3NRPDUN2OZ3P4ORPQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.827187184 (375h26m41.75348188s) [noise=true , address=45.91.116.11:4117 , piece_id=QWTZM6HFPAL3AX474MBFHNVUZNIASMPACIXSVLNANDVLVF7HG3QQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.827594318 (375h26m41.753889018s) [noise=true , address=45.91.116.5:4299 , piece_id=QWABO26BIHTA7HLC6XMH5IL4QC4JZTEBOP2QWIG3ROZHX2PRVKXA, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.827842560 (375h26m41.754137255s) [noise=true , address=77.90.31.19:4843 , piece_id=WW275T6GT6JR5WFTYMAFXP5G5YZJXYMKFU772I5NELO3V4YEHWQQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.828205534 (375h26m41.754500223s) [noise=true , address=104.36.82.4:4803 , piece_id=5R3JBPKYXDOLR7Q4KZ773Y5HFK2KZYYVXGI4YUM4A5WUZ5QDPEVQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.915111848 (375h26m41.841406522s) [upload=300678]: Next returned nil segment
Jul 9 00:38:14.915126722 (375h26m41.841421407s) [upload=300678]: Waiting for error group managing segments...
Jul 9 00:38:14.918322650 (375h26m41.84461734s) [noise=true , address=45.91.116.12:4226 , piece_id=QWG3A45EEBSNR2D4XYIBE3Y32SRK3AWS5QEFZMOVAWBZSC6EASQA, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul 9 00:38:14.918983217 (375h26m41.845277898s) [noise=true , address=77.90.31.16:4733 , piece_id=C2VQHR4WVBP7XSRLADN3NN6OCTO5ROEDC6ZM2XTDNNDMKMACR6DA, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul 9 00:38:14.919620494 (375h26m41.845915182s) [noise=true , address=45.91.116.14:4191 , piece_id=FTHEMV3GWZNLUMSIIMMQ5IRQZPFJ6F4RVGXM2XVSVRH2KWLTMZZA, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul 9 00:38:14.919706240 (375h26m41.846000916s) [noise=true , address=45.91.116.11:4117 , piece_id=QWTZM6HFPAL3AX474MBFHNVUZNIASMPACIXSVLNANDVLVF7HG3QQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul 9 00:38:14.920089094 (375h26m41.846383766s) [noise=true , address=45.91.116.6:4975 , piece_id=CPPBPWYULER3ZOZRD2JXYJOKLAULE3YCV6ZEONQLDUHOXRUPTIAQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul 9 00:38:14.920215430 (375h26m41.846510107s) [noise=true , address=45.91.116.8:4045 , piece_id=GZPXFXFCJMYI3RIZF5KWZI5R3VRIYGUDXC6PS4XZI5WB27JZKVXQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul 9 00:38:14.920519778 (375h26m41.846814455s) [noise=true , address=45.91.116.8:4047 , piece_id=TDXBCU3XF32U6LCCRHR4YAKYWOOY4NMTLULNTLABNTRIBT2PBL6Q, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul 9 00:38:14.920715325 (375h26m41.84701s) [noise=true , address=45.91.116.5:4299 , piece_id=QWABO26BIHTA7HLC6XMH5IL4QC4JZTEBOP2QWIG3ROZHX2PRVKXA, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul 9 00:38:14.920921360 (375h26m41.847216035s) [noise=true , address=77.90.31.19:4843 , piece_id=WW275T6GT6JR5WFTYMAFXP5G5YZJXYMKFU772I5NELO3V4YEHWQQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul 9 00:38:14.921284011 (375h26m41.847578698s) [noise=true , address=45.91.116.10:4083 , piece_id=HFVDYNX5CJHVR743LMTVASKPKCEESO6GRAV3NRPDUN2OZ3P4ORPQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul 9 00:38:14.921294145 (375h26m41.84758882s) [noise=true , address=77.90.31.11:4553 , piece_id=V77ZVDEM2MDH53UYS6PSMMYD2SNLDD4BEA3R3DSG6AFYPMDMG2BQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul 9 00:38:18.659642786 (375h26m45.585937465s) [noise=true , address=104.36.82.16:4293 , piece_id=WY5S2ADFULP3ZIF5LIKSTVJCPEM3CTG3DQQRLXHBMDXB634Y2THA, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul 9 00:38:44.764948400 (375h27m11.691243084s) [noise=true , address=104.36.82.6:4874 , piece_id=J5ZN2VIWDNZY4SZ3GGZXJCHFEEMLPEDVT6443SEITLPHMHRWKG4A, seg_pos={0 0}, upload=300678]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:104.36.82.6:4874): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:38:44.765367526 (375h27m11.691662219s) [noise=true , address=104.36.82.8:4969 , piece_id=ONQRKW57EBGOMVH32QVYYAYHVD6HTLQYII2JKSKGM6SAAYMQ6JPQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 159.223.143.39:50836->104.36.82.8:4969: use of closed network connection; write tcp 159.223.143.39:50836->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50836->104.36.82.8:4969: use of closed network connection
Jul 9 00:38:44.765546875 (375h27m11.691841563s) [noise=true , address=104.36.82.11:4113 , piece_id=75XEEPUONKKHCRZJQLRRL3RJMSILZBWJ6HRVZBNJAHWCC4PQ3YVA, seg_pos={0 0}, upload=300678]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:104.36.82.11:4113): protocol: write tcp 159.223.143.39:53580->104.36.82.11:4113: use of closed network connection; write tcp 159.223.143.39:53580->104.36.82.11:4113: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:53580->104.36.82.11:4113: use of closed network connection
Jul 9 00:38:44.765679043 (375h27m11.691973731s) [noise=true , address=104.36.82.4:4803 , piece_id=5R3JBPKYXDOLR7Q4KZ773Y5HFK2KZYYVXGI4YUM4A5WUZ5QDPEVQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:104.36.82.4:4803): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:38:44.765727083 (375h27m11.692021781s) [seg_pos={0 0}, upload=300678]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Jul 9 00:38:44.796212501 (375h27m11.722507185s) [upload=300678]: Done uploading segments...
Jul 9 00:38:44.796240113 (375h27m11.722534802s) [upload=300678]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 833.37129ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
11 out of 60 is failed (threshold=3) [X.X.......X...X......X.......X...X...X...........XX...X.....] duration:
17.424906649s
FAIL (2025-07-09 00:42:57 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 17.42493518s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
12 out of 60 is failed (threshold=3) [XX.X.......X...X......X.......X...X...X...........XX...X....] duration:
15.357321094s
FAIL (2025-07-09 00:42:28 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 15.357347254s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.................................................X....X.....] duration:
416.437641ms
FAIL (2025-07-09 00:42:16 UTC)
orbiter warning:
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
tion; conn closed: op; piecestore: piecestore close: context canceled
Jul 9 00:37:52.848582652 (375h26m19.774877332s) [seg_pos={0 0}, upload=300672]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 00:37:52.848656023 (375h26m19.774950766s) [upload=300672]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 00:37:52.848661061 (375h26m19.774955738s) [upload=300672]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 00:37:53.034475703 (375h26m19.960770383s) [upload=300672]: Done uploading segments...
Jul 9 00:37:53.034504712 (375h26m19.960799388s) [upload=300672]: Upload finished. err: <nil>
Jul 9 00:38:14.795156836 (375h26m41.721451522s) [upload=300678]: Starting upload
Jul 9 00:38:14.795233303 (375h26m41.721527988s) [upload=300678]: Done starting upload
Jul 9 00:38:14.795335298 (375h26m41.721629977s) [upload=300678]: Uploading segments...
Jul 9 00:38:14.795594071 (375h26m41.721888765s) [upload=300678]: Got next segment. Inline: false
Jul 9 00:38:14.795674023 (375h26m41.721968717s) [upload=300678]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 00:38:14.795689186 (375h26m41.721983879s) [upload=300678]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 00:38:14.823801892 (375h26m41.750096591s) [seg_pos={0 0}, upload=300678]: Begin upload segment...
Jul 9 00:38:14.823863472 (375h26m41.75015817s) [seg_pos={0 0}, upload=300678]: Done begin upload segment.
Jul 9 00:38:14.823896674 (375h26m41.750191376s) [noise=true , address=104.36.82.8:4969 , piece_id=ONQRKW57EBGOMVH32QVYYAYHVD6HTLQYII2JKSKGM6SAAYMQ6JPQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.823907419 (375h26m41.750202111s) [noise=true , address=77.90.31.11:4553 , piece_id=V77ZVDEM2MDH53UYS6PSMMYD2SNLDD4BEA3R3DSG6AFYPMDMG2BQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.824421129 (375h26m41.750715827s) [noise=true , address=45.91.116.8:4045 , piece_id=GZPXFXFCJMYI3RIZF5KWZI5R3VRIYGUDXC6PS4XZI5WB27JZKVXQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.824849933 (375h26m41.751144628s) [noise=true , address=45.91.116.6:4975 , piece_id=CPPBPWYULER3ZOZRD2JXYJOKLAULE3YCV6ZEONQLDUHOXRUPTIAQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.824923234 (375h26m41.751217926s) [noise=true , address=77.90.31.16:4733 , piece_id=C2VQHR4WVBP7XSRLADN3NN6OCTO5ROEDC6ZM2XTDNNDMKMACR6DA, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.825248920 (375h26m41.751543617s) [noise=true , address=45.91.116.12:4226 , piece_id=QWG3A45EEBSNR2D4XYIBE3Y32SRK3AWS5QEFZMOVAWBZSC6EASQA, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.825280427 (375h26m41.751575123s) [noise=true , address=104.36.82.16:4293 , piece_id=WY5S2ADFULP3ZIF5LIKSTVJCPEM3CTG3DQQRLXHBMDXB634Y2THA, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.825569855 (375h26m41.751864541s) [noise=true , address=104.36.82.11:4113 , piece_id=75XEEPUONKKHCRZJQLRRL3RJMSILZBWJ6HRVZBNJAHWCC4PQ3YVA, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.825642542 (375h26m41.751937239s) [noise=true , address=104.36.82.6:4874 , piece_id=J5ZN2VIWDNZY4SZ3GGZXJCHFEEMLPEDVT6443SEITLPHMHRWKG4A, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.825906508 (375h26m41.752201197s) [noise=true , address=45.91.116.14:4191 , piece_id=FTHEMV3GWZNLUMSIIMMQ5IRQZPFJ6F4RVGXM2XVSVRH2KWLTMZZA, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.826326502 (375h26m41.752621198s) [noise=true , address=45.91.116.8:4047 , piece_id=TDXBCU3XF32U6LCCRHR4YAKYWOOY4NMTLULNTLABNTRIBT2PBL6Q, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.826792611 (375h26m41.753087303s) [noise=true , address=45.91.116.10:4083 , piece_id=HFVDYNX5CJHVR743LMTVASKPKCEESO6GRAV3NRPDUN2OZ3P4ORPQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.827187184 (375h26m41.75348188s) [noise=true , address=45.91.116.11:4117 , piece_id=QWTZM6HFPAL3AX474MBFHNVUZNIASMPACIXSVLNANDVLVF7HG3QQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.827594318 (375h26m41.753889018s) [noise=true , address=45.91.116.5:4299 , piece_id=QWABO26BIHTA7HLC6XMH5IL4QC4JZTEBOP2QWIG3ROZHX2PRVKXA, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.827842560 (375h26m41.754137255s) [noise=true , address=77.90.31.19:4843 , piece_id=WW275T6GT6JR5WFTYMAFXP5G5YZJXYMKFU772I5NELO3V4YEHWQQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.828205534 (375h26m41.754500223s) [noise=true , address=104.36.82.4:4803 , piece_id=5R3JBPKYXDOLR7Q4KZ773Y5HFK2KZYYVXGI4YUM4A5WUZ5QDPEVQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.915111848 (375h26m41.841406522s) [upload=300678]: Next returned nil segment
Jul 9 00:38:14.915126722 (375h26m41.841421407s) [upload=300678]: Waiting for error group managing segments...
Jul 9 00:38:14.918322650 (375h26m41.84461734s) [noise=true , address=45.91.116.12:4226 , piece_id=QWG3A45EEBSNR2D4XYIBE3Y32SRK3AWS5QEFZMOVAWBZSC6EASQA, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul 9 00:38:14.918983217 (375h26m41.845277898s) [noise=true , address=77.90.31.16:4733 , piece_id=C2VQHR4WVBP7XSRLADN3NN6OCTO5ROEDC6ZM2XTDNNDMKMACR6DA, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul 9 00:38:14.919620494 (375h26m41.845915182s) [noise=true , address=45.91.116.14:4191 , piece_id=FTHEMV3GWZNLUMSIIMMQ5IRQZPFJ6F4RVGXM2XVSVRH2KWLTMZZA, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul 9 00:38:14.919706240 (375h26m41.846000916s) [noise=true , address=45.91.116.11:4117 , piece_id=QWTZM6HFPAL3AX474MBFHNVUZNIASMPACIXSVLNANDVLVF7HG3QQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul 9 00:38:14.920089094 (375h26m41.846383766s) [noise=true , address=45.91.116.6:4975 , piece_id=CPPBPWYULER3ZOZRD2JXYJOKLAULE3YCV6ZEONQLDUHOXRUPTIAQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul 9 00:38:14.920215430 (375h26m41.846510107s) [noise=true , address=45.91.116.8:4045 , piece_id=GZPXFXFCJMYI3RIZF5KWZI5R3VRIYGUDXC6PS4XZI5WB27JZKVXQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul 9 00:38:14.920519778 (375h26m41.846814455s) [noise=true , address=45.91.116.8:4047 , piece_id=TDXBCU3XF32U6LCCRHR4YAKYWOOY4NMTLULNTLABNTRIBT2PBL6Q, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul 9 00:38:14.920715325 (375h26m41.84701s) [noise=true , address=45.91.116.5:4299 , piece_id=QWABO26BIHTA7HLC6XMH5IL4QC4JZTEBOP2QWIG3ROZHX2PRVKXA, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul 9 00:38:14.920921360 (375h26m41.847216035s) [noise=true , address=77.90.31.19:4843 , piece_id=WW275T6GT6JR5WFTYMAFXP5G5YZJXYMKFU772I5NELO3V4YEHWQQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul 9 00:38:14.921284011 (375h26m41.847578698s) [noise=true , address=45.91.116.10:4083 , piece_id=HFVDYNX5CJHVR743LMTVASKPKCEESO6GRAV3NRPDUN2OZ3P4ORPQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul 9 00:38:14.921294145 (375h26m41.84758882s) [noise=true , address=77.90.31.11:4553 , piece_id=V77ZVDEM2MDH53UYS6PSMMYD2SNLDD4BEA3R3DSG6AFYPMDMG2BQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul 9 00:38:18.659642786 (375h26m45.585937465s) [noise=true , address=104.36.82.16:4293 , piece_id=WY5S2ADFULP3ZIF5LIKSTVJCPEM3CTG3DQQRLXHBMDXB634Y2THA, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul 9 00:38:44.764948400 (375h27m11.691243084s) [noise=true , address=104.36.82.6:4874 , piece_id=J5ZN2VIWDNZY4SZ3GGZXJCHFEEMLPEDVT6443SEITLPHMHRWKG4A, seg_pos={0 0}, upload=300678]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:104.36.82.6:4874): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:38:44.765367526 (375h27m11.691662219s) [noise=true , address=104.36.82.8:4969 , piece_id=ONQRKW57EBGOMVH32QVYYAYHVD6HTLQYII2JKSKGM6SAAYMQ6JPQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 159.223.143.39:50836->104.36.82.8:4969: use of closed network connection; write tcp 159.223.143.39:50836->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50836->104.36.82.8:4969: use of closed network connection
Jul 9 00:38:44.765546875 (375h27m11.691841563s) [noise=true , address=104.36.82.11:4113 , piece_id=75XEEPUONKKHCRZJQLRRL3RJMSILZBWJ6HRVZBNJAHWCC4PQ3YVA, seg_pos={0 0}, upload=300678]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:104.36.82.11:4113): protocol: write tcp 159.223.143.39:53580->104.36.82.11:4113: use of closed network connection; write tcp 159.223.143.39:53580->104.36.82.11:4113: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:53580->104.36.82.11:4113: use of closed network connection
Jul 9 00:38:44.765679043 (375h27m11.691973731s) [noise=true , address=104.36.82.4:4803 , piece_id=5R3JBPKYXDOLR7Q4KZ773Y5HFK2KZYYVXGI4YUM4A5WUZ5QDPEVQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:104.36.82.4:4803): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:38:44.765727083 (375h27m11.692021781s) [seg_pos={0 0}, upload=300678]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Jul 9 00:38:44.796212501 (375h27m11.722507185s) [upload=300678]: Done uploading segments...
Jul 9 00:38:44.796240113 (375h27m11.722534802s) [upload=300678]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 416.463439ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
10 out of 60 is failed (threshold=3) [......X.......X...X................X...........XX...X.XXX...] duration:
49.125374318s
FAIL (2025-07-09 00:41:47 UTC)
context deadline exceeded
Duration: 49.125415216s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [...................................................X....X...] duration:
21.051535934s
FAIL (2025-07-09 00:41:45 UTC)
orbiter warning:
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
tion; conn closed: op; piecestore: piecestore close: context canceled
Jul 9 00:37:52.848582652 (375h26m19.774877332s) [seg_pos={0 0}, upload=300672]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 00:37:52.848656023 (375h26m19.774950766s) [upload=300672]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 00:37:52.848661061 (375h26m19.774955738s) [upload=300672]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 00:37:53.034475703 (375h26m19.960770383s) [upload=300672]: Done uploading segments...
Jul 9 00:37:53.034504712 (375h26m19.960799388s) [upload=300672]: Upload finished. err: <nil>
Jul 9 00:38:14.795156836 (375h26m41.721451522s) [upload=300678]: Starting upload
Jul 9 00:38:14.795233303 (375h26m41.721527988s) [upload=300678]: Done starting upload
Jul 9 00:38:14.795335298 (375h26m41.721629977s) [upload=300678]: Uploading segments...
Jul 9 00:38:14.795594071 (375h26m41.721888765s) [upload=300678]: Got next segment. Inline: false
Jul 9 00:38:14.795674023 (375h26m41.721968717s) [upload=300678]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 00:38:14.795689186 (375h26m41.721983879s) [upload=300678]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 00:38:14.823801892 (375h26m41.750096591s) [seg_pos={0 0}, upload=300678]: Begin upload segment...
Jul 9 00:38:14.823863472 (375h26m41.75015817s) [seg_pos={0 0}, upload=300678]: Done begin upload segment.
Jul 9 00:38:14.823896674 (375h26m41.750191376s) [noise=true , address=104.36.82.8:4969 , piece_id=ONQRKW57EBGOMVH32QVYYAYHVD6HTLQYII2JKSKGM6SAAYMQ6JPQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.823907419 (375h26m41.750202111s) [noise=true , address=77.90.31.11:4553 , piece_id=V77ZVDEM2MDH53UYS6PSMMYD2SNLDD4BEA3R3DSG6AFYPMDMG2BQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.824421129 (375h26m41.750715827s) [noise=true , address=45.91.116.8:4045 , piece_id=GZPXFXFCJMYI3RIZF5KWZI5R3VRIYGUDXC6PS4XZI5WB27JZKVXQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.824849933 (375h26m41.751144628s) [noise=true , address=45.91.116.6:4975 , piece_id=CPPBPWYULER3ZOZRD2JXYJOKLAULE3YCV6ZEONQLDUHOXRUPTIAQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.824923234 (375h26m41.751217926s) [noise=true , address=77.90.31.16:4733 , piece_id=C2VQHR4WVBP7XSRLADN3NN6OCTO5ROEDC6ZM2XTDNNDMKMACR6DA, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.825248920 (375h26m41.751543617s) [noise=true , address=45.91.116.12:4226 , piece_id=QWG3A45EEBSNR2D4XYIBE3Y32SRK3AWS5QEFZMOVAWBZSC6EASQA, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.825280427 (375h26m41.751575123s) [noise=true , address=104.36.82.16:4293 , piece_id=WY5S2ADFULP3ZIF5LIKSTVJCPEM3CTG3DQQRLXHBMDXB634Y2THA, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.825569855 (375h26m41.751864541s) [noise=true , address=104.36.82.11:4113 , piece_id=75XEEPUONKKHCRZJQLRRL3RJMSILZBWJ6HRVZBNJAHWCC4PQ3YVA, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.825642542 (375h26m41.751937239s) [noise=true , address=104.36.82.6:4874 , piece_id=J5ZN2VIWDNZY4SZ3GGZXJCHFEEMLPEDVT6443SEITLPHMHRWKG4A, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.825906508 (375h26m41.752201197s) [noise=true , address=45.91.116.14:4191 , piece_id=FTHEMV3GWZNLUMSIIMMQ5IRQZPFJ6F4RVGXM2XVSVRH2KWLTMZZA, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.826326502 (375h26m41.752621198s) [noise=true , address=45.91.116.8:4047 , piece_id=TDXBCU3XF32U6LCCRHR4YAKYWOOY4NMTLULNTLABNTRIBT2PBL6Q, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.826792611 (375h26m41.753087303s) [noise=true , address=45.91.116.10:4083 , piece_id=HFVDYNX5CJHVR743LMTVASKPKCEESO6GRAV3NRPDUN2OZ3P4ORPQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.827187184 (375h26m41.75348188s) [noise=true , address=45.91.116.11:4117 , piece_id=QWTZM6HFPAL3AX474MBFHNVUZNIASMPACIXSVLNANDVLVF7HG3QQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.827594318 (375h26m41.753889018s) [noise=true , address=45.91.116.5:4299 , piece_id=QWABO26BIHTA7HLC6XMH5IL4QC4JZTEBOP2QWIG3ROZHX2PRVKXA, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.827842560 (375h26m41.754137255s) [noise=true , address=77.90.31.19:4843 , piece_id=WW275T6GT6JR5WFTYMAFXP5G5YZJXYMKFU772I5NELO3V4YEHWQQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.828205534 (375h26m41.754500223s) [noise=true , address=104.36.82.4:4803 , piece_id=5R3JBPKYXDOLR7Q4KZ773Y5HFK2KZYYVXGI4YUM4A5WUZ5QDPEVQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.915111848 (375h26m41.841406522s) [upload=300678]: Next returned nil segment
Jul 9 00:38:14.915126722 (375h26m41.841421407s) [upload=300678]: Waiting for error group managing segments...
Jul 9 00:38:14.918322650 (375h26m41.84461734s) [noise=true , address=45.91.116.12:4226 , piece_id=QWG3A45EEBSNR2D4XYIBE3Y32SRK3AWS5QEFZMOVAWBZSC6EASQA, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul 9 00:38:14.918983217 (375h26m41.845277898s) [noise=true , address=77.90.31.16:4733 , piece_id=C2VQHR4WVBP7XSRLADN3NN6OCTO5ROEDC6ZM2XTDNNDMKMACR6DA, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul 9 00:38:14.919620494 (375h26m41.845915182s) [noise=true , address=45.91.116.14:4191 , piece_id=FTHEMV3GWZNLUMSIIMMQ5IRQZPFJ6F4RVGXM2XVSVRH2KWLTMZZA, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul 9 00:38:14.919706240 (375h26m41.846000916s) [noise=true , address=45.91.116.11:4117 , piece_id=QWTZM6HFPAL3AX474MBFHNVUZNIASMPACIXSVLNANDVLVF7HG3QQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul 9 00:38:14.920089094 (375h26m41.846383766s) [noise=true , address=45.91.116.6:4975 , piece_id=CPPBPWYULER3ZOZRD2JXYJOKLAULE3YCV6ZEONQLDUHOXRUPTIAQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul 9 00:38:14.920215430 (375h26m41.846510107s) [noise=true , address=45.91.116.8:4045 , piece_id=GZPXFXFCJMYI3RIZF5KWZI5R3VRIYGUDXC6PS4XZI5WB27JZKVXQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul 9 00:38:14.920519778 (375h26m41.846814455s) [noise=true , address=45.91.116.8:4047 , piece_id=TDXBCU3XF32U6LCCRHR4YAKYWOOY4NMTLULNTLABNTRIBT2PBL6Q, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul 9 00:38:14.920715325 (375h26m41.84701s) [noise=true , address=45.91.116.5:4299 , piece_id=QWABO26BIHTA7HLC6XMH5IL4QC4JZTEBOP2QWIG3ROZHX2PRVKXA, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul 9 00:38:14.920921360 (375h26m41.847216035s) [noise=true , address=77.90.31.19:4843 , piece_id=WW275T6GT6JR5WFTYMAFXP5G5YZJXYMKFU772I5NELO3V4YEHWQQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul 9 00:38:14.921284011 (375h26m41.847578698s) [noise=true , address=45.91.116.10:4083 , piece_id=HFVDYNX5CJHVR743LMTVASKPKCEESO6GRAV3NRPDUN2OZ3P4ORPQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul 9 00:38:14.921294145 (375h26m41.84758882s) [noise=true , address=77.90.31.11:4553 , piece_id=V77ZVDEM2MDH53UYS6PSMMYD2SNLDD4BEA3R3DSG6AFYPMDMG2BQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul 9 00:38:18.659642786 (375h26m45.585937465s) [noise=true , address=104.36.82.16:4293 , piece_id=WY5S2ADFULP3ZIF5LIKSTVJCPEM3CTG3DQQRLXHBMDXB634Y2THA, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul 9 00:38:44.764948400 (375h27m11.691243084s) [noise=true , address=104.36.82.6:4874 , piece_id=J5ZN2VIWDNZY4SZ3GGZXJCHFEEMLPEDVT6443SEITLPHMHRWKG4A, seg_pos={0 0}, upload=300678]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:104.36.82.6:4874): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:38:44.765367526 (375h27m11.691662219s) [noise=true , address=104.36.82.8:4969 , piece_id=ONQRKW57EBGOMVH32QVYYAYHVD6HTLQYII2JKSKGM6SAAYMQ6JPQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 159.223.143.39:50836->104.36.82.8:4969: use of closed network connection; write tcp 159.223.143.39:50836->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50836->104.36.82.8:4969: use of closed network connection
Jul 9 00:38:44.765546875 (375h27m11.691841563s) [noise=true , address=104.36.82.11:4113 , piece_id=75XEEPUONKKHCRZJQLRRL3RJMSILZBWJ6HRVZBNJAHWCC4PQ3YVA, seg_pos={0 0}, upload=300678]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:104.36.82.11:4113): protocol: write tcp 159.223.143.39:53580->104.36.82.11:4113: use of closed network connection; write tcp 159.223.143.39:53580->104.36.82.11:4113: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:53580->104.36.82.11:4113: use of closed network connection
Jul 9 00:38:44.765679043 (375h27m11.691973731s) [noise=true , address=104.36.82.4:4803 , piece_id=5R3JBPKYXDOLR7Q4KZ773Y5HFK2KZYYVXGI4YUM4A5WUZ5QDPEVQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:104.36.82.4:4803): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:38:44.765727083 (375h27m11.692021781s) [seg_pos={0 0}, upload=300678]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Jul 9 00:38:44.796212501 (375h27m11.722507185s) [upload=300678]: Done uploading segments...
Jul 9 00:38:44.796240113 (375h27m11.722534802s) [upload=300678]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 21.051564792s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
12 out of 60 is failed (threshold=3) [.XX.X.......X...X......X.......X...X...X...........XX...X...] duration:
16.947373795s
FAIL (2025-07-09 00:41:45 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 16.947399267s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
13 out of 60 is failed (threshold=3) [X.XX.X.......X...X......X.......X...X...X...........XX...X..] duration:
21.34759506s
FAIL (2025-07-09 00:41:01 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 21.347621362s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
10 out of 60 is failed (threshold=3) [.......X.......X...X................X...........XX...X.XXX..] duration:
49.243172292s
FAIL (2025-07-09 00:40:58 UTC)
context deadline exceeded
Duration: 49.24320921s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [....................................................X....X..] duration:
387.095651ms
FAIL (2025-07-09 00:40:10 UTC)
orbiter warning:
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
tion; conn closed: op; piecestore: piecestore close: context canceled
Jul 9 00:37:52.848582652 (375h26m19.774877332s) [seg_pos={0 0}, upload=300672]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 00:37:52.848656023 (375h26m19.774950766s) [upload=300672]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 00:37:52.848661061 (375h26m19.774955738s) [upload=300672]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 00:37:53.034475703 (375h26m19.960770383s) [upload=300672]: Done uploading segments...
Jul 9 00:37:53.034504712 (375h26m19.960799388s) [upload=300672]: Upload finished. err: <nil>
Jul 9 00:38:14.795156836 (375h26m41.721451522s) [upload=300678]: Starting upload
Jul 9 00:38:14.795233303 (375h26m41.721527988s) [upload=300678]: Done starting upload
Jul 9 00:38:14.795335298 (375h26m41.721629977s) [upload=300678]: Uploading segments...
Jul 9 00:38:14.795594071 (375h26m41.721888765s) [upload=300678]: Got next segment. Inline: false
Jul 9 00:38:14.795674023 (375h26m41.721968717s) [upload=300678]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 00:38:14.795689186 (375h26m41.721983879s) [upload=300678]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 00:38:14.823801892 (375h26m41.750096591s) [seg_pos={0 0}, upload=300678]: Begin upload segment...
Jul 9 00:38:14.823863472 (375h26m41.75015817s) [seg_pos={0 0}, upload=300678]: Done begin upload segment.
Jul 9 00:38:14.823896674 (375h26m41.750191376s) [noise=true , address=104.36.82.8:4969 , piece_id=ONQRKW57EBGOMVH32QVYYAYHVD6HTLQYII2JKSKGM6SAAYMQ6JPQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.823907419 (375h26m41.750202111s) [noise=true , address=77.90.31.11:4553 , piece_id=V77ZVDEM2MDH53UYS6PSMMYD2SNLDD4BEA3R3DSG6AFYPMDMG2BQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.824421129 (375h26m41.750715827s) [noise=true , address=45.91.116.8:4045 , piece_id=GZPXFXFCJMYI3RIZF5KWZI5R3VRIYGUDXC6PS4XZI5WB27JZKVXQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.824849933 (375h26m41.751144628s) [noise=true , address=45.91.116.6:4975 , piece_id=CPPBPWYULER3ZOZRD2JXYJOKLAULE3YCV6ZEONQLDUHOXRUPTIAQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.824923234 (375h26m41.751217926s) [noise=true , address=77.90.31.16:4733 , piece_id=C2VQHR4WVBP7XSRLADN3NN6OCTO5ROEDC6ZM2XTDNNDMKMACR6DA, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.825248920 (375h26m41.751543617s) [noise=true , address=45.91.116.12:4226 , piece_id=QWG3A45EEBSNR2D4XYIBE3Y32SRK3AWS5QEFZMOVAWBZSC6EASQA, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.825280427 (375h26m41.751575123s) [noise=true , address=104.36.82.16:4293 , piece_id=WY5S2ADFULP3ZIF5LIKSTVJCPEM3CTG3DQQRLXHBMDXB634Y2THA, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.825569855 (375h26m41.751864541s) [noise=true , address=104.36.82.11:4113 , piece_id=75XEEPUONKKHCRZJQLRRL3RJMSILZBWJ6HRVZBNJAHWCC4PQ3YVA, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.825642542 (375h26m41.751937239s) [noise=true , address=104.36.82.6:4874 , piece_id=J5ZN2VIWDNZY4SZ3GGZXJCHFEEMLPEDVT6443SEITLPHMHRWKG4A, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.825906508 (375h26m41.752201197s) [noise=true , address=45.91.116.14:4191 , piece_id=FTHEMV3GWZNLUMSIIMMQ5IRQZPFJ6F4RVGXM2XVSVRH2KWLTMZZA, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.826326502 (375h26m41.752621198s) [noise=true , address=45.91.116.8:4047 , piece_id=TDXBCU3XF32U6LCCRHR4YAKYWOOY4NMTLULNTLABNTRIBT2PBL6Q, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.826792611 (375h26m41.753087303s) [noise=true , address=45.91.116.10:4083 , piece_id=HFVDYNX5CJHVR743LMTVASKPKCEESO6GRAV3NRPDUN2OZ3P4ORPQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.827187184 (375h26m41.75348188s) [noise=true , address=45.91.116.11:4117 , piece_id=QWTZM6HFPAL3AX474MBFHNVUZNIASMPACIXSVLNANDVLVF7HG3QQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.827594318 (375h26m41.753889018s) [noise=true , address=45.91.116.5:4299 , piece_id=QWABO26BIHTA7HLC6XMH5IL4QC4JZTEBOP2QWIG3ROZHX2PRVKXA, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.827842560 (375h26m41.754137255s) [noise=true , address=77.90.31.19:4843 , piece_id=WW275T6GT6JR5WFTYMAFXP5G5YZJXYMKFU772I5NELO3V4YEHWQQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.828205534 (375h26m41.754500223s) [noise=true , address=104.36.82.4:4803 , piece_id=5R3JBPKYXDOLR7Q4KZ773Y5HFK2KZYYVXGI4YUM4A5WUZ5QDPEVQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.915111848 (375h26m41.841406522s) [upload=300678]: Next returned nil segment
Jul 9 00:38:14.915126722 (375h26m41.841421407s) [upload=300678]: Waiting for error group managing segments...
Jul 9 00:38:14.918322650 (375h26m41.84461734s) [noise=true , address=45.91.116.12:4226 , piece_id=QWG3A45EEBSNR2D4XYIBE3Y32SRK3AWS5QEFZMOVAWBZSC6EASQA, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul 9 00:38:14.918983217 (375h26m41.845277898s) [noise=true , address=77.90.31.16:4733 , piece_id=C2VQHR4WVBP7XSRLADN3NN6OCTO5ROEDC6ZM2XTDNNDMKMACR6DA, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul 9 00:38:14.919620494 (375h26m41.845915182s) [noise=true , address=45.91.116.14:4191 , piece_id=FTHEMV3GWZNLUMSIIMMQ5IRQZPFJ6F4RVGXM2XVSVRH2KWLTMZZA, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul 9 00:38:14.919706240 (375h26m41.846000916s) [noise=true , address=45.91.116.11:4117 , piece_id=QWTZM6HFPAL3AX474MBFHNVUZNIASMPACIXSVLNANDVLVF7HG3QQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul 9 00:38:14.920089094 (375h26m41.846383766s) [noise=true , address=45.91.116.6:4975 , piece_id=CPPBPWYULER3ZOZRD2JXYJOKLAULE3YCV6ZEONQLDUHOXRUPTIAQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul 9 00:38:14.920215430 (375h26m41.846510107s) [noise=true , address=45.91.116.8:4045 , piece_id=GZPXFXFCJMYI3RIZF5KWZI5R3VRIYGUDXC6PS4XZI5WB27JZKVXQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul 9 00:38:14.920519778 (375h26m41.846814455s) [noise=true , address=45.91.116.8:4047 , piece_id=TDXBCU3XF32U6LCCRHR4YAKYWOOY4NMTLULNTLABNTRIBT2PBL6Q, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul 9 00:38:14.920715325 (375h26m41.84701s) [noise=true , address=45.91.116.5:4299 , piece_id=QWABO26BIHTA7HLC6XMH5IL4QC4JZTEBOP2QWIG3ROZHX2PRVKXA, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul 9 00:38:14.920921360 (375h26m41.847216035s) [noise=true , address=77.90.31.19:4843 , piece_id=WW275T6GT6JR5WFTYMAFXP5G5YZJXYMKFU772I5NELO3V4YEHWQQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul 9 00:38:14.921284011 (375h26m41.847578698s) [noise=true , address=45.91.116.10:4083 , piece_id=HFVDYNX5CJHVR743LMTVASKPKCEESO6GRAV3NRPDUN2OZ3P4ORPQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul 9 00:38:14.921294145 (375h26m41.84758882s) [noise=true , address=77.90.31.11:4553 , piece_id=V77ZVDEM2MDH53UYS6PSMMYD2SNLDD4BEA3R3DSG6AFYPMDMG2BQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul 9 00:38:18.659642786 (375h26m45.585937465s) [noise=true , address=104.36.82.16:4293 , piece_id=WY5S2ADFULP3ZIF5LIKSTVJCPEM3CTG3DQQRLXHBMDXB634Y2THA, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul 9 00:38:44.764948400 (375h27m11.691243084s) [noise=true , address=104.36.82.6:4874 , piece_id=J5ZN2VIWDNZY4SZ3GGZXJCHFEEMLPEDVT6443SEITLPHMHRWKG4A, seg_pos={0 0}, upload=300678]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:104.36.82.6:4874): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:38:44.765367526 (375h27m11.691662219s) [noise=true , address=104.36.82.8:4969 , piece_id=ONQRKW57EBGOMVH32QVYYAYHVD6HTLQYII2JKSKGM6SAAYMQ6JPQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 159.223.143.39:50836->104.36.82.8:4969: use of closed network connection; write tcp 159.223.143.39:50836->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50836->104.36.82.8:4969: use of closed network connection
Jul 9 00:38:44.765546875 (375h27m11.691841563s) [noise=true , address=104.36.82.11:4113 , piece_id=75XEEPUONKKHCRZJQLRRL3RJMSILZBWJ6HRVZBNJAHWCC4PQ3YVA, seg_pos={0 0}, upload=300678]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:104.36.82.11:4113): protocol: write tcp 159.223.143.39:53580->104.36.82.11:4113: use of closed network connection; write tcp 159.223.143.39:53580->104.36.82.11:4113: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:53580->104.36.82.11:4113: use of closed network connection
Jul 9 00:38:44.765679043 (375h27m11.691973731s) [noise=true , address=104.36.82.4:4803 , piece_id=5R3JBPKYXDOLR7Q4KZ773Y5HFK2KZYYVXGI4YUM4A5WUZ5QDPEVQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:104.36.82.4:4803): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:38:44.765727083 (375h27m11.692021781s) [seg_pos={0 0}, upload=300678]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Jul 9 00:38:44.796212501 (375h27m11.722507185s) [upload=300678]: Done uploading segments...
Jul 9 00:38:44.796240113 (375h27m11.722534802s) [upload=300678]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 387.127605ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
10 out of 60 is failed (threshold=3) [........X.......X...X................X...........XX...X.XXX.] duration:
36.809869597s
FAIL (2025-07-09 00:40:09 UTC)
context deadline exceeded
Duration: 36.809890327s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
13 out of 60 is failed (threshold=3) [.X.XX.X.......X...X......X.......X...X...X...........XX...X.] duration:
7.394279331s
FAIL (2025-07-09 00:39:36 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 7.394306115s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.....................................................X....X.] duration:
22.046553856s
FAIL (2025-07-09 00:39:33 UTC)
orbiter warning:
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
tion; conn closed: op; piecestore: piecestore close: context canceled
Jul 9 00:37:52.848582652 (375h26m19.774877332s) [seg_pos={0 0}, upload=300672]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 00:37:52.848656023 (375h26m19.774950766s) [upload=300672]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 00:37:52.848661061 (375h26m19.774955738s) [upload=300672]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 00:37:53.034475703 (375h26m19.960770383s) [upload=300672]: Done uploading segments...
Jul 9 00:37:53.034504712 (375h26m19.960799388s) [upload=300672]: Upload finished. err: <nil>
Jul 9 00:38:14.795156836 (375h26m41.721451522s) [upload=300678]: Starting upload
Jul 9 00:38:14.795233303 (375h26m41.721527988s) [upload=300678]: Done starting upload
Jul 9 00:38:14.795335298 (375h26m41.721629977s) [upload=300678]: Uploading segments...
Jul 9 00:38:14.795594071 (375h26m41.721888765s) [upload=300678]: Got next segment. Inline: false
Jul 9 00:38:14.795674023 (375h26m41.721968717s) [upload=300678]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 00:38:14.795689186 (375h26m41.721983879s) [upload=300678]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 00:38:14.823801892 (375h26m41.750096591s) [seg_pos={0 0}, upload=300678]: Begin upload segment...
Jul 9 00:38:14.823863472 (375h26m41.75015817s) [seg_pos={0 0}, upload=300678]: Done begin upload segment.
Jul 9 00:38:14.823896674 (375h26m41.750191376s) [noise=true , address=104.36.82.8:4969 , piece_id=ONQRKW57EBGOMVH32QVYYAYHVD6HTLQYII2JKSKGM6SAAYMQ6JPQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.823907419 (375h26m41.750202111s) [noise=true , address=77.90.31.11:4553 , piece_id=V77ZVDEM2MDH53UYS6PSMMYD2SNLDD4BEA3R3DSG6AFYPMDMG2BQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.824421129 (375h26m41.750715827s) [noise=true , address=45.91.116.8:4045 , piece_id=GZPXFXFCJMYI3RIZF5KWZI5R3VRIYGUDXC6PS4XZI5WB27JZKVXQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.824849933 (375h26m41.751144628s) [noise=true , address=45.91.116.6:4975 , piece_id=CPPBPWYULER3ZOZRD2JXYJOKLAULE3YCV6ZEONQLDUHOXRUPTIAQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.824923234 (375h26m41.751217926s) [noise=true , address=77.90.31.16:4733 , piece_id=C2VQHR4WVBP7XSRLADN3NN6OCTO5ROEDC6ZM2XTDNNDMKMACR6DA, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.825248920 (375h26m41.751543617s) [noise=true , address=45.91.116.12:4226 , piece_id=QWG3A45EEBSNR2D4XYIBE3Y32SRK3AWS5QEFZMOVAWBZSC6EASQA, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.825280427 (375h26m41.751575123s) [noise=true , address=104.36.82.16:4293 , piece_id=WY5S2ADFULP3ZIF5LIKSTVJCPEM3CTG3DQQRLXHBMDXB634Y2THA, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.825569855 (375h26m41.751864541s) [noise=true , address=104.36.82.11:4113 , piece_id=75XEEPUONKKHCRZJQLRRL3RJMSILZBWJ6HRVZBNJAHWCC4PQ3YVA, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.825642542 (375h26m41.751937239s) [noise=true , address=104.36.82.6:4874 , piece_id=J5ZN2VIWDNZY4SZ3GGZXJCHFEEMLPEDVT6443SEITLPHMHRWKG4A, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.825906508 (375h26m41.752201197s) [noise=true , address=45.91.116.14:4191 , piece_id=FTHEMV3GWZNLUMSIIMMQ5IRQZPFJ6F4RVGXM2XVSVRH2KWLTMZZA, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.826326502 (375h26m41.752621198s) [noise=true , address=45.91.116.8:4047 , piece_id=TDXBCU3XF32U6LCCRHR4YAKYWOOY4NMTLULNTLABNTRIBT2PBL6Q, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.826792611 (375h26m41.753087303s) [noise=true , address=45.91.116.10:4083 , piece_id=HFVDYNX5CJHVR743LMTVASKPKCEESO6GRAV3NRPDUN2OZ3P4ORPQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.827187184 (375h26m41.75348188s) [noise=true , address=45.91.116.11:4117 , piece_id=QWTZM6HFPAL3AX474MBFHNVUZNIASMPACIXSVLNANDVLVF7HG3QQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.827594318 (375h26m41.753889018s) [noise=true , address=45.91.116.5:4299 , piece_id=QWABO26BIHTA7HLC6XMH5IL4QC4JZTEBOP2QWIG3ROZHX2PRVKXA, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.827842560 (375h26m41.754137255s) [noise=true , address=77.90.31.19:4843 , piece_id=WW275T6GT6JR5WFTYMAFXP5G5YZJXYMKFU772I5NELO3V4YEHWQQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.828205534 (375h26m41.754500223s) [noise=true , address=104.36.82.4:4803 , piece_id=5R3JBPKYXDOLR7Q4KZ773Y5HFK2KZYYVXGI4YUM4A5WUZ5QDPEVQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.915111848 (375h26m41.841406522s) [upload=300678]: Next returned nil segment
Jul 9 00:38:14.915126722 (375h26m41.841421407s) [upload=300678]: Waiting for error group managing segments...
Jul 9 00:38:14.918322650 (375h26m41.84461734s) [noise=true , address=45.91.116.12:4226 , piece_id=QWG3A45EEBSNR2D4XYIBE3Y32SRK3AWS5QEFZMOVAWBZSC6EASQA, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul 9 00:38:14.918983217 (375h26m41.845277898s) [noise=true , address=77.90.31.16:4733 , piece_id=C2VQHR4WVBP7XSRLADN3NN6OCTO5ROEDC6ZM2XTDNNDMKMACR6DA, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul 9 00:38:14.919620494 (375h26m41.845915182s) [noise=true , address=45.91.116.14:4191 , piece_id=FTHEMV3GWZNLUMSIIMMQ5IRQZPFJ6F4RVGXM2XVSVRH2KWLTMZZA, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul 9 00:38:14.919706240 (375h26m41.846000916s) [noise=true , address=45.91.116.11:4117 , piece_id=QWTZM6HFPAL3AX474MBFHNVUZNIASMPACIXSVLNANDVLVF7HG3QQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul 9 00:38:14.920089094 (375h26m41.846383766s) [noise=true , address=45.91.116.6:4975 , piece_id=CPPBPWYULER3ZOZRD2JXYJOKLAULE3YCV6ZEONQLDUHOXRUPTIAQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul 9 00:38:14.920215430 (375h26m41.846510107s) [noise=true , address=45.91.116.8:4045 , piece_id=GZPXFXFCJMYI3RIZF5KWZI5R3VRIYGUDXC6PS4XZI5WB27JZKVXQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul 9 00:38:14.920519778 (375h26m41.846814455s) [noise=true , address=45.91.116.8:4047 , piece_id=TDXBCU3XF32U6LCCRHR4YAKYWOOY4NMTLULNTLABNTRIBT2PBL6Q, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul 9 00:38:14.920715325 (375h26m41.84701s) [noise=true , address=45.91.116.5:4299 , piece_id=QWABO26BIHTA7HLC6XMH5IL4QC4JZTEBOP2QWIG3ROZHX2PRVKXA, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul 9 00:38:14.920921360 (375h26m41.847216035s) [noise=true , address=77.90.31.19:4843 , piece_id=WW275T6GT6JR5WFTYMAFXP5G5YZJXYMKFU772I5NELO3V4YEHWQQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul 9 00:38:14.921284011 (375h26m41.847578698s) [noise=true , address=45.91.116.10:4083 , piece_id=HFVDYNX5CJHVR743LMTVASKPKCEESO6GRAV3NRPDUN2OZ3P4ORPQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul 9 00:38:14.921294145 (375h26m41.84758882s) [noise=true , address=77.90.31.11:4553 , piece_id=V77ZVDEM2MDH53UYS6PSMMYD2SNLDD4BEA3R3DSG6AFYPMDMG2BQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul 9 00:38:18.659642786 (375h26m45.585937465s) [noise=true , address=104.36.82.16:4293 , piece_id=WY5S2ADFULP3ZIF5LIKSTVJCPEM3CTG3DQQRLXHBMDXB634Y2THA, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul 9 00:38:44.764948400 (375h27m11.691243084s) [noise=true , address=104.36.82.6:4874 , piece_id=J5ZN2VIWDNZY4SZ3GGZXJCHFEEMLPEDVT6443SEITLPHMHRWKG4A, seg_pos={0 0}, upload=300678]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:104.36.82.6:4874): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:38:44.765367526 (375h27m11.691662219s) [noise=true , address=104.36.82.8:4969 , piece_id=ONQRKW57EBGOMVH32QVYYAYHVD6HTLQYII2JKSKGM6SAAYMQ6JPQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 159.223.143.39:50836->104.36.82.8:4969: use of closed network connection; write tcp 159.223.143.39:50836->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50836->104.36.82.8:4969: use of closed network connection
Jul 9 00:38:44.765546875 (375h27m11.691841563s) [noise=true , address=104.36.82.11:4113 , piece_id=75XEEPUONKKHCRZJQLRRL3RJMSILZBWJ6HRVZBNJAHWCC4PQ3YVA, seg_pos={0 0}, upload=300678]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:104.36.82.11:4113): protocol: write tcp 159.223.143.39:53580->104.36.82.11:4113: use of closed network connection; write tcp 159.223.143.39:53580->104.36.82.11:4113: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:53580->104.36.82.11:4113: use of closed network connection
Jul 9 00:38:44.765679043 (375h27m11.691973731s) [noise=true , address=104.36.82.4:4803 , piece_id=5R3JBPKYXDOLR7Q4KZ773Y5HFK2KZYYVXGI4YUM4A5WUZ5QDPEVQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:104.36.82.4:4803): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:38:44.765727083 (375h27m11.692021781s) [seg_pos={0 0}, upload=300678]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Jul 9 00:38:44.796212501 (375h27m11.722507185s) [upload=300678]: Done uploading segments...
Jul 9 00:38:44.796240113 (375h27m11.722534802s) [upload=300678]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 22.046587155s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
11 out of 60 is failed (threshold=3) [X........X.......X...X................X...........XX...X.XXX] duration:
1m0.001184029s
FAIL (2025-07-09 00:39:32 UTC)
context deadline exceeded
Duration: 1m0.001211349s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
13 out of 60 is failed (threshold=3) [..X.XX.X.......X...X......X.......X...X...X...........XX...X] duration:
30.000985034s
FAIL (2025-07-09 00:39:26 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 30.001004036s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [......................................................X....X] duration:
30.033060493s
FAIL (2025-07-09 00:38:44 UTC)
orbiter warning:
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
tion; conn closed: op; piecestore: piecestore close: context canceled
Jul 9 00:37:52.848582652 (375h26m19.774877332s) [seg_pos={0 0}, upload=300672]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 00:37:52.848656023 (375h26m19.774950766s) [upload=300672]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 00:37:52.848661061 (375h26m19.774955738s) [upload=300672]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 00:37:53.034475703 (375h26m19.960770383s) [upload=300672]: Done uploading segments...
Jul 9 00:37:53.034504712 (375h26m19.960799388s) [upload=300672]: Upload finished. err: <nil>
Jul 9 00:38:14.795156836 (375h26m41.721451522s) [upload=300678]: Starting upload
Jul 9 00:38:14.795233303 (375h26m41.721527988s) [upload=300678]: Done starting upload
Jul 9 00:38:14.795335298 (375h26m41.721629977s) [upload=300678]: Uploading segments...
Jul 9 00:38:14.795594071 (375h26m41.721888765s) [upload=300678]: Got next segment. Inline: false
Jul 9 00:38:14.795674023 (375h26m41.721968717s) [upload=300678]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 00:38:14.795689186 (375h26m41.721983879s) [upload=300678]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 00:38:14.823801892 (375h26m41.750096591s) [seg_pos={0 0}, upload=300678]: Begin upload segment...
Jul 9 00:38:14.823863472 (375h26m41.75015817s) [seg_pos={0 0}, upload=300678]: Done begin upload segment.
Jul 9 00:38:14.823896674 (375h26m41.750191376s) [noise=true , address=104.36.82.8:4969 , piece_id=ONQRKW57EBGOMVH32QVYYAYHVD6HTLQYII2JKSKGM6SAAYMQ6JPQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.823907419 (375h26m41.750202111s) [noise=true , address=77.90.31.11:4553 , piece_id=V77ZVDEM2MDH53UYS6PSMMYD2SNLDD4BEA3R3DSG6AFYPMDMG2BQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.824421129 (375h26m41.750715827s) [noise=true , address=45.91.116.8:4045 , piece_id=GZPXFXFCJMYI3RIZF5KWZI5R3VRIYGUDXC6PS4XZI5WB27JZKVXQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.824849933 (375h26m41.751144628s) [noise=true , address=45.91.116.6:4975 , piece_id=CPPBPWYULER3ZOZRD2JXYJOKLAULE3YCV6ZEONQLDUHOXRUPTIAQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.824923234 (375h26m41.751217926s) [noise=true , address=77.90.31.16:4733 , piece_id=C2VQHR4WVBP7XSRLADN3NN6OCTO5ROEDC6ZM2XTDNNDMKMACR6DA, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.825248920 (375h26m41.751543617s) [noise=true , address=45.91.116.12:4226 , piece_id=QWG3A45EEBSNR2D4XYIBE3Y32SRK3AWS5QEFZMOVAWBZSC6EASQA, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.825280427 (375h26m41.751575123s) [noise=true , address=104.36.82.16:4293 , piece_id=WY5S2ADFULP3ZIF5LIKSTVJCPEM3CTG3DQQRLXHBMDXB634Y2THA, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.825569855 (375h26m41.751864541s) [noise=true , address=104.36.82.11:4113 , piece_id=75XEEPUONKKHCRZJQLRRL3RJMSILZBWJ6HRVZBNJAHWCC4PQ3YVA, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.825642542 (375h26m41.751937239s) [noise=true , address=104.36.82.6:4874 , piece_id=J5ZN2VIWDNZY4SZ3GGZXJCHFEEMLPEDVT6443SEITLPHMHRWKG4A, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.825906508 (375h26m41.752201197s) [noise=true , address=45.91.116.14:4191 , piece_id=FTHEMV3GWZNLUMSIIMMQ5IRQZPFJ6F4RVGXM2XVSVRH2KWLTMZZA, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.826326502 (375h26m41.752621198s) [noise=true , address=45.91.116.8:4047 , piece_id=TDXBCU3XF32U6LCCRHR4YAKYWOOY4NMTLULNTLABNTRIBT2PBL6Q, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.826792611 (375h26m41.753087303s) [noise=true , address=45.91.116.10:4083 , piece_id=HFVDYNX5CJHVR743LMTVASKPKCEESO6GRAV3NRPDUN2OZ3P4ORPQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.827187184 (375h26m41.75348188s) [noise=true , address=45.91.116.11:4117 , piece_id=QWTZM6HFPAL3AX474MBFHNVUZNIASMPACIXSVLNANDVLVF7HG3QQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.827594318 (375h26m41.753889018s) [noise=true , address=45.91.116.5:4299 , piece_id=QWABO26BIHTA7HLC6XMH5IL4QC4JZTEBOP2QWIG3ROZHX2PRVKXA, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.827842560 (375h26m41.754137255s) [noise=true , address=77.90.31.19:4843 , piece_id=WW275T6GT6JR5WFTYMAFXP5G5YZJXYMKFU772I5NELO3V4YEHWQQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.828205534 (375h26m41.754500223s) [noise=true , address=104.36.82.4:4803 , piece_id=5R3JBPKYXDOLR7Q4KZ773Y5HFK2KZYYVXGI4YUM4A5WUZ5QDPEVQ, seg_pos={0 0}, upload=300678]: Uploading piece...
Jul 9 00:38:14.915111848 (375h26m41.841406522s) [upload=300678]: Next returned nil segment
Jul 9 00:38:14.915126722 (375h26m41.841421407s) [upload=300678]: Waiting for error group managing segments...
Jul 9 00:38:14.918322650 (375h26m41.84461734s) [noise=true , address=45.91.116.12:4226 , piece_id=QWG3A45EEBSNR2D4XYIBE3Y32SRK3AWS5QEFZMOVAWBZSC6EASQA, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul 9 00:38:14.918983217 (375h26m41.845277898s) [noise=true , address=77.90.31.16:4733 , piece_id=C2VQHR4WVBP7XSRLADN3NN6OCTO5ROEDC6ZM2XTDNNDMKMACR6DA, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul 9 00:38:14.919620494 (375h26m41.845915182s) [noise=true , address=45.91.116.14:4191 , piece_id=FTHEMV3GWZNLUMSIIMMQ5IRQZPFJ6F4RVGXM2XVSVRH2KWLTMZZA, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul 9 00:38:14.919706240 (375h26m41.846000916s) [noise=true , address=45.91.116.11:4117 , piece_id=QWTZM6HFPAL3AX474MBFHNVUZNIASMPACIXSVLNANDVLVF7HG3QQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul 9 00:38:14.920089094 (375h26m41.846383766s) [noise=true , address=45.91.116.6:4975 , piece_id=CPPBPWYULER3ZOZRD2JXYJOKLAULE3YCV6ZEONQLDUHOXRUPTIAQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul 9 00:38:14.920215430 (375h26m41.846510107s) [noise=true , address=45.91.116.8:4045 , piece_id=GZPXFXFCJMYI3RIZF5KWZI5R3VRIYGUDXC6PS4XZI5WB27JZKVXQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul 9 00:38:14.920519778 (375h26m41.846814455s) [noise=true , address=45.91.116.8:4047 , piece_id=TDXBCU3XF32U6LCCRHR4YAKYWOOY4NMTLULNTLABNTRIBT2PBL6Q, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul 9 00:38:14.920715325 (375h26m41.84701s) [noise=true , address=45.91.116.5:4299 , piece_id=QWABO26BIHTA7HLC6XMH5IL4QC4JZTEBOP2QWIG3ROZHX2PRVKXA, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul 9 00:38:14.920921360 (375h26m41.847216035s) [noise=true , address=77.90.31.19:4843 , piece_id=WW275T6GT6JR5WFTYMAFXP5G5YZJXYMKFU772I5NELO3V4YEHWQQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul 9 00:38:14.921284011 (375h26m41.847578698s) [noise=true , address=45.91.116.10:4083 , piece_id=HFVDYNX5CJHVR743LMTVASKPKCEESO6GRAV3NRPDUN2OZ3P4ORPQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul 9 00:38:14.921294145 (375h26m41.84758882s) [noise=true , address=77.90.31.11:4553 , piece_id=V77ZVDEM2MDH53UYS6PSMMYD2SNLDD4BEA3R3DSG6AFYPMDMG2BQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul 9 00:38:18.659642786 (375h26m45.585937465s) [noise=true , address=104.36.82.16:4293 , piece_id=WY5S2ADFULP3ZIF5LIKSTVJCPEM3CTG3DQQRLXHBMDXB634Y2THA, seg_pos={0 0}, upload=300678]: Done uploading piece. err: <nil>
Jul 9 00:38:44.764948400 (375h27m11.691243084s) [noise=true , address=104.36.82.6:4874 , piece_id=J5ZN2VIWDNZY4SZ3GGZXJCHFEEMLPEDVT6443SEITLPHMHRWKG4A, seg_pos={0 0}, upload=300678]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:104.36.82.6:4874): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:38:44.765367526 (375h27m11.691662219s) [noise=true , address=104.36.82.8:4969 , piece_id=ONQRKW57EBGOMVH32QVYYAYHVD6HTLQYII2JKSKGM6SAAYMQ6JPQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 159.223.143.39:50836->104.36.82.8:4969: use of closed network connection; write tcp 159.223.143.39:50836->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:50836->104.36.82.8:4969: use of closed network connection
Jul 9 00:38:44.765546875 (375h27m11.691841563s) [noise=true , address=104.36.82.11:4113 , piece_id=75XEEPUONKKHCRZJQLRRL3RJMSILZBWJ6HRVZBNJAHWCC4PQ3YVA, seg_pos={0 0}, upload=300678]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:104.36.82.11:4113): protocol: write tcp 159.223.143.39:53580->104.36.82.11:4113: use of closed network connection; write tcp 159.223.143.39:53580->104.36.82.11:4113: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:53580->104.36.82.11:4113: use of closed network connection
Jul 9 00:38:44.765679043 (375h27m11.691973731s) [noise=true , address=104.36.82.4:4803 , piece_id=5R3JBPKYXDOLR7Q4KZ773Y5HFK2KZYYVXGI4YUM4A5WUZ5QDPEVQ, seg_pos={0 0}, upload=300678]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:104.36.82.4:4803): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:38:44.765727083 (375h27m11.692021781s) [seg_pos={0 0}, upload=300678]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Jul 9 00:38:44.796212501 (375h27m11.722507185s) [upload=300678]: Done uploading segments...
Jul 9 00:38:44.796240113 (375h27m11.722534802s) [upload=300678]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 30.033088428s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
13 out of 60 is failed (threshold=3) [X..X.XX.X.......X...X......X.......X...X...X...........XX...] duration:
13.5052047s
FAIL (2025-07-09 00:38:37 UTC)
orbiter warning: context deadline exceeded
Duration: 13.505242173s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
11 out of 60 is failed (threshold=3) [XX........X.......X...X................X...........XX...X.XX] duration:
1m0.000672503s
FAIL (2025-07-09 00:38:32 UTC)
operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1m0.000684958s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....] duration:
557.098691ms
FAIL (2025-07-09 00:37:53 UTC)
orbiter warning:
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: expected piece hash; context canceled; EOF
Jul 9 00:34:30.136902627 (375h22m57.063197324s) [seg_pos={0 0}, upload=300627]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 00:34:30.137026593 (375h22m57.06332128s) [upload=300627]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 00:34:30.137033272 (375h22m57.063327966s) [upload=300627]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 00:34:30.239293314 (375h22m57.165588013s) [upload=300627]: Done uploading segments...
Jul 9 00:34:30.239339271 (375h22m57.165633956s) [upload=300627]: Upload finished. err: <nil>
Jul 9 00:35:31.142705883 (375h23m58.069000569s) [upload=300640]: Starting upload
Jul 9 00:35:31.142808419 (375h23m58.069103108s) [upload=300640]: Done starting upload
Jul 9 00:35:31.142933499 (375h23m58.069228187s) [upload=300640]: Uploading segments...
Jul 9 00:35:31.143428918 (375h23m58.069723604s) [upload=300640]: Got next segment. Inline: false
Jul 9 00:35:31.143492378 (375h23m58.069787058s) [upload=300640]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 00:35:31.143498141 (375h23m58.069792822s) [upload=300640]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 00:35:31.172809934 (375h23m58.099104631s) [seg_pos={0 0}, upload=300640]: Begin upload segment...
Jul 9 00:35:31.172884798 (375h23m58.099179493s) [seg_pos={0 0}, upload=300640]: Done begin upload segment.
Jul 9 00:35:31.172918708 (375h23m58.0992134s) [noise=true , address=45.91.116.12:4227 , piece_id=YRM7X5XX6RUDWN66E2EOLKYCJW7SUX7YV6JPFF37YAJ6I2XG3R6A, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul 9 00:35:31.172934200 (375h23m58.099228892s) [noise=true , address=104.36.82.7:4910 , piece_id=267AUB3PQ2QAHP3HNVNHCCWDLGYST7WSTXTWNFOBUR56QRUJLXVA, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul 9 00:35:31.173374012 (375h23m58.099668707s) [noise=true , address=104.36.82.15:4257 , piece_id=TCLY7VDUEWNDDCZBA5HO6GDDOOGI22BITL6VVRETPI74G2T46RRA, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul 9 00:35:31.173385067 (375h23m58.099679757s) [noise=true , address=77.90.31.18:4806 , piece_id=6BPPUMQ7JQHH5XPB7UJT4UZQJF4F46PXEBRGIWLSVHDZGHRKDYLA, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul 9 00:35:31.173803824 (375h23m58.10009852s) [noise=true , address=45.91.116.4:4940 , piece_id=RLRQW743C2VJA4OXU6KHUO3546XMDV57EZOCK3YZUUORMT64TAJA, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul 9 00:35:31.173904484 (375h23m58.10019918s) [noise=true , address=45.91.116.10:4081 , piece_id=WCOKGYR2PSNRUPDKYHIWENDISBO53V3XZNB7W4V3QCXJX54UYPEQ, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul 9 00:35:31.174218964 (375h23m58.100513662s) [noise=true , address=45.91.116.8:4045 , piece_id=2SSDQFSIUS5P44VYYESMSIV3B33FK3GTNLFJ7BLETS4ZBUBX23NQ, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul 9 00:35:31.174461657 (375h23m58.100756346s) [noise=true , address=104.36.82.4:4802 , piece_id=66M7CRNO4WP5BZZ3CPK2RCPD3BHJW4RWVEK3XJJA7O7ZBTKJHJBQ, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul 9 00:35:31.174753443 (375h23m58.101048139s) [noise=true , address=77.90.31.12:4590 , piece_id=QPKATZ4XR5G2BDO3SMTY4YPQ4VI27M2E4HCIW3ATEEMOFYUWQ6IQ, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul 9 00:35:31.175058813 (375h23m58.101353622s) [noise=true , address=104.36.82.12:4150 , piece_id=YMNCM2R7X6JG2E4BG5KJ7O7MJC6XWE5IVY6DJ67ZKIWTIHDMIMHQ, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul 9 00:35:31.175146907 (375h23m58.101441596s) [noise=true , address=45.91.116.9:4263 , piece_id=4E3DU6HHBXFQBVZSNSVEIAODSSYZDTZRZ3Y7RPFL4LII7PWRBXFA, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul 9 00:35:31.175470130 (375h23m58.101764827s) [noise=true , address=45.91.116.11:4117 , piece_id=CVPAQT4D65AM6RBUXEFHYAXPZA7AQAG4JWWAMWBDXW3HAV3SQYKQ, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul 9 00:35:31.175522889 (375h23m58.101817587s) [noise=true , address=77.90.31.10:4520 , piece_id=44MZGFISHSDAWO6TU3KWH6K6ZQJ4YPB7A6CKCSXS4B76ROLULA7A, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul 9 00:35:31.175804424 (375h23m58.102099113s) [noise=true , address=104.36.82.10:4077 , piece_id=55HERBR2GAOGJJSGUPEFJPWQM5WIL3ENISEBUCFMZZNDJFDQ5SIA, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul 9 00:35:31.176052223 (375h23m58.102346908s) [noise=true , address=104.36.82.15:4258 , piece_id=WMT5VC4QNRQ2TVBMBXSTUSA2C42DGEMPCCQFFDBS6PLMJKFUI5KA, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul 9 00:35:31.176479692 (375h23m58.102774378s) [noise=true , address=104.36.82.14:4223 , piece_id=7ZRAYI2W4WGAYCLPOXPXSAMITQZB4LWGGGLI4E64BYRZG7RV4RSA, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul 9 00:35:31.266381427 (375h23m58.192676104s) [upload=300640]: Next returned nil segment
Jul 9 00:35:31.266395009 (375h23m58.192689686s) [upload=300640]: Waiting for error group managing segments...
Jul 9 00:35:31.270056616 (375h23m58.19635129s) [noise=true , address=77.90.31.10:4520 , piece_id=44MZGFISHSDAWO6TU3KWH6K6ZQJ4YPB7A6CKCSXS4B76ROLULA7A, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul 9 00:35:31.270776078 (375h23m58.197070768s) [noise=true , address=77.90.31.18:4806 , piece_id=6BPPUMQ7JQHH5XPB7UJT4UZQJF4F46PXEBRGIWLSVHDZGHRKDYLA, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul 9 00:35:31.271385196 (375h23m58.197679869s) [noise=true , address=45.91.116.10:4081 , piece_id=WCOKGYR2PSNRUPDKYHIWENDISBO53V3XZNB7W4V3QCXJX54UYPEQ, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul 9 00:35:31.271606981 (375h23m58.197901662s) [noise=true , address=45.91.116.8:4045 , piece_id=2SSDQFSIUS5P44VYYESMSIV3B33FK3GTNLFJ7BLETS4ZBUBX23NQ, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul 9 00:35:31.271957840 (375h23m58.198252513s) [noise=true , address=45.91.116.12:4227 , piece_id=YRM7X5XX6RUDWN66E2EOLKYCJW7SUX7YV6JPFF37YAJ6I2XG3R6A, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul 9 00:35:31.272530035 (375h23m58.198824725s) [noise=true , address=45.91.116.9:4263 , piece_id=4E3DU6HHBXFQBVZSNSVEIAODSSYZDTZRZ3Y7RPFL4LII7PWRBXFA, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul 9 00:35:31.272809654 (375h23m58.199104333s) [noise=true , address=45.91.116.4:4940 , piece_id=RLRQW743C2VJA4OXU6KHUO3546XMDV57EZOCK3YZUUORMT64TAJA, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul 9 00:35:31.275138939 (375h23m58.201433617s) [noise=true , address=77.90.31.12:4590 , piece_id=QPKATZ4XR5G2BDO3SMTY4YPQ4VI27M2E4HCIW3ATEEMOFYUWQ6IQ, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul 9 00:35:31.585358643 (375h23m58.511653329s) [noise=true , address=45.91.116.11:4117 , piece_id=CVPAQT4D65AM6RBUXEFHYAXPZA7AQAG4JWWAMWBDXW3HAV3SQYKQ, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul 9 00:35:52.939705862 (375h24m19.866000541s) [noise=true , address=104.36.82.7:4910 , piece_id=267AUB3PQ2QAHP3HNVNHCCWDLGYST7WSTXTWNFOBUR56QRUJLXVA, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul 9 00:35:59.619210529 (375h24m26.545505204s) [noise=true , address=104.36.82.12:4150 , piece_id=YMNCM2R7X6JG2E4BG5KJ7O7MJC6XWE5IVY6DJ67ZKIWTIHDMIMHQ, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul 9 00:36:01.109106257 (375h24m28.035400951s) [noise=true , address=104.36.82.10:4077 , piece_id=55HERBR2GAOGJJSGUPEFJPWQM5WIL3ENISEBUCFMZZNDJFDQ5SIA, seg_pos={0 0}, upload=300640]: Done uploading piece. err: ecclient: upload failed (node:12kDJuQP8iLUjB1rXpcNX5TtHzsqa5HvYcUD9Ef64HYTnTXcwLk, address:104.36.82.10:4077): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:36:01.109138267 (375h24m28.035433079s) [noise=true , address=104.36.82.15:4258 , piece_id=WMT5VC4QNRQ2TVBMBXSTUSA2C42DGEMPCCQFFDBS6PLMJKFUI5KA, seg_pos={0 0}, upload=300640]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:104.36.82.15:4258): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:36:01.109262427 (375h24m28.035557124s) [noise=true , address=104.36.82.14:4223 , piece_id=7ZRAYI2W4WGAYCLPOXPXSAMITQZB4LWGGGLI4E64BYRZG7RV4RSA, seg_pos={0 0}, upload=300640]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:36:01.109285927 (375h24m28.035580609s) [noise=true , address=104.36.82.4:4802 , piece_id=66M7CRNO4WP5BZZ3CPK2RCPD3BHJW4RWVEK3XJJA7O7ZBTKJHJBQ, seg_pos={0 0}, upload=300640]: Done uploading piece. err: ecclient: upload failed (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4, address:104.36.82.4:4802): protocol: write tcp 159.223.143.39:54458->104.36.82.4:4802: use of closed network connection; write tcp 159.223.143.39:54458->104.36.82.4:4802: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:54458->104.36.82.4:4802: use of closed network connection
Jul 9 00:36:01.109358127 (375h24m28.03565291s) [noise=true , address=104.36.82.15:4257 , piece_id=TCLY7VDUEWNDDCZBA5HO6GDDOOGI22BITL6VVRETPI74G2T46RRA, seg_pos={0 0}, upload=300640]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:104.36.82.15:4257): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:36:01.109390282 (375h24m28.035685081s) [seg_pos={0 0}, upload=300640]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Jul 9 00:36:01.146868060 (375h24m28.07316275s) [upload=300640]: Done uploading segments...
Jul 9 00:36:01.146918616 (375h24m28.073213304s) [upload=300640]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 557.132998ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
10 out of 60 is failed (threshold=3) [.XX........X.......X...X................X...........XX...X.X] duration:
1m0.00091938s
FAIL (2025-07-09 00:37:32 UTC)
operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1m0.000941631s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...] duration:
867.892022ms
FAIL (2025-07-09 00:37:31 UTC)
orbiter warning:
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: expected piece hash; context canceled; EOF
Jul 9 00:34:30.136902627 (375h22m57.063197324s) [seg_pos={0 0}, upload=300627]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 00:34:30.137026593 (375h22m57.06332128s) [upload=300627]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 00:34:30.137033272 (375h22m57.063327966s) [upload=300627]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 00:34:30.239293314 (375h22m57.165588013s) [upload=300627]: Done uploading segments...
Jul 9 00:34:30.239339271 (375h22m57.165633956s) [upload=300627]: Upload finished. err: <nil>
Jul 9 00:35:31.142705883 (375h23m58.069000569s) [upload=300640]: Starting upload
Jul 9 00:35:31.142808419 (375h23m58.069103108s) [upload=300640]: Done starting upload
Jul 9 00:35:31.142933499 (375h23m58.069228187s) [upload=300640]: Uploading segments...
Jul 9 00:35:31.143428918 (375h23m58.069723604s) [upload=300640]: Got next segment. Inline: false
Jul 9 00:35:31.143492378 (375h23m58.069787058s) [upload=300640]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 00:35:31.143498141 (375h23m58.069792822s) [upload=300640]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 00:35:31.172809934 (375h23m58.099104631s) [seg_pos={0 0}, upload=300640]: Begin upload segment...
Jul 9 00:35:31.172884798 (375h23m58.099179493s) [seg_pos={0 0}, upload=300640]: Done begin upload segment.
Jul 9 00:35:31.172918708 (375h23m58.0992134s) [noise=true , address=45.91.116.12:4227 , piece_id=YRM7X5XX6RUDWN66E2EOLKYCJW7SUX7YV6JPFF37YAJ6I2XG3R6A, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul 9 00:35:31.172934200 (375h23m58.099228892s) [noise=true , address=104.36.82.7:4910 , piece_id=267AUB3PQ2QAHP3HNVNHCCWDLGYST7WSTXTWNFOBUR56QRUJLXVA, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul 9 00:35:31.173374012 (375h23m58.099668707s) [noise=true , address=104.36.82.15:4257 , piece_id=TCLY7VDUEWNDDCZBA5HO6GDDOOGI22BITL6VVRETPI74G2T46RRA, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul 9 00:35:31.173385067 (375h23m58.099679757s) [noise=true , address=77.90.31.18:4806 , piece_id=6BPPUMQ7JQHH5XPB7UJT4UZQJF4F46PXEBRGIWLSVHDZGHRKDYLA, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul 9 00:35:31.173803824 (375h23m58.10009852s) [noise=true , address=45.91.116.4:4940 , piece_id=RLRQW743C2VJA4OXU6KHUO3546XMDV57EZOCK3YZUUORMT64TAJA, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul 9 00:35:31.173904484 (375h23m58.10019918s) [noise=true , address=45.91.116.10:4081 , piece_id=WCOKGYR2PSNRUPDKYHIWENDISBO53V3XZNB7W4V3QCXJX54UYPEQ, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul 9 00:35:31.174218964 (375h23m58.100513662s) [noise=true , address=45.91.116.8:4045 , piece_id=2SSDQFSIUS5P44VYYESMSIV3B33FK3GTNLFJ7BLETS4ZBUBX23NQ, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul 9 00:35:31.174461657 (375h23m58.100756346s) [noise=true , address=104.36.82.4:4802 , piece_id=66M7CRNO4WP5BZZ3CPK2RCPD3BHJW4RWVEK3XJJA7O7ZBTKJHJBQ, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul 9 00:35:31.174753443 (375h23m58.101048139s) [noise=true , address=77.90.31.12:4590 , piece_id=QPKATZ4XR5G2BDO3SMTY4YPQ4VI27M2E4HCIW3ATEEMOFYUWQ6IQ, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul 9 00:35:31.175058813 (375h23m58.101353622s) [noise=true , address=104.36.82.12:4150 , piece_id=YMNCM2R7X6JG2E4BG5KJ7O7MJC6XWE5IVY6DJ67ZKIWTIHDMIMHQ, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul 9 00:35:31.175146907 (375h23m58.101441596s) [noise=true , address=45.91.116.9:4263 , piece_id=4E3DU6HHBXFQBVZSNSVEIAODSSYZDTZRZ3Y7RPFL4LII7PWRBXFA, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul 9 00:35:31.175470130 (375h23m58.101764827s) [noise=true , address=45.91.116.11:4117 , piece_id=CVPAQT4D65AM6RBUXEFHYAXPZA7AQAG4JWWAMWBDXW3HAV3SQYKQ, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul 9 00:35:31.175522889 (375h23m58.101817587s) [noise=true , address=77.90.31.10:4520 , piece_id=44MZGFISHSDAWO6TU3KWH6K6ZQJ4YPB7A6CKCSXS4B76ROLULA7A, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul 9 00:35:31.175804424 (375h23m58.102099113s) [noise=true , address=104.36.82.10:4077 , piece_id=55HERBR2GAOGJJSGUPEFJPWQM5WIL3ENISEBUCFMZZNDJFDQ5SIA, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul 9 00:35:31.176052223 (375h23m58.102346908s) [noise=true , address=104.36.82.15:4258 , piece_id=WMT5VC4QNRQ2TVBMBXSTUSA2C42DGEMPCCQFFDBS6PLMJKFUI5KA, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul 9 00:35:31.176479692 (375h23m58.102774378s) [noise=true , address=104.36.82.14:4223 , piece_id=7ZRAYI2W4WGAYCLPOXPXSAMITQZB4LWGGGLI4E64BYRZG7RV4RSA, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul 9 00:35:31.266381427 (375h23m58.192676104s) [upload=300640]: Next returned nil segment
Jul 9 00:35:31.266395009 (375h23m58.192689686s) [upload=300640]: Waiting for error group managing segments...
Jul 9 00:35:31.270056616 (375h23m58.19635129s) [noise=true , address=77.90.31.10:4520 , piece_id=44MZGFISHSDAWO6TU3KWH6K6ZQJ4YPB7A6CKCSXS4B76ROLULA7A, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul 9 00:35:31.270776078 (375h23m58.197070768s) [noise=true , address=77.90.31.18:4806 , piece_id=6BPPUMQ7JQHH5XPB7UJT4UZQJF4F46PXEBRGIWLSVHDZGHRKDYLA, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul 9 00:35:31.271385196 (375h23m58.197679869s) [noise=true , address=45.91.116.10:4081 , piece_id=WCOKGYR2PSNRUPDKYHIWENDISBO53V3XZNB7W4V3QCXJX54UYPEQ, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul 9 00:35:31.271606981 (375h23m58.197901662s) [noise=true , address=45.91.116.8:4045 , piece_id=2SSDQFSIUS5P44VYYESMSIV3B33FK3GTNLFJ7BLETS4ZBUBX23NQ, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul 9 00:35:31.271957840 (375h23m58.198252513s) [noise=true , address=45.91.116.12:4227 , piece_id=YRM7X5XX6RUDWN66E2EOLKYCJW7SUX7YV6JPFF37YAJ6I2XG3R6A, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul 9 00:35:31.272530035 (375h23m58.198824725s) [noise=true , address=45.91.116.9:4263 , piece_id=4E3DU6HHBXFQBVZSNSVEIAODSSYZDTZRZ3Y7RPFL4LII7PWRBXFA, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul 9 00:35:31.272809654 (375h23m58.199104333s) [noise=true , address=45.91.116.4:4940 , piece_id=RLRQW743C2VJA4OXU6KHUO3546XMDV57EZOCK3YZUUORMT64TAJA, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul 9 00:35:31.275138939 (375h23m58.201433617s) [noise=true , address=77.90.31.12:4590 , piece_id=QPKATZ4XR5G2BDO3SMTY4YPQ4VI27M2E4HCIW3ATEEMOFYUWQ6IQ, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul 9 00:35:31.585358643 (375h23m58.511653329s) [noise=true , address=45.91.116.11:4117 , piece_id=CVPAQT4D65AM6RBUXEFHYAXPZA7AQAG4JWWAMWBDXW3HAV3SQYKQ, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul 9 00:35:52.939705862 (375h24m19.866000541s) [noise=true , address=104.36.82.7:4910 , piece_id=267AUB3PQ2QAHP3HNVNHCCWDLGYST7WSTXTWNFOBUR56QRUJLXVA, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul 9 00:35:59.619210529 (375h24m26.545505204s) [noise=true , address=104.36.82.12:4150 , piece_id=YMNCM2R7X6JG2E4BG5KJ7O7MJC6XWE5IVY6DJ67ZKIWTIHDMIMHQ, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul 9 00:36:01.109106257 (375h24m28.035400951s) [noise=true , address=104.36.82.10:4077 , piece_id=55HERBR2GAOGJJSGUPEFJPWQM5WIL3ENISEBUCFMZZNDJFDQ5SIA, seg_pos={0 0}, upload=300640]: Done uploading piece. err: ecclient: upload failed (node:12kDJuQP8iLUjB1rXpcNX5TtHzsqa5HvYcUD9Ef64HYTnTXcwLk, address:104.36.82.10:4077): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:36:01.109138267 (375h24m28.035433079s) [noise=true , address=104.36.82.15:4258 , piece_id=WMT5VC4QNRQ2TVBMBXSTUSA2C42DGEMPCCQFFDBS6PLMJKFUI5KA, seg_pos={0 0}, upload=300640]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:104.36.82.15:4258): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:36:01.109262427 (375h24m28.035557124s) [noise=true , address=104.36.82.14:4223 , piece_id=7ZRAYI2W4WGAYCLPOXPXSAMITQZB4LWGGGLI4E64BYRZG7RV4RSA, seg_pos={0 0}, upload=300640]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:36:01.109285927 (375h24m28.035580609s) [noise=true , address=104.36.82.4:4802 , piece_id=66M7CRNO4WP5BZZ3CPK2RCPD3BHJW4RWVEK3XJJA7O7ZBTKJHJBQ, seg_pos={0 0}, upload=300640]: Done uploading piece. err: ecclient: upload failed (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4, address:104.36.82.4:4802): protocol: write tcp 159.223.143.39:54458->104.36.82.4:4802: use of closed network connection; write tcp 159.223.143.39:54458->104.36.82.4:4802: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:54458->104.36.82.4:4802: use of closed network connection
Jul 9 00:36:01.109358127 (375h24m28.03565291s) [noise=true , address=104.36.82.15:4257 , piece_id=TCLY7VDUEWNDDCZBA5HO6GDDOOGI22BITL6VVRETPI74G2T46RRA, seg_pos={0 0}, upload=300640]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:104.36.82.15:4257): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:36:01.109390282 (375h24m28.035685081s) [seg_pos={0 0}, upload=300640]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Jul 9 00:36:01.146868060 (375h24m28.07316275s) [upload=300640]: Done uploading segments...
Jul 9 00:36:01.146918616 (375h24m28.073213304s) [upload=300640]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 867.918315ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..] duration:
11.502681916s
FAIL (2025-07-09 00:37:18 UTC)
orbiter warning:
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: expected piece hash; context canceled; EOF
Jul 9 00:34:30.136902627 (375h22m57.063197324s) [seg_pos={0 0}, upload=300627]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 00:34:30.137026593 (375h22m57.06332128s) [upload=300627]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 00:34:30.137033272 (375h22m57.063327966s) [upload=300627]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 00:34:30.239293314 (375h22m57.165588013s) [upload=300627]: Done uploading segments...
Jul 9 00:34:30.239339271 (375h22m57.165633956s) [upload=300627]: Upload finished. err: <nil>
Jul 9 00:35:31.142705883 (375h23m58.069000569s) [upload=300640]: Starting upload
Jul 9 00:35:31.142808419 (375h23m58.069103108s) [upload=300640]: Done starting upload
Jul 9 00:35:31.142933499 (375h23m58.069228187s) [upload=300640]: Uploading segments...
Jul 9 00:35:31.143428918 (375h23m58.069723604s) [upload=300640]: Got next segment. Inline: false
Jul 9 00:35:31.143492378 (375h23m58.069787058s) [upload=300640]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 00:35:31.143498141 (375h23m58.069792822s) [upload=300640]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 00:35:31.172809934 (375h23m58.099104631s) [seg_pos={0 0}, upload=300640]: Begin upload segment...
Jul 9 00:35:31.172884798 (375h23m58.099179493s) [seg_pos={0 0}, upload=300640]: Done begin upload segment.
Jul 9 00:35:31.172918708 (375h23m58.0992134s) [noise=true , address=45.91.116.12:4227 , piece_id=YRM7X5XX6RUDWN66E2EOLKYCJW7SUX7YV6JPFF37YAJ6I2XG3R6A, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul 9 00:35:31.172934200 (375h23m58.099228892s) [noise=true , address=104.36.82.7:4910 , piece_id=267AUB3PQ2QAHP3HNVNHCCWDLGYST7WSTXTWNFOBUR56QRUJLXVA, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul 9 00:35:31.173374012 (375h23m58.099668707s) [noise=true , address=104.36.82.15:4257 , piece_id=TCLY7VDUEWNDDCZBA5HO6GDDOOGI22BITL6VVRETPI74G2T46RRA, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul 9 00:35:31.173385067 (375h23m58.099679757s) [noise=true , address=77.90.31.18:4806 , piece_id=6BPPUMQ7JQHH5XPB7UJT4UZQJF4F46PXEBRGIWLSVHDZGHRKDYLA, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul 9 00:35:31.173803824 (375h23m58.10009852s) [noise=true , address=45.91.116.4:4940 , piece_id=RLRQW743C2VJA4OXU6KHUO3546XMDV57EZOCK3YZUUORMT64TAJA, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul 9 00:35:31.173904484 (375h23m58.10019918s) [noise=true , address=45.91.116.10:4081 , piece_id=WCOKGYR2PSNRUPDKYHIWENDISBO53V3XZNB7W4V3QCXJX54UYPEQ, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul 9 00:35:31.174218964 (375h23m58.100513662s) [noise=true , address=45.91.116.8:4045 , piece_id=2SSDQFSIUS5P44VYYESMSIV3B33FK3GTNLFJ7BLETS4ZBUBX23NQ, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul 9 00:35:31.174461657 (375h23m58.100756346s) [noise=true , address=104.36.82.4:4802 , piece_id=66M7CRNO4WP5BZZ3CPK2RCPD3BHJW4RWVEK3XJJA7O7ZBTKJHJBQ, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul 9 00:35:31.174753443 (375h23m58.101048139s) [noise=true , address=77.90.31.12:4590 , piece_id=QPKATZ4XR5G2BDO3SMTY4YPQ4VI27M2E4HCIW3ATEEMOFYUWQ6IQ, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul 9 00:35:31.175058813 (375h23m58.101353622s) [noise=true , address=104.36.82.12:4150 , piece_id=YMNCM2R7X6JG2E4BG5KJ7O7MJC6XWE5IVY6DJ67ZKIWTIHDMIMHQ, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul 9 00:35:31.175146907 (375h23m58.101441596s) [noise=true , address=45.91.116.9:4263 , piece_id=4E3DU6HHBXFQBVZSNSVEIAODSSYZDTZRZ3Y7RPFL4LII7PWRBXFA, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul 9 00:35:31.175470130 (375h23m58.101764827s) [noise=true , address=45.91.116.11:4117 , piece_id=CVPAQT4D65AM6RBUXEFHYAXPZA7AQAG4JWWAMWBDXW3HAV3SQYKQ, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul 9 00:35:31.175522889 (375h23m58.101817587s) [noise=true , address=77.90.31.10:4520 , piece_id=44MZGFISHSDAWO6TU3KWH6K6ZQJ4YPB7A6CKCSXS4B76ROLULA7A, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul 9 00:35:31.175804424 (375h23m58.102099113s) [noise=true , address=104.36.82.10:4077 , piece_id=55HERBR2GAOGJJSGUPEFJPWQM5WIL3ENISEBUCFMZZNDJFDQ5SIA, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul 9 00:35:31.176052223 (375h23m58.102346908s) [noise=true , address=104.36.82.15:4258 , piece_id=WMT5VC4QNRQ2TVBMBXSTUSA2C42DGEMPCCQFFDBS6PLMJKFUI5KA, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul 9 00:35:31.176479692 (375h23m58.102774378s) [noise=true , address=104.36.82.14:4223 , piece_id=7ZRAYI2W4WGAYCLPOXPXSAMITQZB4LWGGGLI4E64BYRZG7RV4RSA, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul 9 00:35:31.266381427 (375h23m58.192676104s) [upload=300640]: Next returned nil segment
Jul 9 00:35:31.266395009 (375h23m58.192689686s) [upload=300640]: Waiting for error group managing segments...
Jul 9 00:35:31.270056616 (375h23m58.19635129s) [noise=true , address=77.90.31.10:4520 , piece_id=44MZGFISHSDAWO6TU3KWH6K6ZQJ4YPB7A6CKCSXS4B76ROLULA7A, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul 9 00:35:31.270776078 (375h23m58.197070768s) [noise=true , address=77.90.31.18:4806 , piece_id=6BPPUMQ7JQHH5XPB7UJT4UZQJF4F46PXEBRGIWLSVHDZGHRKDYLA, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul 9 00:35:31.271385196 (375h23m58.197679869s) [noise=true , address=45.91.116.10:4081 , piece_id=WCOKGYR2PSNRUPDKYHIWENDISBO53V3XZNB7W4V3QCXJX54UYPEQ, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul 9 00:35:31.271606981 (375h23m58.197901662s) [noise=true , address=45.91.116.8:4045 , piece_id=2SSDQFSIUS5P44VYYESMSIV3B33FK3GTNLFJ7BLETS4ZBUBX23NQ, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul 9 00:35:31.271957840 (375h23m58.198252513s) [noise=true , address=45.91.116.12:4227 , piece_id=YRM7X5XX6RUDWN66E2EOLKYCJW7SUX7YV6JPFF37YAJ6I2XG3R6A, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul 9 00:35:31.272530035 (375h23m58.198824725s) [noise=true , address=45.91.116.9:4263 , piece_id=4E3DU6HHBXFQBVZSNSVEIAODSSYZDTZRZ3Y7RPFL4LII7PWRBXFA, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul 9 00:35:31.272809654 (375h23m58.199104333s) [noise=true , address=45.91.116.4:4940 , piece_id=RLRQW743C2VJA4OXU6KHUO3546XMDV57EZOCK3YZUUORMT64TAJA, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul 9 00:35:31.275138939 (375h23m58.201433617s) [noise=true , address=77.90.31.12:4590 , piece_id=QPKATZ4XR5G2BDO3SMTY4YPQ4VI27M2E4HCIW3ATEEMOFYUWQ6IQ, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul 9 00:35:31.585358643 (375h23m58.511653329s) [noise=true , address=45.91.116.11:4117 , piece_id=CVPAQT4D65AM6RBUXEFHYAXPZA7AQAG4JWWAMWBDXW3HAV3SQYKQ, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul 9 00:35:52.939705862 (375h24m19.866000541s) [noise=true , address=104.36.82.7:4910 , piece_id=267AUB3PQ2QAHP3HNVNHCCWDLGYST7WSTXTWNFOBUR56QRUJLXVA, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul 9 00:35:59.619210529 (375h24m26.545505204s) [noise=true , address=104.36.82.12:4150 , piece_id=YMNCM2R7X6JG2E4BG5KJ7O7MJC6XWE5IVY6DJ67ZKIWTIHDMIMHQ, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul 9 00:36:01.109106257 (375h24m28.035400951s) [noise=true , address=104.36.82.10:4077 , piece_id=55HERBR2GAOGJJSGUPEFJPWQM5WIL3ENISEBUCFMZZNDJFDQ5SIA, seg_pos={0 0}, upload=300640]: Done uploading piece. err: ecclient: upload failed (node:12kDJuQP8iLUjB1rXpcNX5TtHzsqa5HvYcUD9Ef64HYTnTXcwLk, address:104.36.82.10:4077): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:36:01.109138267 (375h24m28.035433079s) [noise=true , address=104.36.82.15:4258 , piece_id=WMT5VC4QNRQ2TVBMBXSTUSA2C42DGEMPCCQFFDBS6PLMJKFUI5KA, seg_pos={0 0}, upload=300640]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:104.36.82.15:4258): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:36:01.109262427 (375h24m28.035557124s) [noise=true , address=104.36.82.14:4223 , piece_id=7ZRAYI2W4WGAYCLPOXPXSAMITQZB4LWGGGLI4E64BYRZG7RV4RSA, seg_pos={0 0}, upload=300640]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:36:01.109285927 (375h24m28.035580609s) [noise=true , address=104.36.82.4:4802 , piece_id=66M7CRNO4WP5BZZ3CPK2RCPD3BHJW4RWVEK3XJJA7O7ZBTKJHJBQ, seg_pos={0 0}, upload=300640]: Done uploading piece. err: ecclient: upload failed (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4, address:104.36.82.4:4802): protocol: write tcp 159.223.143.39:54458->104.36.82.4:4802: use of closed network connection; write tcp 159.223.143.39:54458->104.36.82.4:4802: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:54458->104.36.82.4:4802: use of closed network connection
Jul 9 00:36:01.109358127 (375h24m28.03565291s) [noise=true , address=104.36.82.15:4257 , piece_id=TCLY7VDUEWNDDCZBA5HO6GDDOOGI22BITL6VVRETPI74G2T46RRA, seg_pos={0 0}, upload=300640]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:104.36.82.15:4257): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:36:01.109390282 (375h24m28.035685081s) [seg_pos={0 0}, upload=300640]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Jul 9 00:36:01.146868060 (375h24m28.07316275s) [upload=300640]: Done uploading segments...
Jul 9 00:36:01.146918616 (375h24m28.073213304s) [upload=300640]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 11.502722529s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
14 out of 60 is failed (threshold=3) [XX..X.XX.X.......X...X......X.......X...X...X...........XX..] duration:
1.13078368s
FAIL (2025-07-09 00:37:14 UTC)
orbiter warning: context deadline exceeded
Duration: 1.130811198s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.] duration:
20.256139389s
FAIL (2025-07-09 00:36:45 UTC)
orbiter warning:
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: expected piece hash; context canceled; EOF
Jul 9 00:34:30.136902627 (375h22m57.063197324s) [seg_pos={0 0}, upload=300627]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 00:34:30.137026593 (375h22m57.06332128s) [upload=300627]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 00:34:30.137033272 (375h22m57.063327966s) [upload=300627]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 00:34:30.239293314 (375h22m57.165588013s) [upload=300627]: Done uploading segments...
Jul 9 00:34:30.239339271 (375h22m57.165633956s) [upload=300627]: Upload finished. err: <nil>
Jul 9 00:35:31.142705883 (375h23m58.069000569s) [upload=300640]: Starting upload
Jul 9 00:35:31.142808419 (375h23m58.069103108s) [upload=300640]: Done starting upload
Jul 9 00:35:31.142933499 (375h23m58.069228187s) [upload=300640]: Uploading segments...
Jul 9 00:35:31.143428918 (375h23m58.069723604s) [upload=300640]: Got next segment. Inline: false
Jul 9 00:35:31.143492378 (375h23m58.069787058s) [upload=300640]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 00:35:31.143498141 (375h23m58.069792822s) [upload=300640]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 00:35:31.172809934 (375h23m58.099104631s) [seg_pos={0 0}, upload=300640]: Begin upload segment...
Jul 9 00:35:31.172884798 (375h23m58.099179493s) [seg_pos={0 0}, upload=300640]: Done begin upload segment.
Jul 9 00:35:31.172918708 (375h23m58.0992134s) [noise=true , address=45.91.116.12:4227 , piece_id=YRM7X5XX6RUDWN66E2EOLKYCJW7SUX7YV6JPFF37YAJ6I2XG3R6A, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul 9 00:35:31.172934200 (375h23m58.099228892s) [noise=true , address=104.36.82.7:4910 , piece_id=267AUB3PQ2QAHP3HNVNHCCWDLGYST7WSTXTWNFOBUR56QRUJLXVA, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul 9 00:35:31.173374012 (375h23m58.099668707s) [noise=true , address=104.36.82.15:4257 , piece_id=TCLY7VDUEWNDDCZBA5HO6GDDOOGI22BITL6VVRETPI74G2T46RRA, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul 9 00:35:31.173385067 (375h23m58.099679757s) [noise=true , address=77.90.31.18:4806 , piece_id=6BPPUMQ7JQHH5XPB7UJT4UZQJF4F46PXEBRGIWLSVHDZGHRKDYLA, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul 9 00:35:31.173803824 (375h23m58.10009852s) [noise=true , address=45.91.116.4:4940 , piece_id=RLRQW743C2VJA4OXU6KHUO3546XMDV57EZOCK3YZUUORMT64TAJA, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul 9 00:35:31.173904484 (375h23m58.10019918s) [noise=true , address=45.91.116.10:4081 , piece_id=WCOKGYR2PSNRUPDKYHIWENDISBO53V3XZNB7W4V3QCXJX54UYPEQ, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul 9 00:35:31.174218964 (375h23m58.100513662s) [noise=true , address=45.91.116.8:4045 , piece_id=2SSDQFSIUS5P44VYYESMSIV3B33FK3GTNLFJ7BLETS4ZBUBX23NQ, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul 9 00:35:31.174461657 (375h23m58.100756346s) [noise=true , address=104.36.82.4:4802 , piece_id=66M7CRNO4WP5BZZ3CPK2RCPD3BHJW4RWVEK3XJJA7O7ZBTKJHJBQ, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul 9 00:35:31.174753443 (375h23m58.101048139s) [noise=true , address=77.90.31.12:4590 , piece_id=QPKATZ4XR5G2BDO3SMTY4YPQ4VI27M2E4HCIW3ATEEMOFYUWQ6IQ, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul 9 00:35:31.175058813 (375h23m58.101353622s) [noise=true , address=104.36.82.12:4150 , piece_id=YMNCM2R7X6JG2E4BG5KJ7O7MJC6XWE5IVY6DJ67ZKIWTIHDMIMHQ, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul 9 00:35:31.175146907 (375h23m58.101441596s) [noise=true , address=45.91.116.9:4263 , piece_id=4E3DU6HHBXFQBVZSNSVEIAODSSYZDTZRZ3Y7RPFL4LII7PWRBXFA, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul 9 00:35:31.175470130 (375h23m58.101764827s) [noise=true , address=45.91.116.11:4117 , piece_id=CVPAQT4D65AM6RBUXEFHYAXPZA7AQAG4JWWAMWBDXW3HAV3SQYKQ, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul 9 00:35:31.175522889 (375h23m58.101817587s) [noise=true , address=77.90.31.10:4520 , piece_id=44MZGFISHSDAWO6TU3KWH6K6ZQJ4YPB7A6CKCSXS4B76ROLULA7A, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul 9 00:35:31.175804424 (375h23m58.102099113s) [noise=true , address=104.36.82.10:4077 , piece_id=55HERBR2GAOGJJSGUPEFJPWQM5WIL3ENISEBUCFMZZNDJFDQ5SIA, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul 9 00:35:31.176052223 (375h23m58.102346908s) [noise=true , address=104.36.82.15:4258 , piece_id=WMT5VC4QNRQ2TVBMBXSTUSA2C42DGEMPCCQFFDBS6PLMJKFUI5KA, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul 9 00:35:31.176479692 (375h23m58.102774378s) [noise=true , address=104.36.82.14:4223 , piece_id=7ZRAYI2W4WGAYCLPOXPXSAMITQZB4LWGGGLI4E64BYRZG7RV4RSA, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul 9 00:35:31.266381427 (375h23m58.192676104s) [upload=300640]: Next returned nil segment
Jul 9 00:35:31.266395009 (375h23m58.192689686s) [upload=300640]: Waiting for error group managing segments...
Jul 9 00:35:31.270056616 (375h23m58.19635129s) [noise=true , address=77.90.31.10:4520 , piece_id=44MZGFISHSDAWO6TU3KWH6K6ZQJ4YPB7A6CKCSXS4B76ROLULA7A, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul 9 00:35:31.270776078 (375h23m58.197070768s) [noise=true , address=77.90.31.18:4806 , piece_id=6BPPUMQ7JQHH5XPB7UJT4UZQJF4F46PXEBRGIWLSVHDZGHRKDYLA, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul 9 00:35:31.271385196 (375h23m58.197679869s) [noise=true , address=45.91.116.10:4081 , piece_id=WCOKGYR2PSNRUPDKYHIWENDISBO53V3XZNB7W4V3QCXJX54UYPEQ, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul 9 00:35:31.271606981 (375h23m58.197901662s) [noise=true , address=45.91.116.8:4045 , piece_id=2SSDQFSIUS5P44VYYESMSIV3B33FK3GTNLFJ7BLETS4ZBUBX23NQ, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul 9 00:35:31.271957840 (375h23m58.198252513s) [noise=true , address=45.91.116.12:4227 , piece_id=YRM7X5XX6RUDWN66E2EOLKYCJW7SUX7YV6JPFF37YAJ6I2XG3R6A, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul 9 00:35:31.272530035 (375h23m58.198824725s) [noise=true , address=45.91.116.9:4263 , piece_id=4E3DU6HHBXFQBVZSNSVEIAODSSYZDTZRZ3Y7RPFL4LII7PWRBXFA, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul 9 00:35:31.272809654 (375h23m58.199104333s) [noise=true , address=45.91.116.4:4940 , piece_id=RLRQW743C2VJA4OXU6KHUO3546XMDV57EZOCK3YZUUORMT64TAJA, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul 9 00:35:31.275138939 (375h23m58.201433617s) [noise=true , address=77.90.31.12:4590 , piece_id=QPKATZ4XR5G2BDO3SMTY4YPQ4VI27M2E4HCIW3ATEEMOFYUWQ6IQ, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul 9 00:35:31.585358643 (375h23m58.511653329s) [noise=true , address=45.91.116.11:4117 , piece_id=CVPAQT4D65AM6RBUXEFHYAXPZA7AQAG4JWWAMWBDXW3HAV3SQYKQ, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul 9 00:35:52.939705862 (375h24m19.866000541s) [noise=true , address=104.36.82.7:4910 , piece_id=267AUB3PQ2QAHP3HNVNHCCWDLGYST7WSTXTWNFOBUR56QRUJLXVA, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul 9 00:35:59.619210529 (375h24m26.545505204s) [noise=true , address=104.36.82.12:4150 , piece_id=YMNCM2R7X6JG2E4BG5KJ7O7MJC6XWE5IVY6DJ67ZKIWTIHDMIMHQ, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul 9 00:36:01.109106257 (375h24m28.035400951s) [noise=true , address=104.36.82.10:4077 , piece_id=55HERBR2GAOGJJSGUPEFJPWQM5WIL3ENISEBUCFMZZNDJFDQ5SIA, seg_pos={0 0}, upload=300640]: Done uploading piece. err: ecclient: upload failed (node:12kDJuQP8iLUjB1rXpcNX5TtHzsqa5HvYcUD9Ef64HYTnTXcwLk, address:104.36.82.10:4077): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:36:01.109138267 (375h24m28.035433079s) [noise=true , address=104.36.82.15:4258 , piece_id=WMT5VC4QNRQ2TVBMBXSTUSA2C42DGEMPCCQFFDBS6PLMJKFUI5KA, seg_pos={0 0}, upload=300640]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:104.36.82.15:4258): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:36:01.109262427 (375h24m28.035557124s) [noise=true , address=104.36.82.14:4223 , piece_id=7ZRAYI2W4WGAYCLPOXPXSAMITQZB4LWGGGLI4E64BYRZG7RV4RSA, seg_pos={0 0}, upload=300640]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:36:01.109285927 (375h24m28.035580609s) [noise=true , address=104.36.82.4:4802 , piece_id=66M7CRNO4WP5BZZ3CPK2RCPD3BHJW4RWVEK3XJJA7O7ZBTKJHJBQ, seg_pos={0 0}, upload=300640]: Done uploading piece. err: ecclient: upload failed (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4, address:104.36.82.4:4802): protocol: write tcp 159.223.143.39:54458->104.36.82.4:4802: use of closed network connection; write tcp 159.223.143.39:54458->104.36.82.4:4802: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:54458->104.36.82.4:4802: use of closed network connection
Jul 9 00:36:01.109358127 (375h24m28.03565291s) [noise=true , address=104.36.82.15:4257 , piece_id=TCLY7VDUEWNDDCZBA5HO6GDDOOGI22BITL6VVRETPI74G2T46RRA, seg_pos={0 0}, upload=300640]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:104.36.82.15:4257): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:36:01.109390282 (375h24m28.035685081s) [seg_pos={0 0}, upload=300640]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Jul 9 00:36:01.146868060 (375h24m28.07316275s) [upload=300640]: Done uploading segments...
Jul 9 00:36:01.146918616 (375h24m28.073213304s) [upload=300640]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 20.256170132s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
15 out of 60 is failed (threshold=3) [XXX..X.XX.X.......X...X......X.......X...X...X...........XX.] duration:
796.265748ms
FAIL (2025-07-09 00:36:45 UTC)
orbiter warning: context deadline exceeded
Duration: 796.285711ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
9 out of 60 is failed (threshold=3) [..XX........X.......X...X................X...........XX...X.] duration:
35.493428341s
FAIL (2025-07-09 00:36:19 UTC)
operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 35.493443143s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
15 out of 60 is failed (threshold=3) [.XXX..X.XX.X.......X...X......X.......X...X...X...........XX] duration:
30.001474038s
FAIL (2025-07-09 00:36:05 UTC)
orbiter warning: context deadline exceeded
Duration: 30.001489612s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
30.03949796s
FAIL (2025-07-09 00:36:01 UTC)
orbiter warning:
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: expected piece hash; context canceled; EOF
Jul 9 00:34:30.136902627 (375h22m57.063197324s) [seg_pos={0 0}, upload=300627]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 00:34:30.137026593 (375h22m57.06332128s) [upload=300627]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 00:34:30.137033272 (375h22m57.063327966s) [upload=300627]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 00:34:30.239293314 (375h22m57.165588013s) [upload=300627]: Done uploading segments...
Jul 9 00:34:30.239339271 (375h22m57.165633956s) [upload=300627]: Upload finished. err: <nil>
Jul 9 00:35:31.142705883 (375h23m58.069000569s) [upload=300640]: Starting upload
Jul 9 00:35:31.142808419 (375h23m58.069103108s) [upload=300640]: Done starting upload
Jul 9 00:35:31.142933499 (375h23m58.069228187s) [upload=300640]: Uploading segments...
Jul 9 00:35:31.143428918 (375h23m58.069723604s) [upload=300640]: Got next segment. Inline: false
Jul 9 00:35:31.143492378 (375h23m58.069787058s) [upload=300640]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 00:35:31.143498141 (375h23m58.069792822s) [upload=300640]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 00:35:31.172809934 (375h23m58.099104631s) [seg_pos={0 0}, upload=300640]: Begin upload segment...
Jul 9 00:35:31.172884798 (375h23m58.099179493s) [seg_pos={0 0}, upload=300640]: Done begin upload segment.
Jul 9 00:35:31.172918708 (375h23m58.0992134s) [noise=true , address=45.91.116.12:4227 , piece_id=YRM7X5XX6RUDWN66E2EOLKYCJW7SUX7YV6JPFF37YAJ6I2XG3R6A, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul 9 00:35:31.172934200 (375h23m58.099228892s) [noise=true , address=104.36.82.7:4910 , piece_id=267AUB3PQ2QAHP3HNVNHCCWDLGYST7WSTXTWNFOBUR56QRUJLXVA, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul 9 00:35:31.173374012 (375h23m58.099668707s) [noise=true , address=104.36.82.15:4257 , piece_id=TCLY7VDUEWNDDCZBA5HO6GDDOOGI22BITL6VVRETPI74G2T46RRA, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul 9 00:35:31.173385067 (375h23m58.099679757s) [noise=true , address=77.90.31.18:4806 , piece_id=6BPPUMQ7JQHH5XPB7UJT4UZQJF4F46PXEBRGIWLSVHDZGHRKDYLA, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul 9 00:35:31.173803824 (375h23m58.10009852s) [noise=true , address=45.91.116.4:4940 , piece_id=RLRQW743C2VJA4OXU6KHUO3546XMDV57EZOCK3YZUUORMT64TAJA, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul 9 00:35:31.173904484 (375h23m58.10019918s) [noise=true , address=45.91.116.10:4081 , piece_id=WCOKGYR2PSNRUPDKYHIWENDISBO53V3XZNB7W4V3QCXJX54UYPEQ, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul 9 00:35:31.174218964 (375h23m58.100513662s) [noise=true , address=45.91.116.8:4045 , piece_id=2SSDQFSIUS5P44VYYESMSIV3B33FK3GTNLFJ7BLETS4ZBUBX23NQ, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul 9 00:35:31.174461657 (375h23m58.100756346s) [noise=true , address=104.36.82.4:4802 , piece_id=66M7CRNO4WP5BZZ3CPK2RCPD3BHJW4RWVEK3XJJA7O7ZBTKJHJBQ, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul 9 00:35:31.174753443 (375h23m58.101048139s) [noise=true , address=77.90.31.12:4590 , piece_id=QPKATZ4XR5G2BDO3SMTY4YPQ4VI27M2E4HCIW3ATEEMOFYUWQ6IQ, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul 9 00:35:31.175058813 (375h23m58.101353622s) [noise=true , address=104.36.82.12:4150 , piece_id=YMNCM2R7X6JG2E4BG5KJ7O7MJC6XWE5IVY6DJ67ZKIWTIHDMIMHQ, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul 9 00:35:31.175146907 (375h23m58.101441596s) [noise=true , address=45.91.116.9:4263 , piece_id=4E3DU6HHBXFQBVZSNSVEIAODSSYZDTZRZ3Y7RPFL4LII7PWRBXFA, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul 9 00:35:31.175470130 (375h23m58.101764827s) [noise=true , address=45.91.116.11:4117 , piece_id=CVPAQT4D65AM6RBUXEFHYAXPZA7AQAG4JWWAMWBDXW3HAV3SQYKQ, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul 9 00:35:31.175522889 (375h23m58.101817587s) [noise=true , address=77.90.31.10:4520 , piece_id=44MZGFISHSDAWO6TU3KWH6K6ZQJ4YPB7A6CKCSXS4B76ROLULA7A, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul 9 00:35:31.175804424 (375h23m58.102099113s) [noise=true , address=104.36.82.10:4077 , piece_id=55HERBR2GAOGJJSGUPEFJPWQM5WIL3ENISEBUCFMZZNDJFDQ5SIA, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul 9 00:35:31.176052223 (375h23m58.102346908s) [noise=true , address=104.36.82.15:4258 , piece_id=WMT5VC4QNRQ2TVBMBXSTUSA2C42DGEMPCCQFFDBS6PLMJKFUI5KA, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul 9 00:35:31.176479692 (375h23m58.102774378s) [noise=true , address=104.36.82.14:4223 , piece_id=7ZRAYI2W4WGAYCLPOXPXSAMITQZB4LWGGGLI4E64BYRZG7RV4RSA, seg_pos={0 0}, upload=300640]: Uploading piece...
Jul 9 00:35:31.266381427 (375h23m58.192676104s) [upload=300640]: Next returned nil segment
Jul 9 00:35:31.266395009 (375h23m58.192689686s) [upload=300640]: Waiting for error group managing segments...
Jul 9 00:35:31.270056616 (375h23m58.19635129s) [noise=true , address=77.90.31.10:4520 , piece_id=44MZGFISHSDAWO6TU3KWH6K6ZQJ4YPB7A6CKCSXS4B76ROLULA7A, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul 9 00:35:31.270776078 (375h23m58.197070768s) [noise=true , address=77.90.31.18:4806 , piece_id=6BPPUMQ7JQHH5XPB7UJT4UZQJF4F46PXEBRGIWLSVHDZGHRKDYLA, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul 9 00:35:31.271385196 (375h23m58.197679869s) [noise=true , address=45.91.116.10:4081 , piece_id=WCOKGYR2PSNRUPDKYHIWENDISBO53V3XZNB7W4V3QCXJX54UYPEQ, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul 9 00:35:31.271606981 (375h23m58.197901662s) [noise=true , address=45.91.116.8:4045 , piece_id=2SSDQFSIUS5P44VYYESMSIV3B33FK3GTNLFJ7BLETS4ZBUBX23NQ, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul 9 00:35:31.271957840 (375h23m58.198252513s) [noise=true , address=45.91.116.12:4227 , piece_id=YRM7X5XX6RUDWN66E2EOLKYCJW7SUX7YV6JPFF37YAJ6I2XG3R6A, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul 9 00:35:31.272530035 (375h23m58.198824725s) [noise=true , address=45.91.116.9:4263 , piece_id=4E3DU6HHBXFQBVZSNSVEIAODSSYZDTZRZ3Y7RPFL4LII7PWRBXFA, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul 9 00:35:31.272809654 (375h23m58.199104333s) [noise=true , address=45.91.116.4:4940 , piece_id=RLRQW743C2VJA4OXU6KHUO3546XMDV57EZOCK3YZUUORMT64TAJA, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul 9 00:35:31.275138939 (375h23m58.201433617s) [noise=true , address=77.90.31.12:4590 , piece_id=QPKATZ4XR5G2BDO3SMTY4YPQ4VI27M2E4HCIW3ATEEMOFYUWQ6IQ, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul 9 00:35:31.585358643 (375h23m58.511653329s) [noise=true , address=45.91.116.11:4117 , piece_id=CVPAQT4D65AM6RBUXEFHYAXPZA7AQAG4JWWAMWBDXW3HAV3SQYKQ, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul 9 00:35:52.939705862 (375h24m19.866000541s) [noise=true , address=104.36.82.7:4910 , piece_id=267AUB3PQ2QAHP3HNVNHCCWDLGYST7WSTXTWNFOBUR56QRUJLXVA, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul 9 00:35:59.619210529 (375h24m26.545505204s) [noise=true , address=104.36.82.12:4150 , piece_id=YMNCM2R7X6JG2E4BG5KJ7O7MJC6XWE5IVY6DJ67ZKIWTIHDMIMHQ, seg_pos={0 0}, upload=300640]: Done uploading piece. err: <nil>
Jul 9 00:36:01.109106257 (375h24m28.035400951s) [noise=true , address=104.36.82.10:4077 , piece_id=55HERBR2GAOGJJSGUPEFJPWQM5WIL3ENISEBUCFMZZNDJFDQ5SIA, seg_pos={0 0}, upload=300640]: Done uploading piece. err: ecclient: upload failed (node:12kDJuQP8iLUjB1rXpcNX5TtHzsqa5HvYcUD9Ef64HYTnTXcwLk, address:104.36.82.10:4077): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:36:01.109138267 (375h24m28.035433079s) [noise=true , address=104.36.82.15:4258 , piece_id=WMT5VC4QNRQ2TVBMBXSTUSA2C42DGEMPCCQFFDBS6PLMJKFUI5KA, seg_pos={0 0}, upload=300640]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:104.36.82.15:4258): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:36:01.109262427 (375h24m28.035557124s) [noise=true , address=104.36.82.14:4223 , piece_id=7ZRAYI2W4WGAYCLPOXPXSAMITQZB4LWGGGLI4E64BYRZG7RV4RSA, seg_pos={0 0}, upload=300640]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:36:01.109285927 (375h24m28.035580609s) [noise=true , address=104.36.82.4:4802 , piece_id=66M7CRNO4WP5BZZ3CPK2RCPD3BHJW4RWVEK3XJJA7O7ZBTKJHJBQ, seg_pos={0 0}, upload=300640]: Done uploading piece. err: ecclient: upload failed (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4, address:104.36.82.4:4802): protocol: write tcp 159.223.143.39:54458->104.36.82.4:4802: use of closed network connection; write tcp 159.223.143.39:54458->104.36.82.4:4802: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:54458->104.36.82.4:4802: use of closed network connection
Jul 9 00:36:01.109358127 (375h24m28.03565291s) [noise=true , address=104.36.82.15:4257 , piece_id=TCLY7VDUEWNDDCZBA5HO6GDDOOGI22BITL6VVRETPI74G2T46RRA, seg_pos={0 0}, upload=300640]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:104.36.82.15:4257): protocol: expected piece hash; context deadline exceeded; EOF
Jul 9 00:36:01.109390282 (375h24m28.035685081s) [seg_pos={0 0}, upload=300640]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Jul 9 00:36:01.146868060 (375h24m28.07316275s) [upload=300640]: Done uploading segments...
Jul 9 00:36:01.146918616 (375h24m28.073213304s) [upload=300640]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 30.039522123s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
9 out of 60 is failed (threshold=3) [...XX........X.......X...X................X...........XX...X] duration:
1m0.003061195s
FAIL (2025-07-09 00:35:44 UTC)
operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1m0.003082321s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
14 out of 60 is failed (threshold=3) [..XXX..X.XX.X.......X...X......X.......X...X...X...........X] duration:
30.001168111s
FAIL (2025-07-09 00:35:35 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 30.001266989s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
13 out of 60 is failed (threshold=3) [...XXX..X.XX.X.......X...X......X.......X...X...X...........] duration:
22.417179797s
FAIL (2025-07-09 00:34:37 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 22.417206485s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
8 out of 60 is failed (threshold=3) [....XX........X.......X...X................X...........XX...] duration:
46.679676638s
FAIL (2025-07-09 00:34:32 UTC)
operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 46.679719697s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
13 out of 60 is failed (threshold=3) [....XXX..X.XX.X.......X...X......X.......X...X...X..........] duration:
4.651236251s
FAIL (2025-07-09 00:33:38 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 4.651257811s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
8 out of 60 is failed (threshold=3) [.....XX........X.......X...X................X...........XX..] duration:
31.016017157s
FAIL (2025-07-09 00:33:36 UTC)
operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 31.016038983s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
13 out of 60 is failed (threshold=3) [.....XXX..X.XX.X.......X...X......X.......X...X...X.........] duration:
23.059199159s
FAIL (2025-07-09 00:33:07 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 23.059561122s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
8 out of 60 is failed (threshold=3) [......XX........X.......X...X................X...........XX.] duration:
19.191477012s
FAIL (2025-07-09 00:32:22 UTC)
operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 19.19150505s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
13 out of 60 is failed (threshold=3) [......XXX..X.XX.X.......X...X......X.......X...X...X........] duration:
19.680181903s
FAIL (2025-07-09 00:31:56 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 19.680206193s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
8 out of 60 is failed (threshold=3) [.......XX........X.......X...X................X...........XX] duration:
1m0.001250813s
FAIL (2025-07-09 00:31:54 UTC)
operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1m0.001282615s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
13 out of 60 is failed (threshold=3) [.......XXX..X.XX.X.......X...X......X.......X...X...X.......] duration:
8.934556668s
FAIL (2025-07-09 00:30:57 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 8.934581303s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
7 out of 60 is failed (threshold=3) [........XX........X.......X...X................X...........X] duration:
1m0.000695659s
FAIL (2025-07-09 00:30:54 UTC)
operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1m0.000726378s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
13 out of 60 is failed (threshold=3) [........XXX..X.XX.X.......X...X......X.......X...X...X......] duration:
3.538037361s
FAIL (2025-07-09 00:30:06 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.538072193s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
13 out of 60 is failed (threshold=3) [.........XXX..X.XX.X.......X...X......X.......X...X...X.....] duration:
9.673638064s
FAIL (2025-07-09 00:29:43 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 9.674230232s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
6 out of 60 is failed (threshold=3) [.........XX........X.......X...X................X...........] duration:
37.801642977s
FAIL (2025-07-09 00:29:43 UTC)
operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 37.801669318s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
13 out of 60 is failed (threshold=3) [..........XXX..X.XX.X.......X...X......X.......X...X...X....] duration:
16.99180575s
FAIL (2025-07-09 00:29:34 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 16.991867899s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
13 out of 60 is failed (threshold=3) [...........XXX..X.XX.X.......X...X......X.......X...X...X...] duration:
11.772652971s
FAIL (2025-07-09 00:28:41 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 11.772676014s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
6 out of 60 is failed (threshold=3) [..........XX........X.......X...X................X..........] duration:
19.28800235s
FAIL (2025-07-09 00:28:32 UTC)
operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 19.288029066s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
6 out of 60 is failed (threshold=3) [...........XX........X.......X...X................X.........] duration:
51.300768848s
FAIL (2025-07-09 00:27:57 UTC)
operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 51.301268295s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
13 out of 60 is failed (threshold=3) [............XXX..X.XX.X.......X...X......X.......X...X...X..] duration:
9.467638711s
FAIL (2025-07-09 00:27:48 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 9.467671832s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
6 out of 60 is failed (threshold=3) [............XX........X.......X...X................X........] duration:
44.498314486s
FAIL (2025-07-09 00:27:06 UTC)
operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 44.498441052s
core
uplink
us-select-1
us1-placement-20
|