Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
20 out of 60 is failed (threshold=3) [X.X.....XXX.XX.X...XXX....XX...X..X..X.XX.X..X..............] duration:
524.824654ms
FAIL (2025-07-09 03:04:23 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 524.893842ms
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................] duration:
1.921389006s
FAIL (2025-07-09 03:04:13 UTC)
orbiter warning:
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
02:25:27.135824454 (377h13m54.062119141s) [noise=true , address=104.36.82.12:4150 , piece_id=DRG5SBA3EE4KVQPEQOJZ6SS537SFWYFUYQXOAQMWYTT6FPVSJIPQ, seg_pos={0 0}, upload=302099]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: context canceled; context canceled
Jul 9 02:25:27.135860079 (377h13m54.062154769s) [seg_pos={0 0}, upload=302099]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 02:25:27.135957283 (377h13m54.062251971s) [upload=302099]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:25:27.135964211 (377h13m54.062258897s) [upload=302099]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:25:27.199410713 (377h13m54.125705389s) [upload=302099]: Done uploading segments...
Jul 9 02:25:27.199427901 (377h13m54.125722577s) [upload=302099]: Upload finished. err: <nil>
Jul 9 02:25:57.289077655 (377h14m24.215372329s) [upload=302109]: Starting upload
Jul 9 02:25:57.289186750 (377h14m24.215481424s) [upload=302109]: Done starting upload
Jul 9 02:25:57.289444851 (377h14m24.215739548s) [upload=302109]: Uploading segments...
Jul 9 02:25:57.289623846 (377h14m24.215918535s) [upload=302109]: Got next segment. Inline: false
Jul 9 02:25:57.289696741 (377h14m24.215991432s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:25:57.289703375 (377h14m24.215998067s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:25:57.318313152 (377h14m24.244607838s) [seg_pos={0 0}, upload=302109]: Begin upload segment...
Jul 9 02:25:57.318363286 (377h14m24.244657964s) [seg_pos={0 0}, upload=302109]: Done begin upload segment.
Jul 9 02:25:57.318386772 (377h14m24.244681463s) [noise=true , address=77.90.31.11:4553 , piece_id=TYW7NBJWACIT7OQS7A4WGXUPR3MB2FKAN2OMGAZJERB4X2DLAO2Q, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318412127 (377h14m24.244706825s) [noise=true , address=77.90.31.10:4518 , piece_id=VJ4HWM2X4VSRSUCQL5QLMBYPFBYV4JYS5ZXCBCB6MTMB76EHZFGQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318822149 (377h14m24.245116841s) [noise=true , address=45.91.116.9:4263 , piece_id=5YILAJ5BZBGOURSZE343ST4RMOTWM6ZOB3EEQQWL5WUNYWOGSAPA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318965862 (377h14m24.24526056s) [noise=true , address=45.91.116.8:4045 , piece_id=KM6ZYTHR6JGTARKZQZYIESPKBEU5H2HQFA7OARIKD2ZXDHPFSMGA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.319187516 (377h14m24.245482203s) [noise=true , address=104.36.82.7:4909 , piece_id=ZRHLWDQ5KIDRWPJ5FXAANQTWERKXCR2PPEGN442IOSYSRFAG7VZA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.319523815 (377h14m24.245818512s) [noise=true , address=45.91.116.14:4191 , piece_id=GTR6T4OTGZTEZ76FWQZ46OCH5WB3O5X27EN4AV33ZJ7D5HX4535A, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.320556632 (377h14m24.24685133s) [noise=true , address=104.36.82.2:4732 , piece_id=DJIGHJMJRAULH2L4YT2WGRXPRQJMNKZG5LICHPSKOCHUL6QDV6BQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.321019141 (377h14m24.247313835s) [noise=true , address=77.90.31.17:4770 , piece_id=V36UBBIYOFJC6KFVVBXEVT4QBUIWN257D3YSDDRAGZVXH7ZXHTBA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.322591797 (377h14m24.248886493s) [noise=true , address=77.90.31.13:4627 , piece_id=HULEKFVZNDNKS4SWWGTJUHYZWWYPDCER5W4GFCVLLDL65PAHULFQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323120451 (377h14m24.249415138s) [noise=true , address=45.91.116.4:4939 , piece_id=ANA3EZU7OZWELNG4QEMTEAYZY5H3C3H4IJFT4XKPUB7WIF44FEVQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323544591 (377h14m24.249839288s) [noise=true , address=45.91.116.7:4011 , piece_id=VDOJRN6ACK4LO44BQTULO6BB3CLAVQI7YIC6XLR32CZV4UJSPE6Q, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323950189 (377h14m24.250244889s) [noise=true , address=104.36.82.10:4078 , piece_id=ZFVUQAEV6DDFLPKCZA2XJ2BZHMMM745LGB4JRZJZQPG72WECGMDQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.326585810 (377h14m24.252880507s) [noise=true , address=45.91.116.5:4297 , piece_id=MSDWDLY5PJHQVI5565EKANJVSTMZFEUD2OTYZ52C65XQUICNMC3A, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.327871214 (377h14m24.254165904s) [noise=true , address=104.36.82.6:4875 , piece_id=C275F2JPVAF52E34CEHT5QXCHUFGIQZ7XGMOAOUOT4RA7YOHRQLQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.328605288 (377h14m24.254899986s) [noise=true , address=104.36.82.5:4838 , piece_id=WK3ALBOEMWWTVEA4FKCS352SZ6YYB35LELURNWXQZK32K4HYDRNA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.335409087 (377h14m24.261703784s) [noise=true , address=45.91.116.6:4973 , piece_id=Z7HQQ2O5QUSJKT4EMIDMEI5PIAEHNVRYX7MALUR2GQULRFIBUOKA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.475344637 (377h14m24.401639326s) [upload=302109]: Next returned nil segment
Jul 9 02:25:57.475376181 (377h14m24.401670879s) [upload=302109]: Waiting for error group managing segments...
Jul 9 02:25:57.477269278 (377h14m24.403563958s) [noise=true , address=45.91.116.7:4011 , piece_id=VDOJRN6ACK4LO44BQTULO6BB3CLAVQI7YIC6XLR32CZV4UJSPE6Q, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.477736179 (377h14m24.40403086s) [noise=true , address=45.91.116.9:4263 , piece_id=5YILAJ5BZBGOURSZE343ST4RMOTWM6ZOB3EEQQWL5WUNYWOGSAPA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.478396652 (377h14m24.404691332s) [noise=true , address=77.90.31.10:4518 , piece_id=VJ4HWM2X4VSRSUCQL5QLMBYPFBYV4JYS5ZXCBCB6MTMB76EHZFGQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.478919538 (377h14m24.405214224s) [noise=true , address=45.91.116.14:4191 , piece_id=GTR6T4OTGZTEZ76FWQZ46OCH5WB3O5X27EN4AV33ZJ7D5HX4535A, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481426716 (377h14m24.407721396s) [noise=true , address=45.91.116.4:4939 , piece_id=ANA3EZU7OZWELNG4QEMTEAYZY5H3C3H4IJFT4XKPUB7WIF44FEVQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481718373 (377h14m24.408013051s) [noise=true , address=45.91.116.5:4297 , piece_id=MSDWDLY5PJHQVI5565EKANJVSTMZFEUD2OTYZ52C65XQUICNMC3A, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481930356 (377h14m24.408225033s) [noise=true , address=77.90.31.17:4770 , piece_id=V36UBBIYOFJC6KFVVBXEVT4QBUIWN257D3YSDDRAGZVXH7ZXHTBA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.482376147 (377h14m24.408670821s) [noise=true , address=45.91.116.8:4045 , piece_id=KM6ZYTHR6JGTARKZQZYIESPKBEU5H2HQFA7OARIKD2ZXDHPFSMGA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.482439192 (377h14m24.408733868s) [noise=true , address=45.91.116.6:4973 , piece_id=Z7HQQ2O5QUSJKT4EMIDMEI5PIAEHNVRYX7MALUR2GQULRFIBUOKA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.483047970 (377h14m24.409342644s) [noise=true , address=77.90.31.13:4627 , piece_id=HULEKFVZNDNKS4SWWGTJUHYZWWYPDCER5W4GFCVLLDL65PAHULFQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.497788632 (377h14m24.424083312s) [noise=true , address=77.90.31.11:4553 , piece_id=TYW7NBJWACIT7OQS7A4WGXUPR3MB2FKAN2OMGAZJERB4X2DLAO2Q, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:16.688921171 (377h14m43.615215851s) [noise=true , address=104.36.82.6:4875 , piece_id=C275F2JPVAF52E34CEHT5QXCHUFGIQZ7XGMOAOUOT4RA7YOHRQLQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:27.177893955 (377h14m54.104188635s) [noise=true , address=104.36.82.2:4732 , piece_id=DJIGHJMJRAULH2L4YT2WGRXPRQJMNKZG5LICHPSKOCHUL6QDV6BQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:27.177985584 (377h14m54.104280281s) [seg_pos={0 0}, upload=302109]: Segment reached optimal threshold of 13 pieces.
Jul 9 02:26:27.178189789 (377h14m54.104484461s) [noise=true , address=104.36.82.5:4838 , piece_id=WK3ALBOEMWWTVEA4FKCS352SZ6YYB35LELURNWXQZK32K4HYDRNA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178252042 (377h14m54.104546733s) [noise=true , address=104.36.82.7:4909 , piece_id=ZRHLWDQ5KIDRWPJ5FXAANQTWERKXCR2PPEGN442IOSYSRFAG7VZA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178273922 (377h14m54.104568608s) [noise=true , address=104.36.82.10:4078 , piece_id=ZFVUQAEV6DDFLPKCZA2XJ2BZHMMM745LGB4JRZJZQPG72WECGMDQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178305983 (377h14m54.104600682s) [seg_pos={0 0}, upload=302109]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 02:26:27.178428768 (377h14m54.104723465s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:26:27.178455810 (377h14m54.104750501s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:26:27.232166184 (377h14m54.158460883s) [upload=302109]: Done uploading segments...
Jul 9 02:26:27.232188444 (377h14m54.158483123s) [upload=302109]: Upload finished. err: <nil>
Duration: 1.921423954s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
33 out of 60 is failed (threshold=3) [XX.XX.XXXXXX.XXXXX.XX.....X..XXXXX.XX..XXX.XX.XX..X.........] duration:
671.533917ms
FAIL (2025-07-09 03:04:00 UTC)
operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 671.622604ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
20 out of 60 is failed (threshold=3) [.X.X.....XXX.XX.X...XXX....XX...X..X..X.XX.X..X.............] duration:
963.888704ms
FAIL (2025-07-09 03:03:48 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 963.909165ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
21 out of 60 is failed (threshold=3) [X.X.X.....XXX.XX.X...XXX....XX...X..X..X.XX.X..X............] duration:
517.788517ms
FAIL (2025-07-09 03:03:14 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 517.808833ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
34 out of 60 is failed (threshold=3) [XXX.XX.XXXXXX.XXXXX.XX.....X..XXXXX.XX..XXX.XX.XX..X........] duration:
648.220099ms
FAIL (2025-07-09 03:03:09 UTC)
operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 648.2549ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
438.270474ms
FAIL (2025-07-09 03:03:07 UTC)
orbiter warning:
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
02:25:27.135824454 (377h13m54.062119141s) [noise=true , address=104.36.82.12:4150 , piece_id=DRG5SBA3EE4KVQPEQOJZ6SS537SFWYFUYQXOAQMWYTT6FPVSJIPQ, seg_pos={0 0}, upload=302099]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: context canceled; context canceled
Jul 9 02:25:27.135860079 (377h13m54.062154769s) [seg_pos={0 0}, upload=302099]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 02:25:27.135957283 (377h13m54.062251971s) [upload=302099]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:25:27.135964211 (377h13m54.062258897s) [upload=302099]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:25:27.199410713 (377h13m54.125705389s) [upload=302099]: Done uploading segments...
Jul 9 02:25:27.199427901 (377h13m54.125722577s) [upload=302099]: Upload finished. err: <nil>
Jul 9 02:25:57.289077655 (377h14m24.215372329s) [upload=302109]: Starting upload
Jul 9 02:25:57.289186750 (377h14m24.215481424s) [upload=302109]: Done starting upload
Jul 9 02:25:57.289444851 (377h14m24.215739548s) [upload=302109]: Uploading segments...
Jul 9 02:25:57.289623846 (377h14m24.215918535s) [upload=302109]: Got next segment. Inline: false
Jul 9 02:25:57.289696741 (377h14m24.215991432s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:25:57.289703375 (377h14m24.215998067s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:25:57.318313152 (377h14m24.244607838s) [seg_pos={0 0}, upload=302109]: Begin upload segment...
Jul 9 02:25:57.318363286 (377h14m24.244657964s) [seg_pos={0 0}, upload=302109]: Done begin upload segment.
Jul 9 02:25:57.318386772 (377h14m24.244681463s) [noise=true , address=77.90.31.11:4553 , piece_id=TYW7NBJWACIT7OQS7A4WGXUPR3MB2FKAN2OMGAZJERB4X2DLAO2Q, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318412127 (377h14m24.244706825s) [noise=true , address=77.90.31.10:4518 , piece_id=VJ4HWM2X4VSRSUCQL5QLMBYPFBYV4JYS5ZXCBCB6MTMB76EHZFGQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318822149 (377h14m24.245116841s) [noise=true , address=45.91.116.9:4263 , piece_id=5YILAJ5BZBGOURSZE343ST4RMOTWM6ZOB3EEQQWL5WUNYWOGSAPA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318965862 (377h14m24.24526056s) [noise=true , address=45.91.116.8:4045 , piece_id=KM6ZYTHR6JGTARKZQZYIESPKBEU5H2HQFA7OARIKD2ZXDHPFSMGA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.319187516 (377h14m24.245482203s) [noise=true , address=104.36.82.7:4909 , piece_id=ZRHLWDQ5KIDRWPJ5FXAANQTWERKXCR2PPEGN442IOSYSRFAG7VZA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.319523815 (377h14m24.245818512s) [noise=true , address=45.91.116.14:4191 , piece_id=GTR6T4OTGZTEZ76FWQZ46OCH5WB3O5X27EN4AV33ZJ7D5HX4535A, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.320556632 (377h14m24.24685133s) [noise=true , address=104.36.82.2:4732 , piece_id=DJIGHJMJRAULH2L4YT2WGRXPRQJMNKZG5LICHPSKOCHUL6QDV6BQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.321019141 (377h14m24.247313835s) [noise=true , address=77.90.31.17:4770 , piece_id=V36UBBIYOFJC6KFVVBXEVT4QBUIWN257D3YSDDRAGZVXH7ZXHTBA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.322591797 (377h14m24.248886493s) [noise=true , address=77.90.31.13:4627 , piece_id=HULEKFVZNDNKS4SWWGTJUHYZWWYPDCER5W4GFCVLLDL65PAHULFQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323120451 (377h14m24.249415138s) [noise=true , address=45.91.116.4:4939 , piece_id=ANA3EZU7OZWELNG4QEMTEAYZY5H3C3H4IJFT4XKPUB7WIF44FEVQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323544591 (377h14m24.249839288s) [noise=true , address=45.91.116.7:4011 , piece_id=VDOJRN6ACK4LO44BQTULO6BB3CLAVQI7YIC6XLR32CZV4UJSPE6Q, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323950189 (377h14m24.250244889s) [noise=true , address=104.36.82.10:4078 , piece_id=ZFVUQAEV6DDFLPKCZA2XJ2BZHMMM745LGB4JRZJZQPG72WECGMDQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.326585810 (377h14m24.252880507s) [noise=true , address=45.91.116.5:4297 , piece_id=MSDWDLY5PJHQVI5565EKANJVSTMZFEUD2OTYZ52C65XQUICNMC3A, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.327871214 (377h14m24.254165904s) [noise=true , address=104.36.82.6:4875 , piece_id=C275F2JPVAF52E34CEHT5QXCHUFGIQZ7XGMOAOUOT4RA7YOHRQLQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.328605288 (377h14m24.254899986s) [noise=true , address=104.36.82.5:4838 , piece_id=WK3ALBOEMWWTVEA4FKCS352SZ6YYB35LELURNWXQZK32K4HYDRNA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.335409087 (377h14m24.261703784s) [noise=true , address=45.91.116.6:4973 , piece_id=Z7HQQ2O5QUSJKT4EMIDMEI5PIAEHNVRYX7MALUR2GQULRFIBUOKA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.475344637 (377h14m24.401639326s) [upload=302109]: Next returned nil segment
Jul 9 02:25:57.475376181 (377h14m24.401670879s) [upload=302109]: Waiting for error group managing segments...
Jul 9 02:25:57.477269278 (377h14m24.403563958s) [noise=true , address=45.91.116.7:4011 , piece_id=VDOJRN6ACK4LO44BQTULO6BB3CLAVQI7YIC6XLR32CZV4UJSPE6Q, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.477736179 (377h14m24.40403086s) [noise=true , address=45.91.116.9:4263 , piece_id=5YILAJ5BZBGOURSZE343ST4RMOTWM6ZOB3EEQQWL5WUNYWOGSAPA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.478396652 (377h14m24.404691332s) [noise=true , address=77.90.31.10:4518 , piece_id=VJ4HWM2X4VSRSUCQL5QLMBYPFBYV4JYS5ZXCBCB6MTMB76EHZFGQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.478919538 (377h14m24.405214224s) [noise=true , address=45.91.116.14:4191 , piece_id=GTR6T4OTGZTEZ76FWQZ46OCH5WB3O5X27EN4AV33ZJ7D5HX4535A, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481426716 (377h14m24.407721396s) [noise=true , address=45.91.116.4:4939 , piece_id=ANA3EZU7OZWELNG4QEMTEAYZY5H3C3H4IJFT4XKPUB7WIF44FEVQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481718373 (377h14m24.408013051s) [noise=true , address=45.91.116.5:4297 , piece_id=MSDWDLY5PJHQVI5565EKANJVSTMZFEUD2OTYZ52C65XQUICNMC3A, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481930356 (377h14m24.408225033s) [noise=true , address=77.90.31.17:4770 , piece_id=V36UBBIYOFJC6KFVVBXEVT4QBUIWN257D3YSDDRAGZVXH7ZXHTBA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.482376147 (377h14m24.408670821s) [noise=true , address=45.91.116.8:4045 , piece_id=KM6ZYTHR6JGTARKZQZYIESPKBEU5H2HQFA7OARIKD2ZXDHPFSMGA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.482439192 (377h14m24.408733868s) [noise=true , address=45.91.116.6:4973 , piece_id=Z7HQQ2O5QUSJKT4EMIDMEI5PIAEHNVRYX7MALUR2GQULRFIBUOKA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.483047970 (377h14m24.409342644s) [noise=true , address=77.90.31.13:4627 , piece_id=HULEKFVZNDNKS4SWWGTJUHYZWWYPDCER5W4GFCVLLDL65PAHULFQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.497788632 (377h14m24.424083312s) [noise=true , address=77.90.31.11:4553 , piece_id=TYW7NBJWACIT7OQS7A4WGXUPR3MB2FKAN2OMGAZJERB4X2DLAO2Q, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:16.688921171 (377h14m43.615215851s) [noise=true , address=104.36.82.6:4875 , piece_id=C275F2JPVAF52E34CEHT5QXCHUFGIQZ7XGMOAOUOT4RA7YOHRQLQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:27.177893955 (377h14m54.104188635s) [noise=true , address=104.36.82.2:4732 , piece_id=DJIGHJMJRAULH2L4YT2WGRXPRQJMNKZG5LICHPSKOCHUL6QDV6BQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:27.177985584 (377h14m54.104280281s) [seg_pos={0 0}, upload=302109]: Segment reached optimal threshold of 13 pieces.
Jul 9 02:26:27.178189789 (377h14m54.104484461s) [noise=true , address=104.36.82.5:4838 , piece_id=WK3ALBOEMWWTVEA4FKCS352SZ6YYB35LELURNWXQZK32K4HYDRNA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178252042 (377h14m54.104546733s) [noise=true , address=104.36.82.7:4909 , piece_id=ZRHLWDQ5KIDRWPJ5FXAANQTWERKXCR2PPEGN442IOSYSRFAG7VZA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178273922 (377h14m54.104568608s) [noise=true , address=104.36.82.10:4078 , piece_id=ZFVUQAEV6DDFLPKCZA2XJ2BZHMMM745LGB4JRZJZQPG72WECGMDQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178305983 (377h14m54.104600682s) [seg_pos={0 0}, upload=302109]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 02:26:27.178428768 (377h14m54.104723465s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:26:27.178455810 (377h14m54.104750501s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:26:27.232166184 (377h14m54.158460883s) [upload=302109]: Done uploading segments...
Jul 9 02:26:27.232188444 (377h14m54.158483123s) [upload=302109]: Upload finished. err: <nil>
Duration: 438.300859ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
21 out of 60 is failed (threshold=3) [.X.X.X.....XXX.XX.X...XXX....XX...X..X..X.XX.X..X...........] duration:
463.245937ms
FAIL (2025-07-09 03:02:50 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 463.273664ms
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................] duration:
456.616153ms
FAIL (2025-07-09 03:02:40 UTC)
orbiter warning:
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
02:25:27.135824454 (377h13m54.062119141s) [noise=true , address=104.36.82.12:4150 , piece_id=DRG5SBA3EE4KVQPEQOJZ6SS537SFWYFUYQXOAQMWYTT6FPVSJIPQ, seg_pos={0 0}, upload=302099]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: context canceled; context canceled
Jul 9 02:25:27.135860079 (377h13m54.062154769s) [seg_pos={0 0}, upload=302099]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 02:25:27.135957283 (377h13m54.062251971s) [upload=302099]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:25:27.135964211 (377h13m54.062258897s) [upload=302099]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:25:27.199410713 (377h13m54.125705389s) [upload=302099]: Done uploading segments...
Jul 9 02:25:27.199427901 (377h13m54.125722577s) [upload=302099]: Upload finished. err: <nil>
Jul 9 02:25:57.289077655 (377h14m24.215372329s) [upload=302109]: Starting upload
Jul 9 02:25:57.289186750 (377h14m24.215481424s) [upload=302109]: Done starting upload
Jul 9 02:25:57.289444851 (377h14m24.215739548s) [upload=302109]: Uploading segments...
Jul 9 02:25:57.289623846 (377h14m24.215918535s) [upload=302109]: Got next segment. Inline: false
Jul 9 02:25:57.289696741 (377h14m24.215991432s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:25:57.289703375 (377h14m24.215998067s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:25:57.318313152 (377h14m24.244607838s) [seg_pos={0 0}, upload=302109]: Begin upload segment...
Jul 9 02:25:57.318363286 (377h14m24.244657964s) [seg_pos={0 0}, upload=302109]: Done begin upload segment.
Jul 9 02:25:57.318386772 (377h14m24.244681463s) [noise=true , address=77.90.31.11:4553 , piece_id=TYW7NBJWACIT7OQS7A4WGXUPR3MB2FKAN2OMGAZJERB4X2DLAO2Q, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318412127 (377h14m24.244706825s) [noise=true , address=77.90.31.10:4518 , piece_id=VJ4HWM2X4VSRSUCQL5QLMBYPFBYV4JYS5ZXCBCB6MTMB76EHZFGQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318822149 (377h14m24.245116841s) [noise=true , address=45.91.116.9:4263 , piece_id=5YILAJ5BZBGOURSZE343ST4RMOTWM6ZOB3EEQQWL5WUNYWOGSAPA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318965862 (377h14m24.24526056s) [noise=true , address=45.91.116.8:4045 , piece_id=KM6ZYTHR6JGTARKZQZYIESPKBEU5H2HQFA7OARIKD2ZXDHPFSMGA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.319187516 (377h14m24.245482203s) [noise=true , address=104.36.82.7:4909 , piece_id=ZRHLWDQ5KIDRWPJ5FXAANQTWERKXCR2PPEGN442IOSYSRFAG7VZA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.319523815 (377h14m24.245818512s) [noise=true , address=45.91.116.14:4191 , piece_id=GTR6T4OTGZTEZ76FWQZ46OCH5WB3O5X27EN4AV33ZJ7D5HX4535A, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.320556632 (377h14m24.24685133s) [noise=true , address=104.36.82.2:4732 , piece_id=DJIGHJMJRAULH2L4YT2WGRXPRQJMNKZG5LICHPSKOCHUL6QDV6BQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.321019141 (377h14m24.247313835s) [noise=true , address=77.90.31.17:4770 , piece_id=V36UBBIYOFJC6KFVVBXEVT4QBUIWN257D3YSDDRAGZVXH7ZXHTBA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.322591797 (377h14m24.248886493s) [noise=true , address=77.90.31.13:4627 , piece_id=HULEKFVZNDNKS4SWWGTJUHYZWWYPDCER5W4GFCVLLDL65PAHULFQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323120451 (377h14m24.249415138s) [noise=true , address=45.91.116.4:4939 , piece_id=ANA3EZU7OZWELNG4QEMTEAYZY5H3C3H4IJFT4XKPUB7WIF44FEVQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323544591 (377h14m24.249839288s) [noise=true , address=45.91.116.7:4011 , piece_id=VDOJRN6ACK4LO44BQTULO6BB3CLAVQI7YIC6XLR32CZV4UJSPE6Q, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323950189 (377h14m24.250244889s) [noise=true , address=104.36.82.10:4078 , piece_id=ZFVUQAEV6DDFLPKCZA2XJ2BZHMMM745LGB4JRZJZQPG72WECGMDQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.326585810 (377h14m24.252880507s) [noise=true , address=45.91.116.5:4297 , piece_id=MSDWDLY5PJHQVI5565EKANJVSTMZFEUD2OTYZ52C65XQUICNMC3A, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.327871214 (377h14m24.254165904s) [noise=true , address=104.36.82.6:4875 , piece_id=C275F2JPVAF52E34CEHT5QXCHUFGIQZ7XGMOAOUOT4RA7YOHRQLQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.328605288 (377h14m24.254899986s) [noise=true , address=104.36.82.5:4838 , piece_id=WK3ALBOEMWWTVEA4FKCS352SZ6YYB35LELURNWXQZK32K4HYDRNA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.335409087 (377h14m24.261703784s) [noise=true , address=45.91.116.6:4973 , piece_id=Z7HQQ2O5QUSJKT4EMIDMEI5PIAEHNVRYX7MALUR2GQULRFIBUOKA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.475344637 (377h14m24.401639326s) [upload=302109]: Next returned nil segment
Jul 9 02:25:57.475376181 (377h14m24.401670879s) [upload=302109]: Waiting for error group managing segments...
Jul 9 02:25:57.477269278 (377h14m24.403563958s) [noise=true , address=45.91.116.7:4011 , piece_id=VDOJRN6ACK4LO44BQTULO6BB3CLAVQI7YIC6XLR32CZV4UJSPE6Q, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.477736179 (377h14m24.40403086s) [noise=true , address=45.91.116.9:4263 , piece_id=5YILAJ5BZBGOURSZE343ST4RMOTWM6ZOB3EEQQWL5WUNYWOGSAPA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.478396652 (377h14m24.404691332s) [noise=true , address=77.90.31.10:4518 , piece_id=VJ4HWM2X4VSRSUCQL5QLMBYPFBYV4JYS5ZXCBCB6MTMB76EHZFGQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.478919538 (377h14m24.405214224s) [noise=true , address=45.91.116.14:4191 , piece_id=GTR6T4OTGZTEZ76FWQZ46OCH5WB3O5X27EN4AV33ZJ7D5HX4535A, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481426716 (377h14m24.407721396s) [noise=true , address=45.91.116.4:4939 , piece_id=ANA3EZU7OZWELNG4QEMTEAYZY5H3C3H4IJFT4XKPUB7WIF44FEVQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481718373 (377h14m24.408013051s) [noise=true , address=45.91.116.5:4297 , piece_id=MSDWDLY5PJHQVI5565EKANJVSTMZFEUD2OTYZ52C65XQUICNMC3A, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481930356 (377h14m24.408225033s) [noise=true , address=77.90.31.17:4770 , piece_id=V36UBBIYOFJC6KFVVBXEVT4QBUIWN257D3YSDDRAGZVXH7ZXHTBA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.482376147 (377h14m24.408670821s) [noise=true , address=45.91.116.8:4045 , piece_id=KM6ZYTHR6JGTARKZQZYIESPKBEU5H2HQFA7OARIKD2ZXDHPFSMGA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.482439192 (377h14m24.408733868s) [noise=true , address=45.91.116.6:4973 , piece_id=Z7HQQ2O5QUSJKT4EMIDMEI5PIAEHNVRYX7MALUR2GQULRFIBUOKA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.483047970 (377h14m24.409342644s) [noise=true , address=77.90.31.13:4627 , piece_id=HULEKFVZNDNKS4SWWGTJUHYZWWYPDCER5W4GFCVLLDL65PAHULFQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.497788632 (377h14m24.424083312s) [noise=true , address=77.90.31.11:4553 , piece_id=TYW7NBJWACIT7OQS7A4WGXUPR3MB2FKAN2OMGAZJERB4X2DLAO2Q, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:16.688921171 (377h14m43.615215851s) [noise=true , address=104.36.82.6:4875 , piece_id=C275F2JPVAF52E34CEHT5QXCHUFGIQZ7XGMOAOUOT4RA7YOHRQLQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:27.177893955 (377h14m54.104188635s) [noise=true , address=104.36.82.2:4732 , piece_id=DJIGHJMJRAULH2L4YT2WGRXPRQJMNKZG5LICHPSKOCHUL6QDV6BQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:27.177985584 (377h14m54.104280281s) [seg_pos={0 0}, upload=302109]: Segment reached optimal threshold of 13 pieces.
Jul 9 02:26:27.178189789 (377h14m54.104484461s) [noise=true , address=104.36.82.5:4838 , piece_id=WK3ALBOEMWWTVEA4FKCS352SZ6YYB35LELURNWXQZK32K4HYDRNA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178252042 (377h14m54.104546733s) [noise=true , address=104.36.82.7:4909 , piece_id=ZRHLWDQ5KIDRWPJ5FXAANQTWERKXCR2PPEGN442IOSYSRFAG7VZA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178273922 (377h14m54.104568608s) [noise=true , address=104.36.82.10:4078 , piece_id=ZFVUQAEV6DDFLPKCZA2XJ2BZHMMM745LGB4JRZJZQPG72WECGMDQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178305983 (377h14m54.104600682s) [seg_pos={0 0}, upload=302109]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 02:26:27.178428768 (377h14m54.104723465s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:26:27.178455810 (377h14m54.104750501s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:26:27.232166184 (377h14m54.158460883s) [upload=302109]: Done uploading segments...
Jul 9 02:26:27.232188444 (377h14m54.158483123s) [upload=302109]: Upload finished. err: <nil>
Duration: 456.669399ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......X.....................................................] duration:
416.199325ms
FAIL (2025-07-09 03:02:17 UTC)
orbiter warning:
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
02:25:27.135824454 (377h13m54.062119141s) [noise=true , address=104.36.82.12:4150 , piece_id=DRG5SBA3EE4KVQPEQOJZ6SS537SFWYFUYQXOAQMWYTT6FPVSJIPQ, seg_pos={0 0}, upload=302099]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: context canceled; context canceled
Jul 9 02:25:27.135860079 (377h13m54.062154769s) [seg_pos={0 0}, upload=302099]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 02:25:27.135957283 (377h13m54.062251971s) [upload=302099]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:25:27.135964211 (377h13m54.062258897s) [upload=302099]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:25:27.199410713 (377h13m54.125705389s) [upload=302099]: Done uploading segments...
Jul 9 02:25:27.199427901 (377h13m54.125722577s) [upload=302099]: Upload finished. err: <nil>
Jul 9 02:25:57.289077655 (377h14m24.215372329s) [upload=302109]: Starting upload
Jul 9 02:25:57.289186750 (377h14m24.215481424s) [upload=302109]: Done starting upload
Jul 9 02:25:57.289444851 (377h14m24.215739548s) [upload=302109]: Uploading segments...
Jul 9 02:25:57.289623846 (377h14m24.215918535s) [upload=302109]: Got next segment. Inline: false
Jul 9 02:25:57.289696741 (377h14m24.215991432s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:25:57.289703375 (377h14m24.215998067s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:25:57.318313152 (377h14m24.244607838s) [seg_pos={0 0}, upload=302109]: Begin upload segment...
Jul 9 02:25:57.318363286 (377h14m24.244657964s) [seg_pos={0 0}, upload=302109]: Done begin upload segment.
Jul 9 02:25:57.318386772 (377h14m24.244681463s) [noise=true , address=77.90.31.11:4553 , piece_id=TYW7NBJWACIT7OQS7A4WGXUPR3MB2FKAN2OMGAZJERB4X2DLAO2Q, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318412127 (377h14m24.244706825s) [noise=true , address=77.90.31.10:4518 , piece_id=VJ4HWM2X4VSRSUCQL5QLMBYPFBYV4JYS5ZXCBCB6MTMB76EHZFGQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318822149 (377h14m24.245116841s) [noise=true , address=45.91.116.9:4263 , piece_id=5YILAJ5BZBGOURSZE343ST4RMOTWM6ZOB3EEQQWL5WUNYWOGSAPA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318965862 (377h14m24.24526056s) [noise=true , address=45.91.116.8:4045 , piece_id=KM6ZYTHR6JGTARKZQZYIESPKBEU5H2HQFA7OARIKD2ZXDHPFSMGA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.319187516 (377h14m24.245482203s) [noise=true , address=104.36.82.7:4909 , piece_id=ZRHLWDQ5KIDRWPJ5FXAANQTWERKXCR2PPEGN442IOSYSRFAG7VZA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.319523815 (377h14m24.245818512s) [noise=true , address=45.91.116.14:4191 , piece_id=GTR6T4OTGZTEZ76FWQZ46OCH5WB3O5X27EN4AV33ZJ7D5HX4535A, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.320556632 (377h14m24.24685133s) [noise=true , address=104.36.82.2:4732 , piece_id=DJIGHJMJRAULH2L4YT2WGRXPRQJMNKZG5LICHPSKOCHUL6QDV6BQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.321019141 (377h14m24.247313835s) [noise=true , address=77.90.31.17:4770 , piece_id=V36UBBIYOFJC6KFVVBXEVT4QBUIWN257D3YSDDRAGZVXH7ZXHTBA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.322591797 (377h14m24.248886493s) [noise=true , address=77.90.31.13:4627 , piece_id=HULEKFVZNDNKS4SWWGTJUHYZWWYPDCER5W4GFCVLLDL65PAHULFQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323120451 (377h14m24.249415138s) [noise=true , address=45.91.116.4:4939 , piece_id=ANA3EZU7OZWELNG4QEMTEAYZY5H3C3H4IJFT4XKPUB7WIF44FEVQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323544591 (377h14m24.249839288s) [noise=true , address=45.91.116.7:4011 , piece_id=VDOJRN6ACK4LO44BQTULO6BB3CLAVQI7YIC6XLR32CZV4UJSPE6Q, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323950189 (377h14m24.250244889s) [noise=true , address=104.36.82.10:4078 , piece_id=ZFVUQAEV6DDFLPKCZA2XJ2BZHMMM745LGB4JRZJZQPG72WECGMDQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.326585810 (377h14m24.252880507s) [noise=true , address=45.91.116.5:4297 , piece_id=MSDWDLY5PJHQVI5565EKANJVSTMZFEUD2OTYZ52C65XQUICNMC3A, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.327871214 (377h14m24.254165904s) [noise=true , address=104.36.82.6:4875 , piece_id=C275F2JPVAF52E34CEHT5QXCHUFGIQZ7XGMOAOUOT4RA7YOHRQLQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.328605288 (377h14m24.254899986s) [noise=true , address=104.36.82.5:4838 , piece_id=WK3ALBOEMWWTVEA4FKCS352SZ6YYB35LELURNWXQZK32K4HYDRNA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.335409087 (377h14m24.261703784s) [noise=true , address=45.91.116.6:4973 , piece_id=Z7HQQ2O5QUSJKT4EMIDMEI5PIAEHNVRYX7MALUR2GQULRFIBUOKA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.475344637 (377h14m24.401639326s) [upload=302109]: Next returned nil segment
Jul 9 02:25:57.475376181 (377h14m24.401670879s) [upload=302109]: Waiting for error group managing segments...
Jul 9 02:25:57.477269278 (377h14m24.403563958s) [noise=true , address=45.91.116.7:4011 , piece_id=VDOJRN6ACK4LO44BQTULO6BB3CLAVQI7YIC6XLR32CZV4UJSPE6Q, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.477736179 (377h14m24.40403086s) [noise=true , address=45.91.116.9:4263 , piece_id=5YILAJ5BZBGOURSZE343ST4RMOTWM6ZOB3EEQQWL5WUNYWOGSAPA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.478396652 (377h14m24.404691332s) [noise=true , address=77.90.31.10:4518 , piece_id=VJ4HWM2X4VSRSUCQL5QLMBYPFBYV4JYS5ZXCBCB6MTMB76EHZFGQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.478919538 (377h14m24.405214224s) [noise=true , address=45.91.116.14:4191 , piece_id=GTR6T4OTGZTEZ76FWQZ46OCH5WB3O5X27EN4AV33ZJ7D5HX4535A, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481426716 (377h14m24.407721396s) [noise=true , address=45.91.116.4:4939 , piece_id=ANA3EZU7OZWELNG4QEMTEAYZY5H3C3H4IJFT4XKPUB7WIF44FEVQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481718373 (377h14m24.408013051s) [noise=true , address=45.91.116.5:4297 , piece_id=MSDWDLY5PJHQVI5565EKANJVSTMZFEUD2OTYZ52C65XQUICNMC3A, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481930356 (377h14m24.408225033s) [noise=true , address=77.90.31.17:4770 , piece_id=V36UBBIYOFJC6KFVVBXEVT4QBUIWN257D3YSDDRAGZVXH7ZXHTBA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.482376147 (377h14m24.408670821s) [noise=true , address=45.91.116.8:4045 , piece_id=KM6ZYTHR6JGTARKZQZYIESPKBEU5H2HQFA7OARIKD2ZXDHPFSMGA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.482439192 (377h14m24.408733868s) [noise=true , address=45.91.116.6:4973 , piece_id=Z7HQQ2O5QUSJKT4EMIDMEI5PIAEHNVRYX7MALUR2GQULRFIBUOKA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.483047970 (377h14m24.409342644s) [noise=true , address=77.90.31.13:4627 , piece_id=HULEKFVZNDNKS4SWWGTJUHYZWWYPDCER5W4GFCVLLDL65PAHULFQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.497788632 (377h14m24.424083312s) [noise=true , address=77.90.31.11:4553 , piece_id=TYW7NBJWACIT7OQS7A4WGXUPR3MB2FKAN2OMGAZJERB4X2DLAO2Q, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:16.688921171 (377h14m43.615215851s) [noise=true , address=104.36.82.6:4875 , piece_id=C275F2JPVAF52E34CEHT5QXCHUFGIQZ7XGMOAOUOT4RA7YOHRQLQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:27.177893955 (377h14m54.104188635s) [noise=true , address=104.36.82.2:4732 , piece_id=DJIGHJMJRAULH2L4YT2WGRXPRQJMNKZG5LICHPSKOCHUL6QDV6BQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:27.177985584 (377h14m54.104280281s) [seg_pos={0 0}, upload=302109]: Segment reached optimal threshold of 13 pieces.
Jul 9 02:26:27.178189789 (377h14m54.104484461s) [noise=true , address=104.36.82.5:4838 , piece_id=WK3ALBOEMWWTVEA4FKCS352SZ6YYB35LELURNWXQZK32K4HYDRNA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178252042 (377h14m54.104546733s) [noise=true , address=104.36.82.7:4909 , piece_id=ZRHLWDQ5KIDRWPJ5FXAANQTWERKXCR2PPEGN442IOSYSRFAG7VZA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178273922 (377h14m54.104568608s) [noise=true , address=104.36.82.10:4078 , piece_id=ZFVUQAEV6DDFLPKCZA2XJ2BZHMMM745LGB4JRZJZQPG72WECGMDQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178305983 (377h14m54.104600682s) [seg_pos={0 0}, upload=302109]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 02:26:27.178428768 (377h14m54.104723465s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:26:27.178455810 (377h14m54.104750501s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:26:27.232166184 (377h14m54.158460883s) [upload=302109]: Done uploading segments...
Jul 9 02:26:27.232188444 (377h14m54.158483123s) [upload=302109]: Upload finished. err: <nil>
Duration: 416.231801ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
34 out of 60 is failed (threshold=3) [.XXX.XX.XXXXXX.XXXXX.XX.....X..XXXXX.XX..XXX.XX.XX..X.......] duration:
679.618128ms
FAIL (2025-07-09 03:01:57 UTC)
operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 679.655627ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
21 out of 60 is failed (threshold=3) [..X.X.X.....XXX.XX.X...XXX....XX...X..X..X.XX.X..X..........] duration:
521.485159ms
FAIL (2025-07-09 03:01:49 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 521.512287ms
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......X....................................................] duration:
518.069027ms
FAIL (2025-07-09 03:01:35 UTC)
orbiter warning:
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
02:25:27.135824454 (377h13m54.062119141s) [noise=true , address=104.36.82.12:4150 , piece_id=DRG5SBA3EE4KVQPEQOJZ6SS537SFWYFUYQXOAQMWYTT6FPVSJIPQ, seg_pos={0 0}, upload=302099]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: context canceled; context canceled
Jul 9 02:25:27.135860079 (377h13m54.062154769s) [seg_pos={0 0}, upload=302099]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 02:25:27.135957283 (377h13m54.062251971s) [upload=302099]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:25:27.135964211 (377h13m54.062258897s) [upload=302099]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:25:27.199410713 (377h13m54.125705389s) [upload=302099]: Done uploading segments...
Jul 9 02:25:27.199427901 (377h13m54.125722577s) [upload=302099]: Upload finished. err: <nil>
Jul 9 02:25:57.289077655 (377h14m24.215372329s) [upload=302109]: Starting upload
Jul 9 02:25:57.289186750 (377h14m24.215481424s) [upload=302109]: Done starting upload
Jul 9 02:25:57.289444851 (377h14m24.215739548s) [upload=302109]: Uploading segments...
Jul 9 02:25:57.289623846 (377h14m24.215918535s) [upload=302109]: Got next segment. Inline: false
Jul 9 02:25:57.289696741 (377h14m24.215991432s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:25:57.289703375 (377h14m24.215998067s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:25:57.318313152 (377h14m24.244607838s) [seg_pos={0 0}, upload=302109]: Begin upload segment...
Jul 9 02:25:57.318363286 (377h14m24.244657964s) [seg_pos={0 0}, upload=302109]: Done begin upload segment.
Jul 9 02:25:57.318386772 (377h14m24.244681463s) [noise=true , address=77.90.31.11:4553 , piece_id=TYW7NBJWACIT7OQS7A4WGXUPR3MB2FKAN2OMGAZJERB4X2DLAO2Q, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318412127 (377h14m24.244706825s) [noise=true , address=77.90.31.10:4518 , piece_id=VJ4HWM2X4VSRSUCQL5QLMBYPFBYV4JYS5ZXCBCB6MTMB76EHZFGQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318822149 (377h14m24.245116841s) [noise=true , address=45.91.116.9:4263 , piece_id=5YILAJ5BZBGOURSZE343ST4RMOTWM6ZOB3EEQQWL5WUNYWOGSAPA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318965862 (377h14m24.24526056s) [noise=true , address=45.91.116.8:4045 , piece_id=KM6ZYTHR6JGTARKZQZYIESPKBEU5H2HQFA7OARIKD2ZXDHPFSMGA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.319187516 (377h14m24.245482203s) [noise=true , address=104.36.82.7:4909 , piece_id=ZRHLWDQ5KIDRWPJ5FXAANQTWERKXCR2PPEGN442IOSYSRFAG7VZA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.319523815 (377h14m24.245818512s) [noise=true , address=45.91.116.14:4191 , piece_id=GTR6T4OTGZTEZ76FWQZ46OCH5WB3O5X27EN4AV33ZJ7D5HX4535A, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.320556632 (377h14m24.24685133s) [noise=true , address=104.36.82.2:4732 , piece_id=DJIGHJMJRAULH2L4YT2WGRXPRQJMNKZG5LICHPSKOCHUL6QDV6BQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.321019141 (377h14m24.247313835s) [noise=true , address=77.90.31.17:4770 , piece_id=V36UBBIYOFJC6KFVVBXEVT4QBUIWN257D3YSDDRAGZVXH7ZXHTBA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.322591797 (377h14m24.248886493s) [noise=true , address=77.90.31.13:4627 , piece_id=HULEKFVZNDNKS4SWWGTJUHYZWWYPDCER5W4GFCVLLDL65PAHULFQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323120451 (377h14m24.249415138s) [noise=true , address=45.91.116.4:4939 , piece_id=ANA3EZU7OZWELNG4QEMTEAYZY5H3C3H4IJFT4XKPUB7WIF44FEVQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323544591 (377h14m24.249839288s) [noise=true , address=45.91.116.7:4011 , piece_id=VDOJRN6ACK4LO44BQTULO6BB3CLAVQI7YIC6XLR32CZV4UJSPE6Q, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323950189 (377h14m24.250244889s) [noise=true , address=104.36.82.10:4078 , piece_id=ZFVUQAEV6DDFLPKCZA2XJ2BZHMMM745LGB4JRZJZQPG72WECGMDQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.326585810 (377h14m24.252880507s) [noise=true , address=45.91.116.5:4297 , piece_id=MSDWDLY5PJHQVI5565EKANJVSTMZFEUD2OTYZ52C65XQUICNMC3A, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.327871214 (377h14m24.254165904s) [noise=true , address=104.36.82.6:4875 , piece_id=C275F2JPVAF52E34CEHT5QXCHUFGIQZ7XGMOAOUOT4RA7YOHRQLQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.328605288 (377h14m24.254899986s) [noise=true , address=104.36.82.5:4838 , piece_id=WK3ALBOEMWWTVEA4FKCS352SZ6YYB35LELURNWXQZK32K4HYDRNA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.335409087 (377h14m24.261703784s) [noise=true , address=45.91.116.6:4973 , piece_id=Z7HQQ2O5QUSJKT4EMIDMEI5PIAEHNVRYX7MALUR2GQULRFIBUOKA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.475344637 (377h14m24.401639326s) [upload=302109]: Next returned nil segment
Jul 9 02:25:57.475376181 (377h14m24.401670879s) [upload=302109]: Waiting for error group managing segments...
Jul 9 02:25:57.477269278 (377h14m24.403563958s) [noise=true , address=45.91.116.7:4011 , piece_id=VDOJRN6ACK4LO44BQTULO6BB3CLAVQI7YIC6XLR32CZV4UJSPE6Q, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.477736179 (377h14m24.40403086s) [noise=true , address=45.91.116.9:4263 , piece_id=5YILAJ5BZBGOURSZE343ST4RMOTWM6ZOB3EEQQWL5WUNYWOGSAPA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.478396652 (377h14m24.404691332s) [noise=true , address=77.90.31.10:4518 , piece_id=VJ4HWM2X4VSRSUCQL5QLMBYPFBYV4JYS5ZXCBCB6MTMB76EHZFGQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.478919538 (377h14m24.405214224s) [noise=true , address=45.91.116.14:4191 , piece_id=GTR6T4OTGZTEZ76FWQZ46OCH5WB3O5X27EN4AV33ZJ7D5HX4535A, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481426716 (377h14m24.407721396s) [noise=true , address=45.91.116.4:4939 , piece_id=ANA3EZU7OZWELNG4QEMTEAYZY5H3C3H4IJFT4XKPUB7WIF44FEVQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481718373 (377h14m24.408013051s) [noise=true , address=45.91.116.5:4297 , piece_id=MSDWDLY5PJHQVI5565EKANJVSTMZFEUD2OTYZ52C65XQUICNMC3A, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481930356 (377h14m24.408225033s) [noise=true , address=77.90.31.17:4770 , piece_id=V36UBBIYOFJC6KFVVBXEVT4QBUIWN257D3YSDDRAGZVXH7ZXHTBA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.482376147 (377h14m24.408670821s) [noise=true , address=45.91.116.8:4045 , piece_id=KM6ZYTHR6JGTARKZQZYIESPKBEU5H2HQFA7OARIKD2ZXDHPFSMGA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.482439192 (377h14m24.408733868s) [noise=true , address=45.91.116.6:4973 , piece_id=Z7HQQ2O5QUSJKT4EMIDMEI5PIAEHNVRYX7MALUR2GQULRFIBUOKA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.483047970 (377h14m24.409342644s) [noise=true , address=77.90.31.13:4627 , piece_id=HULEKFVZNDNKS4SWWGTJUHYZWWYPDCER5W4GFCVLLDL65PAHULFQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.497788632 (377h14m24.424083312s) [noise=true , address=77.90.31.11:4553 , piece_id=TYW7NBJWACIT7OQS7A4WGXUPR3MB2FKAN2OMGAZJERB4X2DLAO2Q, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:16.688921171 (377h14m43.615215851s) [noise=true , address=104.36.82.6:4875 , piece_id=C275F2JPVAF52E34CEHT5QXCHUFGIQZ7XGMOAOUOT4RA7YOHRQLQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:27.177893955 (377h14m54.104188635s) [noise=true , address=104.36.82.2:4732 , piece_id=DJIGHJMJRAULH2L4YT2WGRXPRQJMNKZG5LICHPSKOCHUL6QDV6BQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:27.177985584 (377h14m54.104280281s) [seg_pos={0 0}, upload=302109]: Segment reached optimal threshold of 13 pieces.
Jul 9 02:26:27.178189789 (377h14m54.104484461s) [noise=true , address=104.36.82.5:4838 , piece_id=WK3ALBOEMWWTVEA4FKCS352SZ6YYB35LELURNWXQZK32K4HYDRNA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178252042 (377h14m54.104546733s) [noise=true , address=104.36.82.7:4909 , piece_id=ZRHLWDQ5KIDRWPJ5FXAANQTWERKXCR2PPEGN442IOSYSRFAG7VZA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178273922 (377h14m54.104568608s) [noise=true , address=104.36.82.10:4078 , piece_id=ZFVUQAEV6DDFLPKCZA2XJ2BZHMMM745LGB4JRZJZQPG72WECGMDQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178305983 (377h14m54.104600682s) [seg_pos={0 0}, upload=302109]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 02:26:27.178428768 (377h14m54.104723465s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:26:27.178455810 (377h14m54.104750501s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:26:27.232166184 (377h14m54.158460883s) [upload=302109]: Done uploading segments...
Jul 9 02:26:27.232188444 (377h14m54.158483123s) [upload=302109]: Upload finished. err: <nil>
Duration: 518.100243ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
470.840025ms
FAIL (2025-07-09 03:01:35 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
3:56.263910643 (377h2m23.19020535s) [upload=301954]: Got next segment. Inline: false
Jul 9 02:13:56.263954358 (377h2m23.190249034s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:13:56.263959891 (377h2m23.190254572s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:13:56.302204249 (377h2m23.228498944s) [seg_pos={0 0}, upload=301954]: Begin upload segment...
Jul 9 02:13:56.302291587 (377h2m23.22858628s) [seg_pos={0 0}, upload=301954]: Done begin upload segment.
Jul 9 02:13:56.302338947 (377h2m23.228633682s) [noise=true , address=77.90.31.14:4689 , piece_id=LW7OT74AJCCC3TEKY6OZI553YK6YQFQ6C7PORQB53E7FUS24KQKQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.302352599 (377h2m23.228647272s) [noise=true , address=104.36.82.8:4986 , piece_id=KPY5JVLSAEB5GUKUJJSIF5M6IIHTJISEYYAXCGUSJ7OLI5QCIBNQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.302450837 (377h2m23.228745531s) [noise=true , address=77.90.31.16:4739 , piece_id=H6UCV6RURPXKC2H3YGAT2OXV2VZMW2DWIRG6TFD6CVXKOVMDP2VQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.302958774 (377h2m23.229253667s) [noise=true , address=77.90.31.13:4640 , piece_id=TPPN75DX3UCBM346CSVJN3GSOA47QV4FVPYSSAHWCJYEM42OCVLQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303069086 (377h2m23.229363782s) [noise=true , address=104.36.82.17:4345 , piece_id=YI27O5WLBYO3TPCX4MVUATFORY6U3EQMDKQHGKRHIZMTG7WGFB2A, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303495994 (377h2m23.22979069s) [noise=true , address=104.36.82.36:4644 , piece_id=K2G3MAPD7TWYZPKNHQBMEPJRIQDLE75FYVPAZFX4I3GJ4IUAA7XQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303575840 (377h2m23.229870536s) [noise=true , address=77.90.31.21:4898 , piece_id=75XXVNT4EV4D5UTX7O5XFL44IZOYY5LZSI2VPFQZPRERL6W5M23A, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303949062 (377h2m23.230245641s) [noise=true , address=79.127.201.215:4339 , piece_id=XDCTYE2SVI6RAPCXTRVANFCZYWJDZXLBHHJ47NQY6V7D36DEFPGA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304060410 (377h2m23.230355106s) [noise=true , address=104.36.82.41:4768 , piece_id=BGMF33OKUZSZKGIHJYI3RNXEBDE54TX67DYLJ7JCZMRNK6ZAU22Q, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304407993 (377h2m23.230702685s) [noise=true , address=77.90.31.10:4525 , piece_id=GPVSIZIBZWTUCP5RFYVPFYIYIL2LWU5PU3UASQVRQLY7SSSFMQBA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304548115 (377h2m23.230842812s) [noise=true , address=104.36.82.23:4464 , piece_id=JCJ4MGEZIIZ7YCTM6QXC5ZJ7PI4OEJBOMHZYBU57D2SNGWJWB5CQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304795164 (377h2m23.231089862s) [noise=true , address=77.90.31.18:4838 , piece_id=TGWYXA3AF7LDG4KPVCRNEYFDADAG6TQ5DAX7JJVW5BLVVVNECILA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305012510 (377h2m23.231307211s) [noise=true , address=104.36.82.34:4554 , piece_id=57K5SDV3YEGLVMSJGSLXNKEWOLETXE6CWBM2NPVCNKLFHPVQCJBQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305113737 (377h2m23.231408427s) [noise=true , address=104.36.82.4:4834 , piece_id=YLYIASUPNRTK7KOMRYLN7OQLQBPWC7WFGCPDGUVIOEQJWQQ7A7GA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305383755 (377h2m23.231678434s) [noise=true , address=104.36.82.38:4698 , piece_id=TSOXJNQEWODCWFEITH4IYKZI7A75UZEXRQPIQO7LMP6LT3WLNXUA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305483822 (377h2m23.231778517s) [noise=true , address=104.36.82.42:4830 , piece_id=ENYQTUCIF7OWQPCXTYS24OC5QT63ZJJXOY652GLIG7OQPJEQCGGA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305954019 (377h2m23.232248718s) [noise=true , address=77.90.31.21:4907 , piece_id=2EUWATHUQNZ77BJV65ZFKR5WL4D3AUURLBQ32XR2IVBA7P4TITOQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.306425517 (377h2m23.232720213s) [noise=true , address=104.36.82.22:4541 , piece_id=XN4HDFDNZ4YUJTDPOTMMDFX4FE2YVZDGCIMHEMJGWXRPNHQ6EU7A, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.375470790 (377h2m23.301765481s) [upload=301954]: Next returned nil segment
Jul 9 02:13:56.375489934 (377h2m23.30178463s) [upload=301954]: Waiting for error group managing segments...
Jul 9 02:13:56.381456859 (377h2m23.307751539s) [noise=true , address=77.90.31.10:4525 , piece_id=GPVSIZIBZWTUCP5RFYVPFYIYIL2LWU5PU3UASQVRQLY7SSSFMQBA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.381956907 (377h2m23.308251586s) [noise=true , address=77.90.31.14:4689 , piece_id=LW7OT74AJCCC3TEKY6OZI553YK6YQFQ6C7PORQB53E7FUS24KQKQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.382837088 (377h2m23.309131761s) [noise=true , address=77.90.31.21:4898 , piece_id=75XXVNT4EV4D5UTX7O5XFL44IZOYY5LZSI2VPFQZPRERL6W5M23A, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.393489125 (377h2m23.319783812s) [noise=true , address=77.90.31.21:4907 , piece_id=2EUWATHUQNZ77BJV65ZFKR5WL4D3AUURLBQ32XR2IVBA7P4TITOQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.404692801 (377h2m23.330987481s) [noise=true , address=77.90.31.16:4739 , piece_id=H6UCV6RURPXKC2H3YGAT2OXV2VZMW2DWIRG6TFD6CVXKOVMDP2VQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.404986115 (377h2m23.331280795s) [noise=true , address=77.90.31.18:4838 , piece_id=TGWYXA3AF7LDG4KPVCRNEYFDADAG6TQ5DAX7JJVW5BLVVVNECILA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.433785086 (377h2m23.360079766s) [noise=true , address=77.90.31.13:4640 , piece_id=TPPN75DX3UCBM346CSVJN3GSOA47QV4FVPYSSAHWCJYEM42OCVLQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.598499747 (377h2m23.524794427s) [noise=true , address=79.127.201.215:4339 , piece_id=XDCTYE2SVI6RAPCXTRVANFCZYWJDZXLBHHJ47NQY6V7D36DEFPGA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:11.107785584 (377h2m38.034080273s) [noise=true , address=104.36.82.22:4541 , piece_id=XN4HDFDNZ4YUJTDPOTMMDFX4FE2YVZDGCIMHEMJGWXRPNHQ6EU7A, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:12.515567029 (377h2m39.441861713s) [noise=true , address=104.36.82.17:4345 , piece_id=YI27O5WLBYO3TPCX4MVUATFORY6U3EQMDKQHGKRHIZMTG7WGFB2A, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:12.923090587 (377h2m39.849385267s) [noise=true , address=104.36.82.8:4986 , piece_id=KPY5JVLSAEB5GUKUJJSIF5M6IIHTJISEYYAXCGUSJ7OLI5QCIBNQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:13.453172659 (377h2m40.379467346s) [noise=true , address=104.36.82.38:4698 , piece_id=TSOXJNQEWODCWFEITH4IYKZI7A75UZEXRQPIQO7LMP6LT3WLNXUA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:14.416524856 (377h2m41.342819534s) [noise=true , address=104.36.82.34:4554 , piece_id=57K5SDV3YEGLVMSJGSLXNKEWOLETXE6CWBM2NPVCNKLFHPVQCJBQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:14.846730699 (377h2m41.773025379s) [noise=true , address=104.36.82.23:4464 , piece_id=JCJ4MGEZIIZ7YCTM6QXC5ZJ7PI4OEJBOMHZYBU57D2SNGWJWB5CQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:14.846763777 (377h2m41.773058463s) [seg_pos={0 0}, upload=301954]: Segment reached optimal threshold of 14 pieces.
Jul 9 02:14:14.846894905 (377h2m41.773189582s) [noise=true , address=104.36.82.36:4644 , piece_id=K2G3MAPD7TWYZPKNHQBMEPJRIQDLE75FYVPAZFX4I3GJ4IUAA7XQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1oYHbb73ACitbfkkP5P6cmRyTt6kmaS1y49G4vbsvTvVDwByhT): protocol: expected piece hash; context canceled; EOF
Jul 9 02:14:14.846948435 (377h2m41.773243131s) [noise=true , address=104.36.82.4:4834 , piece_id=YLYIASUPNRTK7KOMRYLN7OQLQBPWC7WFGCPDGUVIOEQJWQQ7A7GA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Z241DCYA5Wig2TdEogxTpY6FzVmFbnKtmHcTwxLxfcZafgooy): protocol: expected piece hash; context canceled; EOF
Jul 9 02:14:14.847094561 (377h2m41.773389249s) [noise=true , address=104.36.82.42:4830 , piece_id=ENYQTUCIF7OWQPCXTYS24OC5QT63ZJJXOY652GLIG7OQPJEQCGGA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:142ocSHHtv8BpSebXFNk5qk1Y23a8pbFAZqE3cq6i3RMX3dPw9): protocol: expected piece hash; context canceled; EOF
Jul 9 02:14:14.847127825 (377h2m41.773422511s) [noise=true , address=104.36.82.41:4768 , piece_id=BGMF33OKUZSZKGIHJYI3RNXEBDE54TX67DYLJ7JCZMRNK6ZAU22Q, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: write tcp 159.223.143.39:43796->104.36.82.41:4768: use of closed network connection; write tcp 159.223.143.39:43796->104.36.82.41:4768: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:43796->104.36.82.41:4768: use of closed network connection
Jul 9 02:14:14.847142999 (377h2m41.773437687s) [seg_pos={0 0}, upload=301954]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jul 9 02:14:14.847204962 (377h2m41.773499652s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:14:14.847209974 (377h2m41.773504671s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:14:14.909581364 (377h2m41.83587606s) [upload=301954]: Done uploading segments...
Jul 9 02:14:14.909605483 (377h2m41.835900172s) [upload=301954]: Upload finished. err: <nil>
Duration: 470.872689ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
22 out of 60 is failed (threshold=3) [X..X.X.X.....XXX.XX.X...XXX....XX...X..X..X.XX.X..X.........] duration:
631.510982ms
FAIL (2025-07-09 03:01:33 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 631.535317ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
35 out of 60 is failed (threshold=3) [X.XXX.XX.XXXXXX.XXXXX.XX.....X..XXXXX.XX..XXX.XX.XX..X......] duration:
724.031223ms
FAIL (2025-07-09 03:01:15 UTC)
operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 724.055005ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
23 out of 60 is failed (threshold=3) [XX..X.X.X.....XXX.XX.X...XXX....XX...X..X..X.XX.X..X........] duration:
531.207698ms
FAIL (2025-07-09 03:01:12 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 531.238386ms
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
638.65368ms
FAIL (2025-07-09 03:00:58 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
3:56.263910643 (377h2m23.19020535s) [upload=301954]: Got next segment. Inline: false
Jul 9 02:13:56.263954358 (377h2m23.190249034s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:13:56.263959891 (377h2m23.190254572s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:13:56.302204249 (377h2m23.228498944s) [seg_pos={0 0}, upload=301954]: Begin upload segment...
Jul 9 02:13:56.302291587 (377h2m23.22858628s) [seg_pos={0 0}, upload=301954]: Done begin upload segment.
Jul 9 02:13:56.302338947 (377h2m23.228633682s) [noise=true , address=77.90.31.14:4689 , piece_id=LW7OT74AJCCC3TEKY6OZI553YK6YQFQ6C7PORQB53E7FUS24KQKQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.302352599 (377h2m23.228647272s) [noise=true , address=104.36.82.8:4986 , piece_id=KPY5JVLSAEB5GUKUJJSIF5M6IIHTJISEYYAXCGUSJ7OLI5QCIBNQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.302450837 (377h2m23.228745531s) [noise=true , address=77.90.31.16:4739 , piece_id=H6UCV6RURPXKC2H3YGAT2OXV2VZMW2DWIRG6TFD6CVXKOVMDP2VQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.302958774 (377h2m23.229253667s) [noise=true , address=77.90.31.13:4640 , piece_id=TPPN75DX3UCBM346CSVJN3GSOA47QV4FVPYSSAHWCJYEM42OCVLQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303069086 (377h2m23.229363782s) [noise=true , address=104.36.82.17:4345 , piece_id=YI27O5WLBYO3TPCX4MVUATFORY6U3EQMDKQHGKRHIZMTG7WGFB2A, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303495994 (377h2m23.22979069s) [noise=true , address=104.36.82.36:4644 , piece_id=K2G3MAPD7TWYZPKNHQBMEPJRIQDLE75FYVPAZFX4I3GJ4IUAA7XQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303575840 (377h2m23.229870536s) [noise=true , address=77.90.31.21:4898 , piece_id=75XXVNT4EV4D5UTX7O5XFL44IZOYY5LZSI2VPFQZPRERL6W5M23A, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303949062 (377h2m23.230245641s) [noise=true , address=79.127.201.215:4339 , piece_id=XDCTYE2SVI6RAPCXTRVANFCZYWJDZXLBHHJ47NQY6V7D36DEFPGA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304060410 (377h2m23.230355106s) [noise=true , address=104.36.82.41:4768 , piece_id=BGMF33OKUZSZKGIHJYI3RNXEBDE54TX67DYLJ7JCZMRNK6ZAU22Q, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304407993 (377h2m23.230702685s) [noise=true , address=77.90.31.10:4525 , piece_id=GPVSIZIBZWTUCP5RFYVPFYIYIL2LWU5PU3UASQVRQLY7SSSFMQBA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304548115 (377h2m23.230842812s) [noise=true , address=104.36.82.23:4464 , piece_id=JCJ4MGEZIIZ7YCTM6QXC5ZJ7PI4OEJBOMHZYBU57D2SNGWJWB5CQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304795164 (377h2m23.231089862s) [noise=true , address=77.90.31.18:4838 , piece_id=TGWYXA3AF7LDG4KPVCRNEYFDADAG6TQ5DAX7JJVW5BLVVVNECILA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305012510 (377h2m23.231307211s) [noise=true , address=104.36.82.34:4554 , piece_id=57K5SDV3YEGLVMSJGSLXNKEWOLETXE6CWBM2NPVCNKLFHPVQCJBQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305113737 (377h2m23.231408427s) [noise=true , address=104.36.82.4:4834 , piece_id=YLYIASUPNRTK7KOMRYLN7OQLQBPWC7WFGCPDGUVIOEQJWQQ7A7GA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305383755 (377h2m23.231678434s) [noise=true , address=104.36.82.38:4698 , piece_id=TSOXJNQEWODCWFEITH4IYKZI7A75UZEXRQPIQO7LMP6LT3WLNXUA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305483822 (377h2m23.231778517s) [noise=true , address=104.36.82.42:4830 , piece_id=ENYQTUCIF7OWQPCXTYS24OC5QT63ZJJXOY652GLIG7OQPJEQCGGA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305954019 (377h2m23.232248718s) [noise=true , address=77.90.31.21:4907 , piece_id=2EUWATHUQNZ77BJV65ZFKR5WL4D3AUURLBQ32XR2IVBA7P4TITOQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.306425517 (377h2m23.232720213s) [noise=true , address=104.36.82.22:4541 , piece_id=XN4HDFDNZ4YUJTDPOTMMDFX4FE2YVZDGCIMHEMJGWXRPNHQ6EU7A, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.375470790 (377h2m23.301765481s) [upload=301954]: Next returned nil segment
Jul 9 02:13:56.375489934 (377h2m23.30178463s) [upload=301954]: Waiting for error group managing segments...
Jul 9 02:13:56.381456859 (377h2m23.307751539s) [noise=true , address=77.90.31.10:4525 , piece_id=GPVSIZIBZWTUCP5RFYVPFYIYIL2LWU5PU3UASQVRQLY7SSSFMQBA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.381956907 (377h2m23.308251586s) [noise=true , address=77.90.31.14:4689 , piece_id=LW7OT74AJCCC3TEKY6OZI553YK6YQFQ6C7PORQB53E7FUS24KQKQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.382837088 (377h2m23.309131761s) [noise=true , address=77.90.31.21:4898 , piece_id=75XXVNT4EV4D5UTX7O5XFL44IZOYY5LZSI2VPFQZPRERL6W5M23A, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.393489125 (377h2m23.319783812s) [noise=true , address=77.90.31.21:4907 , piece_id=2EUWATHUQNZ77BJV65ZFKR5WL4D3AUURLBQ32XR2IVBA7P4TITOQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.404692801 (377h2m23.330987481s) [noise=true , address=77.90.31.16:4739 , piece_id=H6UCV6RURPXKC2H3YGAT2OXV2VZMW2DWIRG6TFD6CVXKOVMDP2VQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.404986115 (377h2m23.331280795s) [noise=true , address=77.90.31.18:4838 , piece_id=TGWYXA3AF7LDG4KPVCRNEYFDADAG6TQ5DAX7JJVW5BLVVVNECILA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.433785086 (377h2m23.360079766s) [noise=true , address=77.90.31.13:4640 , piece_id=TPPN75DX3UCBM346CSVJN3GSOA47QV4FVPYSSAHWCJYEM42OCVLQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.598499747 (377h2m23.524794427s) [noise=true , address=79.127.201.215:4339 , piece_id=XDCTYE2SVI6RAPCXTRVANFCZYWJDZXLBHHJ47NQY6V7D36DEFPGA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:11.107785584 (377h2m38.034080273s) [noise=true , address=104.36.82.22:4541 , piece_id=XN4HDFDNZ4YUJTDPOTMMDFX4FE2YVZDGCIMHEMJGWXRPNHQ6EU7A, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:12.515567029 (377h2m39.441861713s) [noise=true , address=104.36.82.17:4345 , piece_id=YI27O5WLBYO3TPCX4MVUATFORY6U3EQMDKQHGKRHIZMTG7WGFB2A, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:12.923090587 (377h2m39.849385267s) [noise=true , address=104.36.82.8:4986 , piece_id=KPY5JVLSAEB5GUKUJJSIF5M6IIHTJISEYYAXCGUSJ7OLI5QCIBNQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:13.453172659 (377h2m40.379467346s) [noise=true , address=104.36.82.38:4698 , piece_id=TSOXJNQEWODCWFEITH4IYKZI7A75UZEXRQPIQO7LMP6LT3WLNXUA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:14.416524856 (377h2m41.342819534s) [noise=true , address=104.36.82.34:4554 , piece_id=57K5SDV3YEGLVMSJGSLXNKEWOLETXE6CWBM2NPVCNKLFHPVQCJBQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:14.846730699 (377h2m41.773025379s) [noise=true , address=104.36.82.23:4464 , piece_id=JCJ4MGEZIIZ7YCTM6QXC5ZJ7PI4OEJBOMHZYBU57D2SNGWJWB5CQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:14.846763777 (377h2m41.773058463s) [seg_pos={0 0}, upload=301954]: Segment reached optimal threshold of 14 pieces.
Jul 9 02:14:14.846894905 (377h2m41.773189582s) [noise=true , address=104.36.82.36:4644 , piece_id=K2G3MAPD7TWYZPKNHQBMEPJRIQDLE75FYVPAZFX4I3GJ4IUAA7XQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1oYHbb73ACitbfkkP5P6cmRyTt6kmaS1y49G4vbsvTvVDwByhT): protocol: expected piece hash; context canceled; EOF
Jul 9 02:14:14.846948435 (377h2m41.773243131s) [noise=true , address=104.36.82.4:4834 , piece_id=YLYIASUPNRTK7KOMRYLN7OQLQBPWC7WFGCPDGUVIOEQJWQQ7A7GA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Z241DCYA5Wig2TdEogxTpY6FzVmFbnKtmHcTwxLxfcZafgooy): protocol: expected piece hash; context canceled; EOF
Jul 9 02:14:14.847094561 (377h2m41.773389249s) [noise=true , address=104.36.82.42:4830 , piece_id=ENYQTUCIF7OWQPCXTYS24OC5QT63ZJJXOY652GLIG7OQPJEQCGGA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:142ocSHHtv8BpSebXFNk5qk1Y23a8pbFAZqE3cq6i3RMX3dPw9): protocol: expected piece hash; context canceled; EOF
Jul 9 02:14:14.847127825 (377h2m41.773422511s) [noise=true , address=104.36.82.41:4768 , piece_id=BGMF33OKUZSZKGIHJYI3RNXEBDE54TX67DYLJ7JCZMRNK6ZAU22Q, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: write tcp 159.223.143.39:43796->104.36.82.41:4768: use of closed network connection; write tcp 159.223.143.39:43796->104.36.82.41:4768: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:43796->104.36.82.41:4768: use of closed network connection
Jul 9 02:14:14.847142999 (377h2m41.773437687s) [seg_pos={0 0}, upload=301954]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jul 9 02:14:14.847204962 (377h2m41.773499652s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:14:14.847209974 (377h2m41.773504671s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:14:14.909581364 (377h2m41.83587606s) [upload=301954]: Done uploading segments...
Jul 9 02:14:14.909605483 (377h2m41.835900172s) [upload=301954]: Upload finished. err: <nil>
Duration: 638.683925ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........X...................................................] duration:
452.578776ms
FAIL (2025-07-09 03:00:49 UTC)
orbiter warning:
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
02:25:27.135824454 (377h13m54.062119141s) [noise=true , address=104.36.82.12:4150 , piece_id=DRG5SBA3EE4KVQPEQOJZ6SS537SFWYFUYQXOAQMWYTT6FPVSJIPQ, seg_pos={0 0}, upload=302099]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: context canceled; context canceled
Jul 9 02:25:27.135860079 (377h13m54.062154769s) [seg_pos={0 0}, upload=302099]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 02:25:27.135957283 (377h13m54.062251971s) [upload=302099]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:25:27.135964211 (377h13m54.062258897s) [upload=302099]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:25:27.199410713 (377h13m54.125705389s) [upload=302099]: Done uploading segments...
Jul 9 02:25:27.199427901 (377h13m54.125722577s) [upload=302099]: Upload finished. err: <nil>
Jul 9 02:25:57.289077655 (377h14m24.215372329s) [upload=302109]: Starting upload
Jul 9 02:25:57.289186750 (377h14m24.215481424s) [upload=302109]: Done starting upload
Jul 9 02:25:57.289444851 (377h14m24.215739548s) [upload=302109]: Uploading segments...
Jul 9 02:25:57.289623846 (377h14m24.215918535s) [upload=302109]: Got next segment. Inline: false
Jul 9 02:25:57.289696741 (377h14m24.215991432s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:25:57.289703375 (377h14m24.215998067s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:25:57.318313152 (377h14m24.244607838s) [seg_pos={0 0}, upload=302109]: Begin upload segment...
Jul 9 02:25:57.318363286 (377h14m24.244657964s) [seg_pos={0 0}, upload=302109]: Done begin upload segment.
Jul 9 02:25:57.318386772 (377h14m24.244681463s) [noise=true , address=77.90.31.11:4553 , piece_id=TYW7NBJWACIT7OQS7A4WGXUPR3MB2FKAN2OMGAZJERB4X2DLAO2Q, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318412127 (377h14m24.244706825s) [noise=true , address=77.90.31.10:4518 , piece_id=VJ4HWM2X4VSRSUCQL5QLMBYPFBYV4JYS5ZXCBCB6MTMB76EHZFGQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318822149 (377h14m24.245116841s) [noise=true , address=45.91.116.9:4263 , piece_id=5YILAJ5BZBGOURSZE343ST4RMOTWM6ZOB3EEQQWL5WUNYWOGSAPA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318965862 (377h14m24.24526056s) [noise=true , address=45.91.116.8:4045 , piece_id=KM6ZYTHR6JGTARKZQZYIESPKBEU5H2HQFA7OARIKD2ZXDHPFSMGA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.319187516 (377h14m24.245482203s) [noise=true , address=104.36.82.7:4909 , piece_id=ZRHLWDQ5KIDRWPJ5FXAANQTWERKXCR2PPEGN442IOSYSRFAG7VZA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.319523815 (377h14m24.245818512s) [noise=true , address=45.91.116.14:4191 , piece_id=GTR6T4OTGZTEZ76FWQZ46OCH5WB3O5X27EN4AV33ZJ7D5HX4535A, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.320556632 (377h14m24.24685133s) [noise=true , address=104.36.82.2:4732 , piece_id=DJIGHJMJRAULH2L4YT2WGRXPRQJMNKZG5LICHPSKOCHUL6QDV6BQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.321019141 (377h14m24.247313835s) [noise=true , address=77.90.31.17:4770 , piece_id=V36UBBIYOFJC6KFVVBXEVT4QBUIWN257D3YSDDRAGZVXH7ZXHTBA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.322591797 (377h14m24.248886493s) [noise=true , address=77.90.31.13:4627 , piece_id=HULEKFVZNDNKS4SWWGTJUHYZWWYPDCER5W4GFCVLLDL65PAHULFQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323120451 (377h14m24.249415138s) [noise=true , address=45.91.116.4:4939 , piece_id=ANA3EZU7OZWELNG4QEMTEAYZY5H3C3H4IJFT4XKPUB7WIF44FEVQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323544591 (377h14m24.249839288s) [noise=true , address=45.91.116.7:4011 , piece_id=VDOJRN6ACK4LO44BQTULO6BB3CLAVQI7YIC6XLR32CZV4UJSPE6Q, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323950189 (377h14m24.250244889s) [noise=true , address=104.36.82.10:4078 , piece_id=ZFVUQAEV6DDFLPKCZA2XJ2BZHMMM745LGB4JRZJZQPG72WECGMDQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.326585810 (377h14m24.252880507s) [noise=true , address=45.91.116.5:4297 , piece_id=MSDWDLY5PJHQVI5565EKANJVSTMZFEUD2OTYZ52C65XQUICNMC3A, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.327871214 (377h14m24.254165904s) [noise=true , address=104.36.82.6:4875 , piece_id=C275F2JPVAF52E34CEHT5QXCHUFGIQZ7XGMOAOUOT4RA7YOHRQLQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.328605288 (377h14m24.254899986s) [noise=true , address=104.36.82.5:4838 , piece_id=WK3ALBOEMWWTVEA4FKCS352SZ6YYB35LELURNWXQZK32K4HYDRNA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.335409087 (377h14m24.261703784s) [noise=true , address=45.91.116.6:4973 , piece_id=Z7HQQ2O5QUSJKT4EMIDMEI5PIAEHNVRYX7MALUR2GQULRFIBUOKA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.475344637 (377h14m24.401639326s) [upload=302109]: Next returned nil segment
Jul 9 02:25:57.475376181 (377h14m24.401670879s) [upload=302109]: Waiting for error group managing segments...
Jul 9 02:25:57.477269278 (377h14m24.403563958s) [noise=true , address=45.91.116.7:4011 , piece_id=VDOJRN6ACK4LO44BQTULO6BB3CLAVQI7YIC6XLR32CZV4UJSPE6Q, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.477736179 (377h14m24.40403086s) [noise=true , address=45.91.116.9:4263 , piece_id=5YILAJ5BZBGOURSZE343ST4RMOTWM6ZOB3EEQQWL5WUNYWOGSAPA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.478396652 (377h14m24.404691332s) [noise=true , address=77.90.31.10:4518 , piece_id=VJ4HWM2X4VSRSUCQL5QLMBYPFBYV4JYS5ZXCBCB6MTMB76EHZFGQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.478919538 (377h14m24.405214224s) [noise=true , address=45.91.116.14:4191 , piece_id=GTR6T4OTGZTEZ76FWQZ46OCH5WB3O5X27EN4AV33ZJ7D5HX4535A, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481426716 (377h14m24.407721396s) [noise=true , address=45.91.116.4:4939 , piece_id=ANA3EZU7OZWELNG4QEMTEAYZY5H3C3H4IJFT4XKPUB7WIF44FEVQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481718373 (377h14m24.408013051s) [noise=true , address=45.91.116.5:4297 , piece_id=MSDWDLY5PJHQVI5565EKANJVSTMZFEUD2OTYZ52C65XQUICNMC3A, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481930356 (377h14m24.408225033s) [noise=true , address=77.90.31.17:4770 , piece_id=V36UBBIYOFJC6KFVVBXEVT4QBUIWN257D3YSDDRAGZVXH7ZXHTBA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.482376147 (377h14m24.408670821s) [noise=true , address=45.91.116.8:4045 , piece_id=KM6ZYTHR6JGTARKZQZYIESPKBEU5H2HQFA7OARIKD2ZXDHPFSMGA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.482439192 (377h14m24.408733868s) [noise=true , address=45.91.116.6:4973 , piece_id=Z7HQQ2O5QUSJKT4EMIDMEI5PIAEHNVRYX7MALUR2GQULRFIBUOKA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.483047970 (377h14m24.409342644s) [noise=true , address=77.90.31.13:4627 , piece_id=HULEKFVZNDNKS4SWWGTJUHYZWWYPDCER5W4GFCVLLDL65PAHULFQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.497788632 (377h14m24.424083312s) [noise=true , address=77.90.31.11:4553 , piece_id=TYW7NBJWACIT7OQS7A4WGXUPR3MB2FKAN2OMGAZJERB4X2DLAO2Q, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:16.688921171 (377h14m43.615215851s) [noise=true , address=104.36.82.6:4875 , piece_id=C275F2JPVAF52E34CEHT5QXCHUFGIQZ7XGMOAOUOT4RA7YOHRQLQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:27.177893955 (377h14m54.104188635s) [noise=true , address=104.36.82.2:4732 , piece_id=DJIGHJMJRAULH2L4YT2WGRXPRQJMNKZG5LICHPSKOCHUL6QDV6BQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:27.177985584 (377h14m54.104280281s) [seg_pos={0 0}, upload=302109]: Segment reached optimal threshold of 13 pieces.
Jul 9 02:26:27.178189789 (377h14m54.104484461s) [noise=true , address=104.36.82.5:4838 , piece_id=WK3ALBOEMWWTVEA4FKCS352SZ6YYB35LELURNWXQZK32K4HYDRNA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178252042 (377h14m54.104546733s) [noise=true , address=104.36.82.7:4909 , piece_id=ZRHLWDQ5KIDRWPJ5FXAANQTWERKXCR2PPEGN442IOSYSRFAG7VZA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178273922 (377h14m54.104568608s) [noise=true , address=104.36.82.10:4078 , piece_id=ZFVUQAEV6DDFLPKCZA2XJ2BZHMMM745LGB4JRZJZQPG72WECGMDQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178305983 (377h14m54.104600682s) [seg_pos={0 0}, upload=302109]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 02:26:27.178428768 (377h14m54.104723465s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:26:27.178455810 (377h14m54.104750501s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:26:27.232166184 (377h14m54.158460883s) [upload=302109]: Done uploading segments...
Jul 9 02:26:27.232188444 (377h14m54.158483123s) [upload=302109]: Upload finished. err: <nil>
Duration: 452.609596ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
36 out of 60 is failed (threshold=3) [XX.XXX.XX.XXXXXX.XXXXX.XX.....X..XXXXX.XX..XXX.XX.XX..X.....] duration:
754.406484ms
FAIL (2025-07-09 03:00:47 UTC)
operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 754.432465ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
37 out of 60 is failed (threshold=3) [XXX.XXX.XX.XXXXXX.XXXXX.XX.....X..XXXXX.XX..XXX.XX.XX..X....] duration:
930.641176ms
FAIL (2025-07-09 03:00:21 UTC)
operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 930.666124ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
23 out of 60 is failed (threshold=3) [.XX..X.X.X.....XXX.XX.X...XXX....XX...X..X..X.XX.X..X.......] duration:
824.313021ms
FAIL (2025-07-09 03:00:18 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 824.336508ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
37 out of 60 is failed (threshold=3) [.XXX.XXX.XX.XXXXXX.XXXXX.XX.....X..XXXXX.XX..XXX.XX.XX..X...] duration:
960.208865ms
FAIL (2025-07-09 03:00:02 UTC)
operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 960.237174ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
445.363711ms
FAIL (2025-07-09 02:59:55 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
3:56.263910643 (377h2m23.19020535s) [upload=301954]: Got next segment. Inline: false
Jul 9 02:13:56.263954358 (377h2m23.190249034s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:13:56.263959891 (377h2m23.190254572s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:13:56.302204249 (377h2m23.228498944s) [seg_pos={0 0}, upload=301954]: Begin upload segment...
Jul 9 02:13:56.302291587 (377h2m23.22858628s) [seg_pos={0 0}, upload=301954]: Done begin upload segment.
Jul 9 02:13:56.302338947 (377h2m23.228633682s) [noise=true , address=77.90.31.14:4689 , piece_id=LW7OT74AJCCC3TEKY6OZI553YK6YQFQ6C7PORQB53E7FUS24KQKQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.302352599 (377h2m23.228647272s) [noise=true , address=104.36.82.8:4986 , piece_id=KPY5JVLSAEB5GUKUJJSIF5M6IIHTJISEYYAXCGUSJ7OLI5QCIBNQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.302450837 (377h2m23.228745531s) [noise=true , address=77.90.31.16:4739 , piece_id=H6UCV6RURPXKC2H3YGAT2OXV2VZMW2DWIRG6TFD6CVXKOVMDP2VQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.302958774 (377h2m23.229253667s) [noise=true , address=77.90.31.13:4640 , piece_id=TPPN75DX3UCBM346CSVJN3GSOA47QV4FVPYSSAHWCJYEM42OCVLQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303069086 (377h2m23.229363782s) [noise=true , address=104.36.82.17:4345 , piece_id=YI27O5WLBYO3TPCX4MVUATFORY6U3EQMDKQHGKRHIZMTG7WGFB2A, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303495994 (377h2m23.22979069s) [noise=true , address=104.36.82.36:4644 , piece_id=K2G3MAPD7TWYZPKNHQBMEPJRIQDLE75FYVPAZFX4I3GJ4IUAA7XQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303575840 (377h2m23.229870536s) [noise=true , address=77.90.31.21:4898 , piece_id=75XXVNT4EV4D5UTX7O5XFL44IZOYY5LZSI2VPFQZPRERL6W5M23A, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303949062 (377h2m23.230245641s) [noise=true , address=79.127.201.215:4339 , piece_id=XDCTYE2SVI6RAPCXTRVANFCZYWJDZXLBHHJ47NQY6V7D36DEFPGA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304060410 (377h2m23.230355106s) [noise=true , address=104.36.82.41:4768 , piece_id=BGMF33OKUZSZKGIHJYI3RNXEBDE54TX67DYLJ7JCZMRNK6ZAU22Q, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304407993 (377h2m23.230702685s) [noise=true , address=77.90.31.10:4525 , piece_id=GPVSIZIBZWTUCP5RFYVPFYIYIL2LWU5PU3UASQVRQLY7SSSFMQBA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304548115 (377h2m23.230842812s) [noise=true , address=104.36.82.23:4464 , piece_id=JCJ4MGEZIIZ7YCTM6QXC5ZJ7PI4OEJBOMHZYBU57D2SNGWJWB5CQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304795164 (377h2m23.231089862s) [noise=true , address=77.90.31.18:4838 , piece_id=TGWYXA3AF7LDG4KPVCRNEYFDADAG6TQ5DAX7JJVW5BLVVVNECILA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305012510 (377h2m23.231307211s) [noise=true , address=104.36.82.34:4554 , piece_id=57K5SDV3YEGLVMSJGSLXNKEWOLETXE6CWBM2NPVCNKLFHPVQCJBQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305113737 (377h2m23.231408427s) [noise=true , address=104.36.82.4:4834 , piece_id=YLYIASUPNRTK7KOMRYLN7OQLQBPWC7WFGCPDGUVIOEQJWQQ7A7GA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305383755 (377h2m23.231678434s) [noise=true , address=104.36.82.38:4698 , piece_id=TSOXJNQEWODCWFEITH4IYKZI7A75UZEXRQPIQO7LMP6LT3WLNXUA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305483822 (377h2m23.231778517s) [noise=true , address=104.36.82.42:4830 , piece_id=ENYQTUCIF7OWQPCXTYS24OC5QT63ZJJXOY652GLIG7OQPJEQCGGA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305954019 (377h2m23.232248718s) [noise=true , address=77.90.31.21:4907 , piece_id=2EUWATHUQNZ77BJV65ZFKR5WL4D3AUURLBQ32XR2IVBA7P4TITOQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.306425517 (377h2m23.232720213s) [noise=true , address=104.36.82.22:4541 , piece_id=XN4HDFDNZ4YUJTDPOTMMDFX4FE2YVZDGCIMHEMJGWXRPNHQ6EU7A, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.375470790 (377h2m23.301765481s) [upload=301954]: Next returned nil segment
Jul 9 02:13:56.375489934 (377h2m23.30178463s) [upload=301954]: Waiting for error group managing segments...
Jul 9 02:13:56.381456859 (377h2m23.307751539s) [noise=true , address=77.90.31.10:4525 , piece_id=GPVSIZIBZWTUCP5RFYVPFYIYIL2LWU5PU3UASQVRQLY7SSSFMQBA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.381956907 (377h2m23.308251586s) [noise=true , address=77.90.31.14:4689 , piece_id=LW7OT74AJCCC3TEKY6OZI553YK6YQFQ6C7PORQB53E7FUS24KQKQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.382837088 (377h2m23.309131761s) [noise=true , address=77.90.31.21:4898 , piece_id=75XXVNT4EV4D5UTX7O5XFL44IZOYY5LZSI2VPFQZPRERL6W5M23A, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.393489125 (377h2m23.319783812s) [noise=true , address=77.90.31.21:4907 , piece_id=2EUWATHUQNZ77BJV65ZFKR5WL4D3AUURLBQ32XR2IVBA7P4TITOQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.404692801 (377h2m23.330987481s) [noise=true , address=77.90.31.16:4739 , piece_id=H6UCV6RURPXKC2H3YGAT2OXV2VZMW2DWIRG6TFD6CVXKOVMDP2VQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.404986115 (377h2m23.331280795s) [noise=true , address=77.90.31.18:4838 , piece_id=TGWYXA3AF7LDG4KPVCRNEYFDADAG6TQ5DAX7JJVW5BLVVVNECILA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.433785086 (377h2m23.360079766s) [noise=true , address=77.90.31.13:4640 , piece_id=TPPN75DX3UCBM346CSVJN3GSOA47QV4FVPYSSAHWCJYEM42OCVLQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.598499747 (377h2m23.524794427s) [noise=true , address=79.127.201.215:4339 , piece_id=XDCTYE2SVI6RAPCXTRVANFCZYWJDZXLBHHJ47NQY6V7D36DEFPGA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:11.107785584 (377h2m38.034080273s) [noise=true , address=104.36.82.22:4541 , piece_id=XN4HDFDNZ4YUJTDPOTMMDFX4FE2YVZDGCIMHEMJGWXRPNHQ6EU7A, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:12.515567029 (377h2m39.441861713s) [noise=true , address=104.36.82.17:4345 , piece_id=YI27O5WLBYO3TPCX4MVUATFORY6U3EQMDKQHGKRHIZMTG7WGFB2A, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:12.923090587 (377h2m39.849385267s) [noise=true , address=104.36.82.8:4986 , piece_id=KPY5JVLSAEB5GUKUJJSIF5M6IIHTJISEYYAXCGUSJ7OLI5QCIBNQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:13.453172659 (377h2m40.379467346s) [noise=true , address=104.36.82.38:4698 , piece_id=TSOXJNQEWODCWFEITH4IYKZI7A75UZEXRQPIQO7LMP6LT3WLNXUA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:14.416524856 (377h2m41.342819534s) [noise=true , address=104.36.82.34:4554 , piece_id=57K5SDV3YEGLVMSJGSLXNKEWOLETXE6CWBM2NPVCNKLFHPVQCJBQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:14.846730699 (377h2m41.773025379s) [noise=true , address=104.36.82.23:4464 , piece_id=JCJ4MGEZIIZ7YCTM6QXC5ZJ7PI4OEJBOMHZYBU57D2SNGWJWB5CQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:14.846763777 (377h2m41.773058463s) [seg_pos={0 0}, upload=301954]: Segment reached optimal threshold of 14 pieces.
Jul 9 02:14:14.846894905 (377h2m41.773189582s) [noise=true , address=104.36.82.36:4644 , piece_id=K2G3MAPD7TWYZPKNHQBMEPJRIQDLE75FYVPAZFX4I3GJ4IUAA7XQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1oYHbb73ACitbfkkP5P6cmRyTt6kmaS1y49G4vbsvTvVDwByhT): protocol: expected piece hash; context canceled; EOF
Jul 9 02:14:14.846948435 (377h2m41.773243131s) [noise=true , address=104.36.82.4:4834 , piece_id=YLYIASUPNRTK7KOMRYLN7OQLQBPWC7WFGCPDGUVIOEQJWQQ7A7GA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Z241DCYA5Wig2TdEogxTpY6FzVmFbnKtmHcTwxLxfcZafgooy): protocol: expected piece hash; context canceled; EOF
Jul 9 02:14:14.847094561 (377h2m41.773389249s) [noise=true , address=104.36.82.42:4830 , piece_id=ENYQTUCIF7OWQPCXTYS24OC5QT63ZJJXOY652GLIG7OQPJEQCGGA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:142ocSHHtv8BpSebXFNk5qk1Y23a8pbFAZqE3cq6i3RMX3dPw9): protocol: expected piece hash; context canceled; EOF
Jul 9 02:14:14.847127825 (377h2m41.773422511s) [noise=true , address=104.36.82.41:4768 , piece_id=BGMF33OKUZSZKGIHJYI3RNXEBDE54TX67DYLJ7JCZMRNK6ZAU22Q, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: write tcp 159.223.143.39:43796->104.36.82.41:4768: use of closed network connection; write tcp 159.223.143.39:43796->104.36.82.41:4768: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:43796->104.36.82.41:4768: use of closed network connection
Jul 9 02:14:14.847142999 (377h2m41.773437687s) [seg_pos={0 0}, upload=301954]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jul 9 02:14:14.847204962 (377h2m41.773499652s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:14:14.847209974 (377h2m41.773504671s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:14:14.909581364 (377h2m41.83587606s) [upload=301954]: Done uploading segments...
Jul 9 02:14:14.909605483 (377h2m41.835900172s) [upload=301954]: Upload finished. err: <nil>
Duration: 445.393923ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........X..................................................] duration:
413.834715ms
FAIL (2025-07-09 02:59:44 UTC)
orbiter warning:
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
02:25:27.135824454 (377h13m54.062119141s) [noise=true , address=104.36.82.12:4150 , piece_id=DRG5SBA3EE4KVQPEQOJZ6SS537SFWYFUYQXOAQMWYTT6FPVSJIPQ, seg_pos={0 0}, upload=302099]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: context canceled; context canceled
Jul 9 02:25:27.135860079 (377h13m54.062154769s) [seg_pos={0 0}, upload=302099]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 02:25:27.135957283 (377h13m54.062251971s) [upload=302099]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:25:27.135964211 (377h13m54.062258897s) [upload=302099]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:25:27.199410713 (377h13m54.125705389s) [upload=302099]: Done uploading segments...
Jul 9 02:25:27.199427901 (377h13m54.125722577s) [upload=302099]: Upload finished. err: <nil>
Jul 9 02:25:57.289077655 (377h14m24.215372329s) [upload=302109]: Starting upload
Jul 9 02:25:57.289186750 (377h14m24.215481424s) [upload=302109]: Done starting upload
Jul 9 02:25:57.289444851 (377h14m24.215739548s) [upload=302109]: Uploading segments...
Jul 9 02:25:57.289623846 (377h14m24.215918535s) [upload=302109]: Got next segment. Inline: false
Jul 9 02:25:57.289696741 (377h14m24.215991432s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:25:57.289703375 (377h14m24.215998067s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:25:57.318313152 (377h14m24.244607838s) [seg_pos={0 0}, upload=302109]: Begin upload segment...
Jul 9 02:25:57.318363286 (377h14m24.244657964s) [seg_pos={0 0}, upload=302109]: Done begin upload segment.
Jul 9 02:25:57.318386772 (377h14m24.244681463s) [noise=true , address=77.90.31.11:4553 , piece_id=TYW7NBJWACIT7OQS7A4WGXUPR3MB2FKAN2OMGAZJERB4X2DLAO2Q, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318412127 (377h14m24.244706825s) [noise=true , address=77.90.31.10:4518 , piece_id=VJ4HWM2X4VSRSUCQL5QLMBYPFBYV4JYS5ZXCBCB6MTMB76EHZFGQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318822149 (377h14m24.245116841s) [noise=true , address=45.91.116.9:4263 , piece_id=5YILAJ5BZBGOURSZE343ST4RMOTWM6ZOB3EEQQWL5WUNYWOGSAPA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318965862 (377h14m24.24526056s) [noise=true , address=45.91.116.8:4045 , piece_id=KM6ZYTHR6JGTARKZQZYIESPKBEU5H2HQFA7OARIKD2ZXDHPFSMGA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.319187516 (377h14m24.245482203s) [noise=true , address=104.36.82.7:4909 , piece_id=ZRHLWDQ5KIDRWPJ5FXAANQTWERKXCR2PPEGN442IOSYSRFAG7VZA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.319523815 (377h14m24.245818512s) [noise=true , address=45.91.116.14:4191 , piece_id=GTR6T4OTGZTEZ76FWQZ46OCH5WB3O5X27EN4AV33ZJ7D5HX4535A, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.320556632 (377h14m24.24685133s) [noise=true , address=104.36.82.2:4732 , piece_id=DJIGHJMJRAULH2L4YT2WGRXPRQJMNKZG5LICHPSKOCHUL6QDV6BQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.321019141 (377h14m24.247313835s) [noise=true , address=77.90.31.17:4770 , piece_id=V36UBBIYOFJC6KFVVBXEVT4QBUIWN257D3YSDDRAGZVXH7ZXHTBA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.322591797 (377h14m24.248886493s) [noise=true , address=77.90.31.13:4627 , piece_id=HULEKFVZNDNKS4SWWGTJUHYZWWYPDCER5W4GFCVLLDL65PAHULFQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323120451 (377h14m24.249415138s) [noise=true , address=45.91.116.4:4939 , piece_id=ANA3EZU7OZWELNG4QEMTEAYZY5H3C3H4IJFT4XKPUB7WIF44FEVQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323544591 (377h14m24.249839288s) [noise=true , address=45.91.116.7:4011 , piece_id=VDOJRN6ACK4LO44BQTULO6BB3CLAVQI7YIC6XLR32CZV4UJSPE6Q, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323950189 (377h14m24.250244889s) [noise=true , address=104.36.82.10:4078 , piece_id=ZFVUQAEV6DDFLPKCZA2XJ2BZHMMM745LGB4JRZJZQPG72WECGMDQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.326585810 (377h14m24.252880507s) [noise=true , address=45.91.116.5:4297 , piece_id=MSDWDLY5PJHQVI5565EKANJVSTMZFEUD2OTYZ52C65XQUICNMC3A, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.327871214 (377h14m24.254165904s) [noise=true , address=104.36.82.6:4875 , piece_id=C275F2JPVAF52E34CEHT5QXCHUFGIQZ7XGMOAOUOT4RA7YOHRQLQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.328605288 (377h14m24.254899986s) [noise=true , address=104.36.82.5:4838 , piece_id=WK3ALBOEMWWTVEA4FKCS352SZ6YYB35LELURNWXQZK32K4HYDRNA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.335409087 (377h14m24.261703784s) [noise=true , address=45.91.116.6:4973 , piece_id=Z7HQQ2O5QUSJKT4EMIDMEI5PIAEHNVRYX7MALUR2GQULRFIBUOKA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.475344637 (377h14m24.401639326s) [upload=302109]: Next returned nil segment
Jul 9 02:25:57.475376181 (377h14m24.401670879s) [upload=302109]: Waiting for error group managing segments...
Jul 9 02:25:57.477269278 (377h14m24.403563958s) [noise=true , address=45.91.116.7:4011 , piece_id=VDOJRN6ACK4LO44BQTULO6BB3CLAVQI7YIC6XLR32CZV4UJSPE6Q, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.477736179 (377h14m24.40403086s) [noise=true , address=45.91.116.9:4263 , piece_id=5YILAJ5BZBGOURSZE343ST4RMOTWM6ZOB3EEQQWL5WUNYWOGSAPA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.478396652 (377h14m24.404691332s) [noise=true , address=77.90.31.10:4518 , piece_id=VJ4HWM2X4VSRSUCQL5QLMBYPFBYV4JYS5ZXCBCB6MTMB76EHZFGQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.478919538 (377h14m24.405214224s) [noise=true , address=45.91.116.14:4191 , piece_id=GTR6T4OTGZTEZ76FWQZ46OCH5WB3O5X27EN4AV33ZJ7D5HX4535A, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481426716 (377h14m24.407721396s) [noise=true , address=45.91.116.4:4939 , piece_id=ANA3EZU7OZWELNG4QEMTEAYZY5H3C3H4IJFT4XKPUB7WIF44FEVQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481718373 (377h14m24.408013051s) [noise=true , address=45.91.116.5:4297 , piece_id=MSDWDLY5PJHQVI5565EKANJVSTMZFEUD2OTYZ52C65XQUICNMC3A, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481930356 (377h14m24.408225033s) [noise=true , address=77.90.31.17:4770 , piece_id=V36UBBIYOFJC6KFVVBXEVT4QBUIWN257D3YSDDRAGZVXH7ZXHTBA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.482376147 (377h14m24.408670821s) [noise=true , address=45.91.116.8:4045 , piece_id=KM6ZYTHR6JGTARKZQZYIESPKBEU5H2HQFA7OARIKD2ZXDHPFSMGA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.482439192 (377h14m24.408733868s) [noise=true , address=45.91.116.6:4973 , piece_id=Z7HQQ2O5QUSJKT4EMIDMEI5PIAEHNVRYX7MALUR2GQULRFIBUOKA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.483047970 (377h14m24.409342644s) [noise=true , address=77.90.31.13:4627 , piece_id=HULEKFVZNDNKS4SWWGTJUHYZWWYPDCER5W4GFCVLLDL65PAHULFQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.497788632 (377h14m24.424083312s) [noise=true , address=77.90.31.11:4553 , piece_id=TYW7NBJWACIT7OQS7A4WGXUPR3MB2FKAN2OMGAZJERB4X2DLAO2Q, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:16.688921171 (377h14m43.615215851s) [noise=true , address=104.36.82.6:4875 , piece_id=C275F2JPVAF52E34CEHT5QXCHUFGIQZ7XGMOAOUOT4RA7YOHRQLQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:27.177893955 (377h14m54.104188635s) [noise=true , address=104.36.82.2:4732 , piece_id=DJIGHJMJRAULH2L4YT2WGRXPRQJMNKZG5LICHPSKOCHUL6QDV6BQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:27.177985584 (377h14m54.104280281s) [seg_pos={0 0}, upload=302109]: Segment reached optimal threshold of 13 pieces.
Jul 9 02:26:27.178189789 (377h14m54.104484461s) [noise=true , address=104.36.82.5:4838 , piece_id=WK3ALBOEMWWTVEA4FKCS352SZ6YYB35LELURNWXQZK32K4HYDRNA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178252042 (377h14m54.104546733s) [noise=true , address=104.36.82.7:4909 , piece_id=ZRHLWDQ5KIDRWPJ5FXAANQTWERKXCR2PPEGN442IOSYSRFAG7VZA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178273922 (377h14m54.104568608s) [noise=true , address=104.36.82.10:4078 , piece_id=ZFVUQAEV6DDFLPKCZA2XJ2BZHMMM745LGB4JRZJZQPG72WECGMDQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178305983 (377h14m54.104600682s) [seg_pos={0 0}, upload=302109]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 02:26:27.178428768 (377h14m54.104723465s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:26:27.178455810 (377h14m54.104750501s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:26:27.232166184 (377h14m54.158460883s) [upload=302109]: Done uploading segments...
Jul 9 02:26:27.232188444 (377h14m54.158483123s) [upload=302109]: Upload finished. err: <nil>
Duration: 413.859469ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
38 out of 60 is failed (threshold=3) [X.XXX.XXX.XX.XXXXXX.XXXXX.XX.....X..XXXXX.XX..XXX.XX.XX..X..] duration:
1.056528515s
FAIL (2025-07-09 02:59:39 UTC)
operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.05654826s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
24 out of 60 is failed (threshold=3) [X.XX..X.X.X.....XXX.XX.X...XXX....XX...X..X..X.XX.X..X......] duration:
581.597259ms
FAIL (2025-07-09 02:59:35 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 581.615561ms
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........X.................................................] duration:
418.330324ms
FAIL (2025-07-09 02:59:24 UTC)
orbiter warning:
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
02:25:27.135824454 (377h13m54.062119141s) [noise=true , address=104.36.82.12:4150 , piece_id=DRG5SBA3EE4KVQPEQOJZ6SS537SFWYFUYQXOAQMWYTT6FPVSJIPQ, seg_pos={0 0}, upload=302099]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: context canceled; context canceled
Jul 9 02:25:27.135860079 (377h13m54.062154769s) [seg_pos={0 0}, upload=302099]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 02:25:27.135957283 (377h13m54.062251971s) [upload=302099]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:25:27.135964211 (377h13m54.062258897s) [upload=302099]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:25:27.199410713 (377h13m54.125705389s) [upload=302099]: Done uploading segments...
Jul 9 02:25:27.199427901 (377h13m54.125722577s) [upload=302099]: Upload finished. err: <nil>
Jul 9 02:25:57.289077655 (377h14m24.215372329s) [upload=302109]: Starting upload
Jul 9 02:25:57.289186750 (377h14m24.215481424s) [upload=302109]: Done starting upload
Jul 9 02:25:57.289444851 (377h14m24.215739548s) [upload=302109]: Uploading segments...
Jul 9 02:25:57.289623846 (377h14m24.215918535s) [upload=302109]: Got next segment. Inline: false
Jul 9 02:25:57.289696741 (377h14m24.215991432s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:25:57.289703375 (377h14m24.215998067s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:25:57.318313152 (377h14m24.244607838s) [seg_pos={0 0}, upload=302109]: Begin upload segment...
Jul 9 02:25:57.318363286 (377h14m24.244657964s) [seg_pos={0 0}, upload=302109]: Done begin upload segment.
Jul 9 02:25:57.318386772 (377h14m24.244681463s) [noise=true , address=77.90.31.11:4553 , piece_id=TYW7NBJWACIT7OQS7A4WGXUPR3MB2FKAN2OMGAZJERB4X2DLAO2Q, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318412127 (377h14m24.244706825s) [noise=true , address=77.90.31.10:4518 , piece_id=VJ4HWM2X4VSRSUCQL5QLMBYPFBYV4JYS5ZXCBCB6MTMB76EHZFGQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318822149 (377h14m24.245116841s) [noise=true , address=45.91.116.9:4263 , piece_id=5YILAJ5BZBGOURSZE343ST4RMOTWM6ZOB3EEQQWL5WUNYWOGSAPA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318965862 (377h14m24.24526056s) [noise=true , address=45.91.116.8:4045 , piece_id=KM6ZYTHR6JGTARKZQZYIESPKBEU5H2HQFA7OARIKD2ZXDHPFSMGA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.319187516 (377h14m24.245482203s) [noise=true , address=104.36.82.7:4909 , piece_id=ZRHLWDQ5KIDRWPJ5FXAANQTWERKXCR2PPEGN442IOSYSRFAG7VZA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.319523815 (377h14m24.245818512s) [noise=true , address=45.91.116.14:4191 , piece_id=GTR6T4OTGZTEZ76FWQZ46OCH5WB3O5X27EN4AV33ZJ7D5HX4535A, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.320556632 (377h14m24.24685133s) [noise=true , address=104.36.82.2:4732 , piece_id=DJIGHJMJRAULH2L4YT2WGRXPRQJMNKZG5LICHPSKOCHUL6QDV6BQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.321019141 (377h14m24.247313835s) [noise=true , address=77.90.31.17:4770 , piece_id=V36UBBIYOFJC6KFVVBXEVT4QBUIWN257D3YSDDRAGZVXH7ZXHTBA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.322591797 (377h14m24.248886493s) [noise=true , address=77.90.31.13:4627 , piece_id=HULEKFVZNDNKS4SWWGTJUHYZWWYPDCER5W4GFCVLLDL65PAHULFQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323120451 (377h14m24.249415138s) [noise=true , address=45.91.116.4:4939 , piece_id=ANA3EZU7OZWELNG4QEMTEAYZY5H3C3H4IJFT4XKPUB7WIF44FEVQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323544591 (377h14m24.249839288s) [noise=true , address=45.91.116.7:4011 , piece_id=VDOJRN6ACK4LO44BQTULO6BB3CLAVQI7YIC6XLR32CZV4UJSPE6Q, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323950189 (377h14m24.250244889s) [noise=true , address=104.36.82.10:4078 , piece_id=ZFVUQAEV6DDFLPKCZA2XJ2BZHMMM745LGB4JRZJZQPG72WECGMDQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.326585810 (377h14m24.252880507s) [noise=true , address=45.91.116.5:4297 , piece_id=MSDWDLY5PJHQVI5565EKANJVSTMZFEUD2OTYZ52C65XQUICNMC3A, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.327871214 (377h14m24.254165904s) [noise=true , address=104.36.82.6:4875 , piece_id=C275F2JPVAF52E34CEHT5QXCHUFGIQZ7XGMOAOUOT4RA7YOHRQLQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.328605288 (377h14m24.254899986s) [noise=true , address=104.36.82.5:4838 , piece_id=WK3ALBOEMWWTVEA4FKCS352SZ6YYB35LELURNWXQZK32K4HYDRNA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.335409087 (377h14m24.261703784s) [noise=true , address=45.91.116.6:4973 , piece_id=Z7HQQ2O5QUSJKT4EMIDMEI5PIAEHNVRYX7MALUR2GQULRFIBUOKA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.475344637 (377h14m24.401639326s) [upload=302109]: Next returned nil segment
Jul 9 02:25:57.475376181 (377h14m24.401670879s) [upload=302109]: Waiting for error group managing segments...
Jul 9 02:25:57.477269278 (377h14m24.403563958s) [noise=true , address=45.91.116.7:4011 , piece_id=VDOJRN6ACK4LO44BQTULO6BB3CLAVQI7YIC6XLR32CZV4UJSPE6Q, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.477736179 (377h14m24.40403086s) [noise=true , address=45.91.116.9:4263 , piece_id=5YILAJ5BZBGOURSZE343ST4RMOTWM6ZOB3EEQQWL5WUNYWOGSAPA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.478396652 (377h14m24.404691332s) [noise=true , address=77.90.31.10:4518 , piece_id=VJ4HWM2X4VSRSUCQL5QLMBYPFBYV4JYS5ZXCBCB6MTMB76EHZFGQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.478919538 (377h14m24.405214224s) [noise=true , address=45.91.116.14:4191 , piece_id=GTR6T4OTGZTEZ76FWQZ46OCH5WB3O5X27EN4AV33ZJ7D5HX4535A, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481426716 (377h14m24.407721396s) [noise=true , address=45.91.116.4:4939 , piece_id=ANA3EZU7OZWELNG4QEMTEAYZY5H3C3H4IJFT4XKPUB7WIF44FEVQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481718373 (377h14m24.408013051s) [noise=true , address=45.91.116.5:4297 , piece_id=MSDWDLY5PJHQVI5565EKANJVSTMZFEUD2OTYZ52C65XQUICNMC3A, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481930356 (377h14m24.408225033s) [noise=true , address=77.90.31.17:4770 , piece_id=V36UBBIYOFJC6KFVVBXEVT4QBUIWN257D3YSDDRAGZVXH7ZXHTBA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.482376147 (377h14m24.408670821s) [noise=true , address=45.91.116.8:4045 , piece_id=KM6ZYTHR6JGTARKZQZYIESPKBEU5H2HQFA7OARIKD2ZXDHPFSMGA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.482439192 (377h14m24.408733868s) [noise=true , address=45.91.116.6:4973 , piece_id=Z7HQQ2O5QUSJKT4EMIDMEI5PIAEHNVRYX7MALUR2GQULRFIBUOKA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.483047970 (377h14m24.409342644s) [noise=true , address=77.90.31.13:4627 , piece_id=HULEKFVZNDNKS4SWWGTJUHYZWWYPDCER5W4GFCVLLDL65PAHULFQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.497788632 (377h14m24.424083312s) [noise=true , address=77.90.31.11:4553 , piece_id=TYW7NBJWACIT7OQS7A4WGXUPR3MB2FKAN2OMGAZJERB4X2DLAO2Q, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:16.688921171 (377h14m43.615215851s) [noise=true , address=104.36.82.6:4875 , piece_id=C275F2JPVAF52E34CEHT5QXCHUFGIQZ7XGMOAOUOT4RA7YOHRQLQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:27.177893955 (377h14m54.104188635s) [noise=true , address=104.36.82.2:4732 , piece_id=DJIGHJMJRAULH2L4YT2WGRXPRQJMNKZG5LICHPSKOCHUL6QDV6BQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:27.177985584 (377h14m54.104280281s) [seg_pos={0 0}, upload=302109]: Segment reached optimal threshold of 13 pieces.
Jul 9 02:26:27.178189789 (377h14m54.104484461s) [noise=true , address=104.36.82.5:4838 , piece_id=WK3ALBOEMWWTVEA4FKCS352SZ6YYB35LELURNWXQZK32K4HYDRNA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178252042 (377h14m54.104546733s) [noise=true , address=104.36.82.7:4909 , piece_id=ZRHLWDQ5KIDRWPJ5FXAANQTWERKXCR2PPEGN442IOSYSRFAG7VZA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178273922 (377h14m54.104568608s) [noise=true , address=104.36.82.10:4078 , piece_id=ZFVUQAEV6DDFLPKCZA2XJ2BZHMMM745LGB4JRZJZQPG72WECGMDQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178305983 (377h14m54.104600682s) [seg_pos={0 0}, upload=302109]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 02:26:27.178428768 (377h14m54.104723465s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:26:27.178455810 (377h14m54.104750501s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:26:27.232166184 (377h14m54.158460883s) [upload=302109]: Done uploading segments...
Jul 9 02:26:27.232188444 (377h14m54.158483123s) [upload=302109]: Upload finished. err: <nil>
Duration: 418.369847ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........X................................................] duration:
529.209671ms
FAIL (2025-07-09 02:59:03 UTC)
orbiter warning:
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
02:25:27.135824454 (377h13m54.062119141s) [noise=true , address=104.36.82.12:4150 , piece_id=DRG5SBA3EE4KVQPEQOJZ6SS537SFWYFUYQXOAQMWYTT6FPVSJIPQ, seg_pos={0 0}, upload=302099]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: context canceled; context canceled
Jul 9 02:25:27.135860079 (377h13m54.062154769s) [seg_pos={0 0}, upload=302099]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 02:25:27.135957283 (377h13m54.062251971s) [upload=302099]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:25:27.135964211 (377h13m54.062258897s) [upload=302099]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:25:27.199410713 (377h13m54.125705389s) [upload=302099]: Done uploading segments...
Jul 9 02:25:27.199427901 (377h13m54.125722577s) [upload=302099]: Upload finished. err: <nil>
Jul 9 02:25:57.289077655 (377h14m24.215372329s) [upload=302109]: Starting upload
Jul 9 02:25:57.289186750 (377h14m24.215481424s) [upload=302109]: Done starting upload
Jul 9 02:25:57.289444851 (377h14m24.215739548s) [upload=302109]: Uploading segments...
Jul 9 02:25:57.289623846 (377h14m24.215918535s) [upload=302109]: Got next segment. Inline: false
Jul 9 02:25:57.289696741 (377h14m24.215991432s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:25:57.289703375 (377h14m24.215998067s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:25:57.318313152 (377h14m24.244607838s) [seg_pos={0 0}, upload=302109]: Begin upload segment...
Jul 9 02:25:57.318363286 (377h14m24.244657964s) [seg_pos={0 0}, upload=302109]: Done begin upload segment.
Jul 9 02:25:57.318386772 (377h14m24.244681463s) [noise=true , address=77.90.31.11:4553 , piece_id=TYW7NBJWACIT7OQS7A4WGXUPR3MB2FKAN2OMGAZJERB4X2DLAO2Q, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318412127 (377h14m24.244706825s) [noise=true , address=77.90.31.10:4518 , piece_id=VJ4HWM2X4VSRSUCQL5QLMBYPFBYV4JYS5ZXCBCB6MTMB76EHZFGQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318822149 (377h14m24.245116841s) [noise=true , address=45.91.116.9:4263 , piece_id=5YILAJ5BZBGOURSZE343ST4RMOTWM6ZOB3EEQQWL5WUNYWOGSAPA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318965862 (377h14m24.24526056s) [noise=true , address=45.91.116.8:4045 , piece_id=KM6ZYTHR6JGTARKZQZYIESPKBEU5H2HQFA7OARIKD2ZXDHPFSMGA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.319187516 (377h14m24.245482203s) [noise=true , address=104.36.82.7:4909 , piece_id=ZRHLWDQ5KIDRWPJ5FXAANQTWERKXCR2PPEGN442IOSYSRFAG7VZA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.319523815 (377h14m24.245818512s) [noise=true , address=45.91.116.14:4191 , piece_id=GTR6T4OTGZTEZ76FWQZ46OCH5WB3O5X27EN4AV33ZJ7D5HX4535A, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.320556632 (377h14m24.24685133s) [noise=true , address=104.36.82.2:4732 , piece_id=DJIGHJMJRAULH2L4YT2WGRXPRQJMNKZG5LICHPSKOCHUL6QDV6BQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.321019141 (377h14m24.247313835s) [noise=true , address=77.90.31.17:4770 , piece_id=V36UBBIYOFJC6KFVVBXEVT4QBUIWN257D3YSDDRAGZVXH7ZXHTBA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.322591797 (377h14m24.248886493s) [noise=true , address=77.90.31.13:4627 , piece_id=HULEKFVZNDNKS4SWWGTJUHYZWWYPDCER5W4GFCVLLDL65PAHULFQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323120451 (377h14m24.249415138s) [noise=true , address=45.91.116.4:4939 , piece_id=ANA3EZU7OZWELNG4QEMTEAYZY5H3C3H4IJFT4XKPUB7WIF44FEVQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323544591 (377h14m24.249839288s) [noise=true , address=45.91.116.7:4011 , piece_id=VDOJRN6ACK4LO44BQTULO6BB3CLAVQI7YIC6XLR32CZV4UJSPE6Q, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323950189 (377h14m24.250244889s) [noise=true , address=104.36.82.10:4078 , piece_id=ZFVUQAEV6DDFLPKCZA2XJ2BZHMMM745LGB4JRZJZQPG72WECGMDQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.326585810 (377h14m24.252880507s) [noise=true , address=45.91.116.5:4297 , piece_id=MSDWDLY5PJHQVI5565EKANJVSTMZFEUD2OTYZ52C65XQUICNMC3A, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.327871214 (377h14m24.254165904s) [noise=true , address=104.36.82.6:4875 , piece_id=C275F2JPVAF52E34CEHT5QXCHUFGIQZ7XGMOAOUOT4RA7YOHRQLQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.328605288 (377h14m24.254899986s) [noise=true , address=104.36.82.5:4838 , piece_id=WK3ALBOEMWWTVEA4FKCS352SZ6YYB35LELURNWXQZK32K4HYDRNA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.335409087 (377h14m24.261703784s) [noise=true , address=45.91.116.6:4973 , piece_id=Z7HQQ2O5QUSJKT4EMIDMEI5PIAEHNVRYX7MALUR2GQULRFIBUOKA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.475344637 (377h14m24.401639326s) [upload=302109]: Next returned nil segment
Jul 9 02:25:57.475376181 (377h14m24.401670879s) [upload=302109]: Waiting for error group managing segments...
Jul 9 02:25:57.477269278 (377h14m24.403563958s) [noise=true , address=45.91.116.7:4011 , piece_id=VDOJRN6ACK4LO44BQTULO6BB3CLAVQI7YIC6XLR32CZV4UJSPE6Q, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.477736179 (377h14m24.40403086s) [noise=true , address=45.91.116.9:4263 , piece_id=5YILAJ5BZBGOURSZE343ST4RMOTWM6ZOB3EEQQWL5WUNYWOGSAPA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.478396652 (377h14m24.404691332s) [noise=true , address=77.90.31.10:4518 , piece_id=VJ4HWM2X4VSRSUCQL5QLMBYPFBYV4JYS5ZXCBCB6MTMB76EHZFGQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.478919538 (377h14m24.405214224s) [noise=true , address=45.91.116.14:4191 , piece_id=GTR6T4OTGZTEZ76FWQZ46OCH5WB3O5X27EN4AV33ZJ7D5HX4535A, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481426716 (377h14m24.407721396s) [noise=true , address=45.91.116.4:4939 , piece_id=ANA3EZU7OZWELNG4QEMTEAYZY5H3C3H4IJFT4XKPUB7WIF44FEVQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481718373 (377h14m24.408013051s) [noise=true , address=45.91.116.5:4297 , piece_id=MSDWDLY5PJHQVI5565EKANJVSTMZFEUD2OTYZ52C65XQUICNMC3A, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481930356 (377h14m24.408225033s) [noise=true , address=77.90.31.17:4770 , piece_id=V36UBBIYOFJC6KFVVBXEVT4QBUIWN257D3YSDDRAGZVXH7ZXHTBA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.482376147 (377h14m24.408670821s) [noise=true , address=45.91.116.8:4045 , piece_id=KM6ZYTHR6JGTARKZQZYIESPKBEU5H2HQFA7OARIKD2ZXDHPFSMGA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.482439192 (377h14m24.408733868s) [noise=true , address=45.91.116.6:4973 , piece_id=Z7HQQ2O5QUSJKT4EMIDMEI5PIAEHNVRYX7MALUR2GQULRFIBUOKA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.483047970 (377h14m24.409342644s) [noise=true , address=77.90.31.13:4627 , piece_id=HULEKFVZNDNKS4SWWGTJUHYZWWYPDCER5W4GFCVLLDL65PAHULFQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.497788632 (377h14m24.424083312s) [noise=true , address=77.90.31.11:4553 , piece_id=TYW7NBJWACIT7OQS7A4WGXUPR3MB2FKAN2OMGAZJERB4X2DLAO2Q, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:16.688921171 (377h14m43.615215851s) [noise=true , address=104.36.82.6:4875 , piece_id=C275F2JPVAF52E34CEHT5QXCHUFGIQZ7XGMOAOUOT4RA7YOHRQLQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:27.177893955 (377h14m54.104188635s) [noise=true , address=104.36.82.2:4732 , piece_id=DJIGHJMJRAULH2L4YT2WGRXPRQJMNKZG5LICHPSKOCHUL6QDV6BQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:27.177985584 (377h14m54.104280281s) [seg_pos={0 0}, upload=302109]: Segment reached optimal threshold of 13 pieces.
Jul 9 02:26:27.178189789 (377h14m54.104484461s) [noise=true , address=104.36.82.5:4838 , piece_id=WK3ALBOEMWWTVEA4FKCS352SZ6YYB35LELURNWXQZK32K4HYDRNA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178252042 (377h14m54.104546733s) [noise=true , address=104.36.82.7:4909 , piece_id=ZRHLWDQ5KIDRWPJ5FXAANQTWERKXCR2PPEGN442IOSYSRFAG7VZA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178273922 (377h14m54.104568608s) [noise=true , address=104.36.82.10:4078 , piece_id=ZFVUQAEV6DDFLPKCZA2XJ2BZHMMM745LGB4JRZJZQPG72WECGMDQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178305983 (377h14m54.104600682s) [seg_pos={0 0}, upload=302109]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 02:26:27.178428768 (377h14m54.104723465s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:26:27.178455810 (377h14m54.104750501s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:26:27.232166184 (377h14m54.158460883s) [upload=302109]: Done uploading segments...
Jul 9 02:26:27.232188444 (377h14m54.158483123s) [upload=302109]: Upload finished. err: <nil>
Duration: 529.241948ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
39 out of 60 is failed (threshold=3) [XX.XXX.XXX.XX.XXXXXX.XXXXX.XX.....X..XXXXX.XX..XXX.XX.XX..X.] duration:
3.720647567s
FAIL (2025-07-09 02:58:55 UTC)
operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.72065979s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
25 out of 60 is failed (threshold=3) [XX.XX..X.X.X.....XXX.XX.X...XXX....XX...X..X..X.XX.X..X.....] duration:
531.304038ms
FAIL (2025-07-09 02:58:54 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 531.327058ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
39 out of 60 is failed (threshold=3) [.XX.XXX.XXX.XX.XXXXXX.XXXXX.XX.....X..XXXXX.XX..XXX.XX.XX..X] duration:
1m0.001717439s
FAIL (2025-07-09 02:58:51 UTC)
operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1m0.001740756s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................] duration:
502.754244ms
FAIL (2025-07-09 02:58:46 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
3:56.263910643 (377h2m23.19020535s) [upload=301954]: Got next segment. Inline: false
Jul 9 02:13:56.263954358 (377h2m23.190249034s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:13:56.263959891 (377h2m23.190254572s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:13:56.302204249 (377h2m23.228498944s) [seg_pos={0 0}, upload=301954]: Begin upload segment...
Jul 9 02:13:56.302291587 (377h2m23.22858628s) [seg_pos={0 0}, upload=301954]: Done begin upload segment.
Jul 9 02:13:56.302338947 (377h2m23.228633682s) [noise=true , address=77.90.31.14:4689 , piece_id=LW7OT74AJCCC3TEKY6OZI553YK6YQFQ6C7PORQB53E7FUS24KQKQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.302352599 (377h2m23.228647272s) [noise=true , address=104.36.82.8:4986 , piece_id=KPY5JVLSAEB5GUKUJJSIF5M6IIHTJISEYYAXCGUSJ7OLI5QCIBNQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.302450837 (377h2m23.228745531s) [noise=true , address=77.90.31.16:4739 , piece_id=H6UCV6RURPXKC2H3YGAT2OXV2VZMW2DWIRG6TFD6CVXKOVMDP2VQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.302958774 (377h2m23.229253667s) [noise=true , address=77.90.31.13:4640 , piece_id=TPPN75DX3UCBM346CSVJN3GSOA47QV4FVPYSSAHWCJYEM42OCVLQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303069086 (377h2m23.229363782s) [noise=true , address=104.36.82.17:4345 , piece_id=YI27O5WLBYO3TPCX4MVUATFORY6U3EQMDKQHGKRHIZMTG7WGFB2A, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303495994 (377h2m23.22979069s) [noise=true , address=104.36.82.36:4644 , piece_id=K2G3MAPD7TWYZPKNHQBMEPJRIQDLE75FYVPAZFX4I3GJ4IUAA7XQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303575840 (377h2m23.229870536s) [noise=true , address=77.90.31.21:4898 , piece_id=75XXVNT4EV4D5UTX7O5XFL44IZOYY5LZSI2VPFQZPRERL6W5M23A, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303949062 (377h2m23.230245641s) [noise=true , address=79.127.201.215:4339 , piece_id=XDCTYE2SVI6RAPCXTRVANFCZYWJDZXLBHHJ47NQY6V7D36DEFPGA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304060410 (377h2m23.230355106s) [noise=true , address=104.36.82.41:4768 , piece_id=BGMF33OKUZSZKGIHJYI3RNXEBDE54TX67DYLJ7JCZMRNK6ZAU22Q, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304407993 (377h2m23.230702685s) [noise=true , address=77.90.31.10:4525 , piece_id=GPVSIZIBZWTUCP5RFYVPFYIYIL2LWU5PU3UASQVRQLY7SSSFMQBA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304548115 (377h2m23.230842812s) [noise=true , address=104.36.82.23:4464 , piece_id=JCJ4MGEZIIZ7YCTM6QXC5ZJ7PI4OEJBOMHZYBU57D2SNGWJWB5CQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304795164 (377h2m23.231089862s) [noise=true , address=77.90.31.18:4838 , piece_id=TGWYXA3AF7LDG4KPVCRNEYFDADAG6TQ5DAX7JJVW5BLVVVNECILA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305012510 (377h2m23.231307211s) [noise=true , address=104.36.82.34:4554 , piece_id=57K5SDV3YEGLVMSJGSLXNKEWOLETXE6CWBM2NPVCNKLFHPVQCJBQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305113737 (377h2m23.231408427s) [noise=true , address=104.36.82.4:4834 , piece_id=YLYIASUPNRTK7KOMRYLN7OQLQBPWC7WFGCPDGUVIOEQJWQQ7A7GA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305383755 (377h2m23.231678434s) [noise=true , address=104.36.82.38:4698 , piece_id=TSOXJNQEWODCWFEITH4IYKZI7A75UZEXRQPIQO7LMP6LT3WLNXUA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305483822 (377h2m23.231778517s) [noise=true , address=104.36.82.42:4830 , piece_id=ENYQTUCIF7OWQPCXTYS24OC5QT63ZJJXOY652GLIG7OQPJEQCGGA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305954019 (377h2m23.232248718s) [noise=true , address=77.90.31.21:4907 , piece_id=2EUWATHUQNZ77BJV65ZFKR5WL4D3AUURLBQ32XR2IVBA7P4TITOQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.306425517 (377h2m23.232720213s) [noise=true , address=104.36.82.22:4541 , piece_id=XN4HDFDNZ4YUJTDPOTMMDFX4FE2YVZDGCIMHEMJGWXRPNHQ6EU7A, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.375470790 (377h2m23.301765481s) [upload=301954]: Next returned nil segment
Jul 9 02:13:56.375489934 (377h2m23.30178463s) [upload=301954]: Waiting for error group managing segments...
Jul 9 02:13:56.381456859 (377h2m23.307751539s) [noise=true , address=77.90.31.10:4525 , piece_id=GPVSIZIBZWTUCP5RFYVPFYIYIL2LWU5PU3UASQVRQLY7SSSFMQBA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.381956907 (377h2m23.308251586s) [noise=true , address=77.90.31.14:4689 , piece_id=LW7OT74AJCCC3TEKY6OZI553YK6YQFQ6C7PORQB53E7FUS24KQKQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.382837088 (377h2m23.309131761s) [noise=true , address=77.90.31.21:4898 , piece_id=75XXVNT4EV4D5UTX7O5XFL44IZOYY5LZSI2VPFQZPRERL6W5M23A, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.393489125 (377h2m23.319783812s) [noise=true , address=77.90.31.21:4907 , piece_id=2EUWATHUQNZ77BJV65ZFKR5WL4D3AUURLBQ32XR2IVBA7P4TITOQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.404692801 (377h2m23.330987481s) [noise=true , address=77.90.31.16:4739 , piece_id=H6UCV6RURPXKC2H3YGAT2OXV2VZMW2DWIRG6TFD6CVXKOVMDP2VQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.404986115 (377h2m23.331280795s) [noise=true , address=77.90.31.18:4838 , piece_id=TGWYXA3AF7LDG4KPVCRNEYFDADAG6TQ5DAX7JJVW5BLVVVNECILA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.433785086 (377h2m23.360079766s) [noise=true , address=77.90.31.13:4640 , piece_id=TPPN75DX3UCBM346CSVJN3GSOA47QV4FVPYSSAHWCJYEM42OCVLQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.598499747 (377h2m23.524794427s) [noise=true , address=79.127.201.215:4339 , piece_id=XDCTYE2SVI6RAPCXTRVANFCZYWJDZXLBHHJ47NQY6V7D36DEFPGA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:11.107785584 (377h2m38.034080273s) [noise=true , address=104.36.82.22:4541 , piece_id=XN4HDFDNZ4YUJTDPOTMMDFX4FE2YVZDGCIMHEMJGWXRPNHQ6EU7A, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:12.515567029 (377h2m39.441861713s) [noise=true , address=104.36.82.17:4345 , piece_id=YI27O5WLBYO3TPCX4MVUATFORY6U3EQMDKQHGKRHIZMTG7WGFB2A, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:12.923090587 (377h2m39.849385267s) [noise=true , address=104.36.82.8:4986 , piece_id=KPY5JVLSAEB5GUKUJJSIF5M6IIHTJISEYYAXCGUSJ7OLI5QCIBNQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:13.453172659 (377h2m40.379467346s) [noise=true , address=104.36.82.38:4698 , piece_id=TSOXJNQEWODCWFEITH4IYKZI7A75UZEXRQPIQO7LMP6LT3WLNXUA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:14.416524856 (377h2m41.342819534s) [noise=true , address=104.36.82.34:4554 , piece_id=57K5SDV3YEGLVMSJGSLXNKEWOLETXE6CWBM2NPVCNKLFHPVQCJBQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:14.846730699 (377h2m41.773025379s) [noise=true , address=104.36.82.23:4464 , piece_id=JCJ4MGEZIIZ7YCTM6QXC5ZJ7PI4OEJBOMHZYBU57D2SNGWJWB5CQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:14.846763777 (377h2m41.773058463s) [seg_pos={0 0}, upload=301954]: Segment reached optimal threshold of 14 pieces.
Jul 9 02:14:14.846894905 (377h2m41.773189582s) [noise=true , address=104.36.82.36:4644 , piece_id=K2G3MAPD7TWYZPKNHQBMEPJRIQDLE75FYVPAZFX4I3GJ4IUAA7XQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1oYHbb73ACitbfkkP5P6cmRyTt6kmaS1y49G4vbsvTvVDwByhT): protocol: expected piece hash; context canceled; EOF
Jul 9 02:14:14.846948435 (377h2m41.773243131s) [noise=true , address=104.36.82.4:4834 , piece_id=YLYIASUPNRTK7KOMRYLN7OQLQBPWC7WFGCPDGUVIOEQJWQQ7A7GA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Z241DCYA5Wig2TdEogxTpY6FzVmFbnKtmHcTwxLxfcZafgooy): protocol: expected piece hash; context canceled; EOF
Jul 9 02:14:14.847094561 (377h2m41.773389249s) [noise=true , address=104.36.82.42:4830 , piece_id=ENYQTUCIF7OWQPCXTYS24OC5QT63ZJJXOY652GLIG7OQPJEQCGGA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:142ocSHHtv8BpSebXFNk5qk1Y23a8pbFAZqE3cq6i3RMX3dPw9): protocol: expected piece hash; context canceled; EOF
Jul 9 02:14:14.847127825 (377h2m41.773422511s) [noise=true , address=104.36.82.41:4768 , piece_id=BGMF33OKUZSZKGIHJYI3RNXEBDE54TX67DYLJ7JCZMRNK6ZAU22Q, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: write tcp 159.223.143.39:43796->104.36.82.41:4768: use of closed network connection; write tcp 159.223.143.39:43796->104.36.82.41:4768: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:43796->104.36.82.41:4768: use of closed network connection
Jul 9 02:14:14.847142999 (377h2m41.773437687s) [seg_pos={0 0}, upload=301954]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jul 9 02:14:14.847204962 (377h2m41.773499652s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:14:14.847209974 (377h2m41.773504671s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:14:14.909581364 (377h2m41.83587606s) [upload=301954]: Done uploading segments...
Jul 9 02:14:14.909605483 (377h2m41.835900172s) [upload=301954]: Upload finished. err: <nil>
Duration: 502.784486ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
443.935649ms
FAIL (2025-07-09 02:58:27 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
3:56.263910643 (377h2m23.19020535s) [upload=301954]: Got next segment. Inline: false
Jul 9 02:13:56.263954358 (377h2m23.190249034s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:13:56.263959891 (377h2m23.190254572s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:13:56.302204249 (377h2m23.228498944s) [seg_pos={0 0}, upload=301954]: Begin upload segment...
Jul 9 02:13:56.302291587 (377h2m23.22858628s) [seg_pos={0 0}, upload=301954]: Done begin upload segment.
Jul 9 02:13:56.302338947 (377h2m23.228633682s) [noise=true , address=77.90.31.14:4689 , piece_id=LW7OT74AJCCC3TEKY6OZI553YK6YQFQ6C7PORQB53E7FUS24KQKQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.302352599 (377h2m23.228647272s) [noise=true , address=104.36.82.8:4986 , piece_id=KPY5JVLSAEB5GUKUJJSIF5M6IIHTJISEYYAXCGUSJ7OLI5QCIBNQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.302450837 (377h2m23.228745531s) [noise=true , address=77.90.31.16:4739 , piece_id=H6UCV6RURPXKC2H3YGAT2OXV2VZMW2DWIRG6TFD6CVXKOVMDP2VQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.302958774 (377h2m23.229253667s) [noise=true , address=77.90.31.13:4640 , piece_id=TPPN75DX3UCBM346CSVJN3GSOA47QV4FVPYSSAHWCJYEM42OCVLQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303069086 (377h2m23.229363782s) [noise=true , address=104.36.82.17:4345 , piece_id=YI27O5WLBYO3TPCX4MVUATFORY6U3EQMDKQHGKRHIZMTG7WGFB2A, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303495994 (377h2m23.22979069s) [noise=true , address=104.36.82.36:4644 , piece_id=K2G3MAPD7TWYZPKNHQBMEPJRIQDLE75FYVPAZFX4I3GJ4IUAA7XQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303575840 (377h2m23.229870536s) [noise=true , address=77.90.31.21:4898 , piece_id=75XXVNT4EV4D5UTX7O5XFL44IZOYY5LZSI2VPFQZPRERL6W5M23A, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303949062 (377h2m23.230245641s) [noise=true , address=79.127.201.215:4339 , piece_id=XDCTYE2SVI6RAPCXTRVANFCZYWJDZXLBHHJ47NQY6V7D36DEFPGA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304060410 (377h2m23.230355106s) [noise=true , address=104.36.82.41:4768 , piece_id=BGMF33OKUZSZKGIHJYI3RNXEBDE54TX67DYLJ7JCZMRNK6ZAU22Q, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304407993 (377h2m23.230702685s) [noise=true , address=77.90.31.10:4525 , piece_id=GPVSIZIBZWTUCP5RFYVPFYIYIL2LWU5PU3UASQVRQLY7SSSFMQBA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304548115 (377h2m23.230842812s) [noise=true , address=104.36.82.23:4464 , piece_id=JCJ4MGEZIIZ7YCTM6QXC5ZJ7PI4OEJBOMHZYBU57D2SNGWJWB5CQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304795164 (377h2m23.231089862s) [noise=true , address=77.90.31.18:4838 , piece_id=TGWYXA3AF7LDG4KPVCRNEYFDADAG6TQ5DAX7JJVW5BLVVVNECILA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305012510 (377h2m23.231307211s) [noise=true , address=104.36.82.34:4554 , piece_id=57K5SDV3YEGLVMSJGSLXNKEWOLETXE6CWBM2NPVCNKLFHPVQCJBQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305113737 (377h2m23.231408427s) [noise=true , address=104.36.82.4:4834 , piece_id=YLYIASUPNRTK7KOMRYLN7OQLQBPWC7WFGCPDGUVIOEQJWQQ7A7GA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305383755 (377h2m23.231678434s) [noise=true , address=104.36.82.38:4698 , piece_id=TSOXJNQEWODCWFEITH4IYKZI7A75UZEXRQPIQO7LMP6LT3WLNXUA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305483822 (377h2m23.231778517s) [noise=true , address=104.36.82.42:4830 , piece_id=ENYQTUCIF7OWQPCXTYS24OC5QT63ZJJXOY652GLIG7OQPJEQCGGA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305954019 (377h2m23.232248718s) [noise=true , address=77.90.31.21:4907 , piece_id=2EUWATHUQNZ77BJV65ZFKR5WL4D3AUURLBQ32XR2IVBA7P4TITOQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.306425517 (377h2m23.232720213s) [noise=true , address=104.36.82.22:4541 , piece_id=XN4HDFDNZ4YUJTDPOTMMDFX4FE2YVZDGCIMHEMJGWXRPNHQ6EU7A, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.375470790 (377h2m23.301765481s) [upload=301954]: Next returned nil segment
Jul 9 02:13:56.375489934 (377h2m23.30178463s) [upload=301954]: Waiting for error group managing segments...
Jul 9 02:13:56.381456859 (377h2m23.307751539s) [noise=true , address=77.90.31.10:4525 , piece_id=GPVSIZIBZWTUCP5RFYVPFYIYIL2LWU5PU3UASQVRQLY7SSSFMQBA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.381956907 (377h2m23.308251586s) [noise=true , address=77.90.31.14:4689 , piece_id=LW7OT74AJCCC3TEKY6OZI553YK6YQFQ6C7PORQB53E7FUS24KQKQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.382837088 (377h2m23.309131761s) [noise=true , address=77.90.31.21:4898 , piece_id=75XXVNT4EV4D5UTX7O5XFL44IZOYY5LZSI2VPFQZPRERL6W5M23A, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.393489125 (377h2m23.319783812s) [noise=true , address=77.90.31.21:4907 , piece_id=2EUWATHUQNZ77BJV65ZFKR5WL4D3AUURLBQ32XR2IVBA7P4TITOQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.404692801 (377h2m23.330987481s) [noise=true , address=77.90.31.16:4739 , piece_id=H6UCV6RURPXKC2H3YGAT2OXV2VZMW2DWIRG6TFD6CVXKOVMDP2VQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.404986115 (377h2m23.331280795s) [noise=true , address=77.90.31.18:4838 , piece_id=TGWYXA3AF7LDG4KPVCRNEYFDADAG6TQ5DAX7JJVW5BLVVVNECILA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.433785086 (377h2m23.360079766s) [noise=true , address=77.90.31.13:4640 , piece_id=TPPN75DX3UCBM346CSVJN3GSOA47QV4FVPYSSAHWCJYEM42OCVLQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.598499747 (377h2m23.524794427s) [noise=true , address=79.127.201.215:4339 , piece_id=XDCTYE2SVI6RAPCXTRVANFCZYWJDZXLBHHJ47NQY6V7D36DEFPGA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:11.107785584 (377h2m38.034080273s) [noise=true , address=104.36.82.22:4541 , piece_id=XN4HDFDNZ4YUJTDPOTMMDFX4FE2YVZDGCIMHEMJGWXRPNHQ6EU7A, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:12.515567029 (377h2m39.441861713s) [noise=true , address=104.36.82.17:4345 , piece_id=YI27O5WLBYO3TPCX4MVUATFORY6U3EQMDKQHGKRHIZMTG7WGFB2A, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:12.923090587 (377h2m39.849385267s) [noise=true , address=104.36.82.8:4986 , piece_id=KPY5JVLSAEB5GUKUJJSIF5M6IIHTJISEYYAXCGUSJ7OLI5QCIBNQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:13.453172659 (377h2m40.379467346s) [noise=true , address=104.36.82.38:4698 , piece_id=TSOXJNQEWODCWFEITH4IYKZI7A75UZEXRQPIQO7LMP6LT3WLNXUA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:14.416524856 (377h2m41.342819534s) [noise=true , address=104.36.82.34:4554 , piece_id=57K5SDV3YEGLVMSJGSLXNKEWOLETXE6CWBM2NPVCNKLFHPVQCJBQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:14.846730699 (377h2m41.773025379s) [noise=true , address=104.36.82.23:4464 , piece_id=JCJ4MGEZIIZ7YCTM6QXC5ZJ7PI4OEJBOMHZYBU57D2SNGWJWB5CQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:14.846763777 (377h2m41.773058463s) [seg_pos={0 0}, upload=301954]: Segment reached optimal threshold of 14 pieces.
Jul 9 02:14:14.846894905 (377h2m41.773189582s) [noise=true , address=104.36.82.36:4644 , piece_id=K2G3MAPD7TWYZPKNHQBMEPJRIQDLE75FYVPAZFX4I3GJ4IUAA7XQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1oYHbb73ACitbfkkP5P6cmRyTt6kmaS1y49G4vbsvTvVDwByhT): protocol: expected piece hash; context canceled; EOF
Jul 9 02:14:14.846948435 (377h2m41.773243131s) [noise=true , address=104.36.82.4:4834 , piece_id=YLYIASUPNRTK7KOMRYLN7OQLQBPWC7WFGCPDGUVIOEQJWQQ7A7GA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Z241DCYA5Wig2TdEogxTpY6FzVmFbnKtmHcTwxLxfcZafgooy): protocol: expected piece hash; context canceled; EOF
Jul 9 02:14:14.847094561 (377h2m41.773389249s) [noise=true , address=104.36.82.42:4830 , piece_id=ENYQTUCIF7OWQPCXTYS24OC5QT63ZJJXOY652GLIG7OQPJEQCGGA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:142ocSHHtv8BpSebXFNk5qk1Y23a8pbFAZqE3cq6i3RMX3dPw9): protocol: expected piece hash; context canceled; EOF
Jul 9 02:14:14.847127825 (377h2m41.773422511s) [noise=true , address=104.36.82.41:4768 , piece_id=BGMF33OKUZSZKGIHJYI3RNXEBDE54TX67DYLJ7JCZMRNK6ZAU22Q, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: write tcp 159.223.143.39:43796->104.36.82.41:4768: use of closed network connection; write tcp 159.223.143.39:43796->104.36.82.41:4768: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:43796->104.36.82.41:4768: use of closed network connection
Jul 9 02:14:14.847142999 (377h2m41.773437687s) [seg_pos={0 0}, upload=301954]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jul 9 02:14:14.847204962 (377h2m41.773499652s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:14:14.847209974 (377h2m41.773504671s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:14:14.909581364 (377h2m41.83587606s) [upload=301954]: Done uploading segments...
Jul 9 02:14:14.909605483 (377h2m41.835900172s) [upload=301954]: Upload finished. err: <nil>
Duration: 443.969684ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [............X...............................................] duration:
548.99098ms
FAIL (2025-07-09 02:58:13 UTC)
orbiter warning:
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
02:25:27.135824454 (377h13m54.062119141s) [noise=true , address=104.36.82.12:4150 , piece_id=DRG5SBA3EE4KVQPEQOJZ6SS537SFWYFUYQXOAQMWYTT6FPVSJIPQ, seg_pos={0 0}, upload=302099]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: context canceled; context canceled
Jul 9 02:25:27.135860079 (377h13m54.062154769s) [seg_pos={0 0}, upload=302099]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 02:25:27.135957283 (377h13m54.062251971s) [upload=302099]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:25:27.135964211 (377h13m54.062258897s) [upload=302099]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:25:27.199410713 (377h13m54.125705389s) [upload=302099]: Done uploading segments...
Jul 9 02:25:27.199427901 (377h13m54.125722577s) [upload=302099]: Upload finished. err: <nil>
Jul 9 02:25:57.289077655 (377h14m24.215372329s) [upload=302109]: Starting upload
Jul 9 02:25:57.289186750 (377h14m24.215481424s) [upload=302109]: Done starting upload
Jul 9 02:25:57.289444851 (377h14m24.215739548s) [upload=302109]: Uploading segments...
Jul 9 02:25:57.289623846 (377h14m24.215918535s) [upload=302109]: Got next segment. Inline: false
Jul 9 02:25:57.289696741 (377h14m24.215991432s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:25:57.289703375 (377h14m24.215998067s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:25:57.318313152 (377h14m24.244607838s) [seg_pos={0 0}, upload=302109]: Begin upload segment...
Jul 9 02:25:57.318363286 (377h14m24.244657964s) [seg_pos={0 0}, upload=302109]: Done begin upload segment.
Jul 9 02:25:57.318386772 (377h14m24.244681463s) [noise=true , address=77.90.31.11:4553 , piece_id=TYW7NBJWACIT7OQS7A4WGXUPR3MB2FKAN2OMGAZJERB4X2DLAO2Q, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318412127 (377h14m24.244706825s) [noise=true , address=77.90.31.10:4518 , piece_id=VJ4HWM2X4VSRSUCQL5QLMBYPFBYV4JYS5ZXCBCB6MTMB76EHZFGQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318822149 (377h14m24.245116841s) [noise=true , address=45.91.116.9:4263 , piece_id=5YILAJ5BZBGOURSZE343ST4RMOTWM6ZOB3EEQQWL5WUNYWOGSAPA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318965862 (377h14m24.24526056s) [noise=true , address=45.91.116.8:4045 , piece_id=KM6ZYTHR6JGTARKZQZYIESPKBEU5H2HQFA7OARIKD2ZXDHPFSMGA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.319187516 (377h14m24.245482203s) [noise=true , address=104.36.82.7:4909 , piece_id=ZRHLWDQ5KIDRWPJ5FXAANQTWERKXCR2PPEGN442IOSYSRFAG7VZA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.319523815 (377h14m24.245818512s) [noise=true , address=45.91.116.14:4191 , piece_id=GTR6T4OTGZTEZ76FWQZ46OCH5WB3O5X27EN4AV33ZJ7D5HX4535A, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.320556632 (377h14m24.24685133s) [noise=true , address=104.36.82.2:4732 , piece_id=DJIGHJMJRAULH2L4YT2WGRXPRQJMNKZG5LICHPSKOCHUL6QDV6BQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.321019141 (377h14m24.247313835s) [noise=true , address=77.90.31.17:4770 , piece_id=V36UBBIYOFJC6KFVVBXEVT4QBUIWN257D3YSDDRAGZVXH7ZXHTBA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.322591797 (377h14m24.248886493s) [noise=true , address=77.90.31.13:4627 , piece_id=HULEKFVZNDNKS4SWWGTJUHYZWWYPDCER5W4GFCVLLDL65PAHULFQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323120451 (377h14m24.249415138s) [noise=true , address=45.91.116.4:4939 , piece_id=ANA3EZU7OZWELNG4QEMTEAYZY5H3C3H4IJFT4XKPUB7WIF44FEVQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323544591 (377h14m24.249839288s) [noise=true , address=45.91.116.7:4011 , piece_id=VDOJRN6ACK4LO44BQTULO6BB3CLAVQI7YIC6XLR32CZV4UJSPE6Q, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323950189 (377h14m24.250244889s) [noise=true , address=104.36.82.10:4078 , piece_id=ZFVUQAEV6DDFLPKCZA2XJ2BZHMMM745LGB4JRZJZQPG72WECGMDQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.326585810 (377h14m24.252880507s) [noise=true , address=45.91.116.5:4297 , piece_id=MSDWDLY5PJHQVI5565EKANJVSTMZFEUD2OTYZ52C65XQUICNMC3A, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.327871214 (377h14m24.254165904s) [noise=true , address=104.36.82.6:4875 , piece_id=C275F2JPVAF52E34CEHT5QXCHUFGIQZ7XGMOAOUOT4RA7YOHRQLQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.328605288 (377h14m24.254899986s) [noise=true , address=104.36.82.5:4838 , piece_id=WK3ALBOEMWWTVEA4FKCS352SZ6YYB35LELURNWXQZK32K4HYDRNA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.335409087 (377h14m24.261703784s) [noise=true , address=45.91.116.6:4973 , piece_id=Z7HQQ2O5QUSJKT4EMIDMEI5PIAEHNVRYX7MALUR2GQULRFIBUOKA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.475344637 (377h14m24.401639326s) [upload=302109]: Next returned nil segment
Jul 9 02:25:57.475376181 (377h14m24.401670879s) [upload=302109]: Waiting for error group managing segments...
Jul 9 02:25:57.477269278 (377h14m24.403563958s) [noise=true , address=45.91.116.7:4011 , piece_id=VDOJRN6ACK4LO44BQTULO6BB3CLAVQI7YIC6XLR32CZV4UJSPE6Q, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.477736179 (377h14m24.40403086s) [noise=true , address=45.91.116.9:4263 , piece_id=5YILAJ5BZBGOURSZE343ST4RMOTWM6ZOB3EEQQWL5WUNYWOGSAPA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.478396652 (377h14m24.404691332s) [noise=true , address=77.90.31.10:4518 , piece_id=VJ4HWM2X4VSRSUCQL5QLMBYPFBYV4JYS5ZXCBCB6MTMB76EHZFGQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.478919538 (377h14m24.405214224s) [noise=true , address=45.91.116.14:4191 , piece_id=GTR6T4OTGZTEZ76FWQZ46OCH5WB3O5X27EN4AV33ZJ7D5HX4535A, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481426716 (377h14m24.407721396s) [noise=true , address=45.91.116.4:4939 , piece_id=ANA3EZU7OZWELNG4QEMTEAYZY5H3C3H4IJFT4XKPUB7WIF44FEVQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481718373 (377h14m24.408013051s) [noise=true , address=45.91.116.5:4297 , piece_id=MSDWDLY5PJHQVI5565EKANJVSTMZFEUD2OTYZ52C65XQUICNMC3A, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481930356 (377h14m24.408225033s) [noise=true , address=77.90.31.17:4770 , piece_id=V36UBBIYOFJC6KFVVBXEVT4QBUIWN257D3YSDDRAGZVXH7ZXHTBA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.482376147 (377h14m24.408670821s) [noise=true , address=45.91.116.8:4045 , piece_id=KM6ZYTHR6JGTARKZQZYIESPKBEU5H2HQFA7OARIKD2ZXDHPFSMGA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.482439192 (377h14m24.408733868s) [noise=true , address=45.91.116.6:4973 , piece_id=Z7HQQ2O5QUSJKT4EMIDMEI5PIAEHNVRYX7MALUR2GQULRFIBUOKA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.483047970 (377h14m24.409342644s) [noise=true , address=77.90.31.13:4627 , piece_id=HULEKFVZNDNKS4SWWGTJUHYZWWYPDCER5W4GFCVLLDL65PAHULFQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.497788632 (377h14m24.424083312s) [noise=true , address=77.90.31.11:4553 , piece_id=TYW7NBJWACIT7OQS7A4WGXUPR3MB2FKAN2OMGAZJERB4X2DLAO2Q, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:16.688921171 (377h14m43.615215851s) [noise=true , address=104.36.82.6:4875 , piece_id=C275F2JPVAF52E34CEHT5QXCHUFGIQZ7XGMOAOUOT4RA7YOHRQLQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:27.177893955 (377h14m54.104188635s) [noise=true , address=104.36.82.2:4732 , piece_id=DJIGHJMJRAULH2L4YT2WGRXPRQJMNKZG5LICHPSKOCHUL6QDV6BQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:27.177985584 (377h14m54.104280281s) [seg_pos={0 0}, upload=302109]: Segment reached optimal threshold of 13 pieces.
Jul 9 02:26:27.178189789 (377h14m54.104484461s) [noise=true , address=104.36.82.5:4838 , piece_id=WK3ALBOEMWWTVEA4FKCS352SZ6YYB35LELURNWXQZK32K4HYDRNA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178252042 (377h14m54.104546733s) [noise=true , address=104.36.82.7:4909 , piece_id=ZRHLWDQ5KIDRWPJ5FXAANQTWERKXCR2PPEGN442IOSYSRFAG7VZA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178273922 (377h14m54.104568608s) [noise=true , address=104.36.82.10:4078 , piece_id=ZFVUQAEV6DDFLPKCZA2XJ2BZHMMM745LGB4JRZJZQPG72WECGMDQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178305983 (377h14m54.104600682s) [seg_pos={0 0}, upload=302109]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 02:26:27.178428768 (377h14m54.104723465s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:26:27.178455810 (377h14m54.104750501s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:26:27.232166184 (377h14m54.158460883s) [upload=302109]: Done uploading segments...
Jul 9 02:26:27.232188444 (377h14m54.158483123s) [upload=302109]: Upload finished. err: <nil>
Duration: 549.041025ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
25 out of 60 is failed (threshold=3) [.XX.XX..X.X.X.....XXX.XX.X...XXX....XX...X..X..X.XX.X..X....] duration:
1.256044059s
FAIL (2025-07-09 02:57:46 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.256064363s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................] duration:
11.493104751s
FAIL (2025-07-09 02:57:24 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
3:56.263910643 (377h2m23.19020535s) [upload=301954]: Got next segment. Inline: false
Jul 9 02:13:56.263954358 (377h2m23.190249034s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:13:56.263959891 (377h2m23.190254572s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:13:56.302204249 (377h2m23.228498944s) [seg_pos={0 0}, upload=301954]: Begin upload segment...
Jul 9 02:13:56.302291587 (377h2m23.22858628s) [seg_pos={0 0}, upload=301954]: Done begin upload segment.
Jul 9 02:13:56.302338947 (377h2m23.228633682s) [noise=true , address=77.90.31.14:4689 , piece_id=LW7OT74AJCCC3TEKY6OZI553YK6YQFQ6C7PORQB53E7FUS24KQKQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.302352599 (377h2m23.228647272s) [noise=true , address=104.36.82.8:4986 , piece_id=KPY5JVLSAEB5GUKUJJSIF5M6IIHTJISEYYAXCGUSJ7OLI5QCIBNQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.302450837 (377h2m23.228745531s) [noise=true , address=77.90.31.16:4739 , piece_id=H6UCV6RURPXKC2H3YGAT2OXV2VZMW2DWIRG6TFD6CVXKOVMDP2VQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.302958774 (377h2m23.229253667s) [noise=true , address=77.90.31.13:4640 , piece_id=TPPN75DX3UCBM346CSVJN3GSOA47QV4FVPYSSAHWCJYEM42OCVLQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303069086 (377h2m23.229363782s) [noise=true , address=104.36.82.17:4345 , piece_id=YI27O5WLBYO3TPCX4MVUATFORY6U3EQMDKQHGKRHIZMTG7WGFB2A, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303495994 (377h2m23.22979069s) [noise=true , address=104.36.82.36:4644 , piece_id=K2G3MAPD7TWYZPKNHQBMEPJRIQDLE75FYVPAZFX4I3GJ4IUAA7XQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303575840 (377h2m23.229870536s) [noise=true , address=77.90.31.21:4898 , piece_id=75XXVNT4EV4D5UTX7O5XFL44IZOYY5LZSI2VPFQZPRERL6W5M23A, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303949062 (377h2m23.230245641s) [noise=true , address=79.127.201.215:4339 , piece_id=XDCTYE2SVI6RAPCXTRVANFCZYWJDZXLBHHJ47NQY6V7D36DEFPGA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304060410 (377h2m23.230355106s) [noise=true , address=104.36.82.41:4768 , piece_id=BGMF33OKUZSZKGIHJYI3RNXEBDE54TX67DYLJ7JCZMRNK6ZAU22Q, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304407993 (377h2m23.230702685s) [noise=true , address=77.90.31.10:4525 , piece_id=GPVSIZIBZWTUCP5RFYVPFYIYIL2LWU5PU3UASQVRQLY7SSSFMQBA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304548115 (377h2m23.230842812s) [noise=true , address=104.36.82.23:4464 , piece_id=JCJ4MGEZIIZ7YCTM6QXC5ZJ7PI4OEJBOMHZYBU57D2SNGWJWB5CQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304795164 (377h2m23.231089862s) [noise=true , address=77.90.31.18:4838 , piece_id=TGWYXA3AF7LDG4KPVCRNEYFDADAG6TQ5DAX7JJVW5BLVVVNECILA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305012510 (377h2m23.231307211s) [noise=true , address=104.36.82.34:4554 , piece_id=57K5SDV3YEGLVMSJGSLXNKEWOLETXE6CWBM2NPVCNKLFHPVQCJBQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305113737 (377h2m23.231408427s) [noise=true , address=104.36.82.4:4834 , piece_id=YLYIASUPNRTK7KOMRYLN7OQLQBPWC7WFGCPDGUVIOEQJWQQ7A7GA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305383755 (377h2m23.231678434s) [noise=true , address=104.36.82.38:4698 , piece_id=TSOXJNQEWODCWFEITH4IYKZI7A75UZEXRQPIQO7LMP6LT3WLNXUA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305483822 (377h2m23.231778517s) [noise=true , address=104.36.82.42:4830 , piece_id=ENYQTUCIF7OWQPCXTYS24OC5QT63ZJJXOY652GLIG7OQPJEQCGGA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305954019 (377h2m23.232248718s) [noise=true , address=77.90.31.21:4907 , piece_id=2EUWATHUQNZ77BJV65ZFKR5WL4D3AUURLBQ32XR2IVBA7P4TITOQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.306425517 (377h2m23.232720213s) [noise=true , address=104.36.82.22:4541 , piece_id=XN4HDFDNZ4YUJTDPOTMMDFX4FE2YVZDGCIMHEMJGWXRPNHQ6EU7A, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.375470790 (377h2m23.301765481s) [upload=301954]: Next returned nil segment
Jul 9 02:13:56.375489934 (377h2m23.30178463s) [upload=301954]: Waiting for error group managing segments...
Jul 9 02:13:56.381456859 (377h2m23.307751539s) [noise=true , address=77.90.31.10:4525 , piece_id=GPVSIZIBZWTUCP5RFYVPFYIYIL2LWU5PU3UASQVRQLY7SSSFMQBA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.381956907 (377h2m23.308251586s) [noise=true , address=77.90.31.14:4689 , piece_id=LW7OT74AJCCC3TEKY6OZI553YK6YQFQ6C7PORQB53E7FUS24KQKQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.382837088 (377h2m23.309131761s) [noise=true , address=77.90.31.21:4898 , piece_id=75XXVNT4EV4D5UTX7O5XFL44IZOYY5LZSI2VPFQZPRERL6W5M23A, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.393489125 (377h2m23.319783812s) [noise=true , address=77.90.31.21:4907 , piece_id=2EUWATHUQNZ77BJV65ZFKR5WL4D3AUURLBQ32XR2IVBA7P4TITOQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.404692801 (377h2m23.330987481s) [noise=true , address=77.90.31.16:4739 , piece_id=H6UCV6RURPXKC2H3YGAT2OXV2VZMW2DWIRG6TFD6CVXKOVMDP2VQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.404986115 (377h2m23.331280795s) [noise=true , address=77.90.31.18:4838 , piece_id=TGWYXA3AF7LDG4KPVCRNEYFDADAG6TQ5DAX7JJVW5BLVVVNECILA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.433785086 (377h2m23.360079766s) [noise=true , address=77.90.31.13:4640 , piece_id=TPPN75DX3UCBM346CSVJN3GSOA47QV4FVPYSSAHWCJYEM42OCVLQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.598499747 (377h2m23.524794427s) [noise=true , address=79.127.201.215:4339 , piece_id=XDCTYE2SVI6RAPCXTRVANFCZYWJDZXLBHHJ47NQY6V7D36DEFPGA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:11.107785584 (377h2m38.034080273s) [noise=true , address=104.36.82.22:4541 , piece_id=XN4HDFDNZ4YUJTDPOTMMDFX4FE2YVZDGCIMHEMJGWXRPNHQ6EU7A, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:12.515567029 (377h2m39.441861713s) [noise=true , address=104.36.82.17:4345 , piece_id=YI27O5WLBYO3TPCX4MVUATFORY6U3EQMDKQHGKRHIZMTG7WGFB2A, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:12.923090587 (377h2m39.849385267s) [noise=true , address=104.36.82.8:4986 , piece_id=KPY5JVLSAEB5GUKUJJSIF5M6IIHTJISEYYAXCGUSJ7OLI5QCIBNQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:13.453172659 (377h2m40.379467346s) [noise=true , address=104.36.82.38:4698 , piece_id=TSOXJNQEWODCWFEITH4IYKZI7A75UZEXRQPIQO7LMP6LT3WLNXUA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:14.416524856 (377h2m41.342819534s) [noise=true , address=104.36.82.34:4554 , piece_id=57K5SDV3YEGLVMSJGSLXNKEWOLETXE6CWBM2NPVCNKLFHPVQCJBQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:14.846730699 (377h2m41.773025379s) [noise=true , address=104.36.82.23:4464 , piece_id=JCJ4MGEZIIZ7YCTM6QXC5ZJ7PI4OEJBOMHZYBU57D2SNGWJWB5CQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:14.846763777 (377h2m41.773058463s) [seg_pos={0 0}, upload=301954]: Segment reached optimal threshold of 14 pieces.
Jul 9 02:14:14.846894905 (377h2m41.773189582s) [noise=true , address=104.36.82.36:4644 , piece_id=K2G3MAPD7TWYZPKNHQBMEPJRIQDLE75FYVPAZFX4I3GJ4IUAA7XQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1oYHbb73ACitbfkkP5P6cmRyTt6kmaS1y49G4vbsvTvVDwByhT): protocol: expected piece hash; context canceled; EOF
Jul 9 02:14:14.846948435 (377h2m41.773243131s) [noise=true , address=104.36.82.4:4834 , piece_id=YLYIASUPNRTK7KOMRYLN7OQLQBPWC7WFGCPDGUVIOEQJWQQ7A7GA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Z241DCYA5Wig2TdEogxTpY6FzVmFbnKtmHcTwxLxfcZafgooy): protocol: expected piece hash; context canceled; EOF
Jul 9 02:14:14.847094561 (377h2m41.773389249s) [noise=true , address=104.36.82.42:4830 , piece_id=ENYQTUCIF7OWQPCXTYS24OC5QT63ZJJXOY652GLIG7OQPJEQCGGA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:142ocSHHtv8BpSebXFNk5qk1Y23a8pbFAZqE3cq6i3RMX3dPw9): protocol: expected piece hash; context canceled; EOF
Jul 9 02:14:14.847127825 (377h2m41.773422511s) [noise=true , address=104.36.82.41:4768 , piece_id=BGMF33OKUZSZKGIHJYI3RNXEBDE54TX67DYLJ7JCZMRNK6ZAU22Q, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: write tcp 159.223.143.39:43796->104.36.82.41:4768: use of closed network connection; write tcp 159.223.143.39:43796->104.36.82.41:4768: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:43796->104.36.82.41:4768: use of closed network connection
Jul 9 02:14:14.847142999 (377h2m41.773437687s) [seg_pos={0 0}, upload=301954]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jul 9 02:14:14.847204962 (377h2m41.773499652s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:14:14.847209974 (377h2m41.773504671s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:14:14.909581364 (377h2m41.83587606s) [upload=301954]: Done uploading segments...
Jul 9 02:14:14.909605483 (377h2m41.835900172s) [upload=301954]: Upload finished. err: <nil>
Duration: 11.493197049s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
38 out of 60 is failed (threshold=3) [..XX.XXX.XXX.XX.XXXXXX.XXXXX.XX.....X..XXXXX.XX..XXX.XX.XX..] duration:
17.783213368s
FAIL (2025-07-09 02:57:19 UTC)
context deadline exceeded
Duration: 17.783235013s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............X..............................................] duration:
418.012588ms
FAIL (2025-07-09 02:57:09 UTC)
orbiter warning:
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
02:25:27.135824454 (377h13m54.062119141s) [noise=true , address=104.36.82.12:4150 , piece_id=DRG5SBA3EE4KVQPEQOJZ6SS537SFWYFUYQXOAQMWYTT6FPVSJIPQ, seg_pos={0 0}, upload=302099]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: context canceled; context canceled
Jul 9 02:25:27.135860079 (377h13m54.062154769s) [seg_pos={0 0}, upload=302099]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 02:25:27.135957283 (377h13m54.062251971s) [upload=302099]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:25:27.135964211 (377h13m54.062258897s) [upload=302099]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:25:27.199410713 (377h13m54.125705389s) [upload=302099]: Done uploading segments...
Jul 9 02:25:27.199427901 (377h13m54.125722577s) [upload=302099]: Upload finished. err: <nil>
Jul 9 02:25:57.289077655 (377h14m24.215372329s) [upload=302109]: Starting upload
Jul 9 02:25:57.289186750 (377h14m24.215481424s) [upload=302109]: Done starting upload
Jul 9 02:25:57.289444851 (377h14m24.215739548s) [upload=302109]: Uploading segments...
Jul 9 02:25:57.289623846 (377h14m24.215918535s) [upload=302109]: Got next segment. Inline: false
Jul 9 02:25:57.289696741 (377h14m24.215991432s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:25:57.289703375 (377h14m24.215998067s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:25:57.318313152 (377h14m24.244607838s) [seg_pos={0 0}, upload=302109]: Begin upload segment...
Jul 9 02:25:57.318363286 (377h14m24.244657964s) [seg_pos={0 0}, upload=302109]: Done begin upload segment.
Jul 9 02:25:57.318386772 (377h14m24.244681463s) [noise=true , address=77.90.31.11:4553 , piece_id=TYW7NBJWACIT7OQS7A4WGXUPR3MB2FKAN2OMGAZJERB4X2DLAO2Q, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318412127 (377h14m24.244706825s) [noise=true , address=77.90.31.10:4518 , piece_id=VJ4HWM2X4VSRSUCQL5QLMBYPFBYV4JYS5ZXCBCB6MTMB76EHZFGQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318822149 (377h14m24.245116841s) [noise=true , address=45.91.116.9:4263 , piece_id=5YILAJ5BZBGOURSZE343ST4RMOTWM6ZOB3EEQQWL5WUNYWOGSAPA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318965862 (377h14m24.24526056s) [noise=true , address=45.91.116.8:4045 , piece_id=KM6ZYTHR6JGTARKZQZYIESPKBEU5H2HQFA7OARIKD2ZXDHPFSMGA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.319187516 (377h14m24.245482203s) [noise=true , address=104.36.82.7:4909 , piece_id=ZRHLWDQ5KIDRWPJ5FXAANQTWERKXCR2PPEGN442IOSYSRFAG7VZA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.319523815 (377h14m24.245818512s) [noise=true , address=45.91.116.14:4191 , piece_id=GTR6T4OTGZTEZ76FWQZ46OCH5WB3O5X27EN4AV33ZJ7D5HX4535A, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.320556632 (377h14m24.24685133s) [noise=true , address=104.36.82.2:4732 , piece_id=DJIGHJMJRAULH2L4YT2WGRXPRQJMNKZG5LICHPSKOCHUL6QDV6BQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.321019141 (377h14m24.247313835s) [noise=true , address=77.90.31.17:4770 , piece_id=V36UBBIYOFJC6KFVVBXEVT4QBUIWN257D3YSDDRAGZVXH7ZXHTBA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.322591797 (377h14m24.248886493s) [noise=true , address=77.90.31.13:4627 , piece_id=HULEKFVZNDNKS4SWWGTJUHYZWWYPDCER5W4GFCVLLDL65PAHULFQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323120451 (377h14m24.249415138s) [noise=true , address=45.91.116.4:4939 , piece_id=ANA3EZU7OZWELNG4QEMTEAYZY5H3C3H4IJFT4XKPUB7WIF44FEVQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323544591 (377h14m24.249839288s) [noise=true , address=45.91.116.7:4011 , piece_id=VDOJRN6ACK4LO44BQTULO6BB3CLAVQI7YIC6XLR32CZV4UJSPE6Q, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323950189 (377h14m24.250244889s) [noise=true , address=104.36.82.10:4078 , piece_id=ZFVUQAEV6DDFLPKCZA2XJ2BZHMMM745LGB4JRZJZQPG72WECGMDQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.326585810 (377h14m24.252880507s) [noise=true , address=45.91.116.5:4297 , piece_id=MSDWDLY5PJHQVI5565EKANJVSTMZFEUD2OTYZ52C65XQUICNMC3A, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.327871214 (377h14m24.254165904s) [noise=true , address=104.36.82.6:4875 , piece_id=C275F2JPVAF52E34CEHT5QXCHUFGIQZ7XGMOAOUOT4RA7YOHRQLQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.328605288 (377h14m24.254899986s) [noise=true , address=104.36.82.5:4838 , piece_id=WK3ALBOEMWWTVEA4FKCS352SZ6YYB35LELURNWXQZK32K4HYDRNA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.335409087 (377h14m24.261703784s) [noise=true , address=45.91.116.6:4973 , piece_id=Z7HQQ2O5QUSJKT4EMIDMEI5PIAEHNVRYX7MALUR2GQULRFIBUOKA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.475344637 (377h14m24.401639326s) [upload=302109]: Next returned nil segment
Jul 9 02:25:57.475376181 (377h14m24.401670879s) [upload=302109]: Waiting for error group managing segments...
Jul 9 02:25:57.477269278 (377h14m24.403563958s) [noise=true , address=45.91.116.7:4011 , piece_id=VDOJRN6ACK4LO44BQTULO6BB3CLAVQI7YIC6XLR32CZV4UJSPE6Q, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.477736179 (377h14m24.40403086s) [noise=true , address=45.91.116.9:4263 , piece_id=5YILAJ5BZBGOURSZE343ST4RMOTWM6ZOB3EEQQWL5WUNYWOGSAPA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.478396652 (377h14m24.404691332s) [noise=true , address=77.90.31.10:4518 , piece_id=VJ4HWM2X4VSRSUCQL5QLMBYPFBYV4JYS5ZXCBCB6MTMB76EHZFGQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.478919538 (377h14m24.405214224s) [noise=true , address=45.91.116.14:4191 , piece_id=GTR6T4OTGZTEZ76FWQZ46OCH5WB3O5X27EN4AV33ZJ7D5HX4535A, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481426716 (377h14m24.407721396s) [noise=true , address=45.91.116.4:4939 , piece_id=ANA3EZU7OZWELNG4QEMTEAYZY5H3C3H4IJFT4XKPUB7WIF44FEVQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481718373 (377h14m24.408013051s) [noise=true , address=45.91.116.5:4297 , piece_id=MSDWDLY5PJHQVI5565EKANJVSTMZFEUD2OTYZ52C65XQUICNMC3A, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481930356 (377h14m24.408225033s) [noise=true , address=77.90.31.17:4770 , piece_id=V36UBBIYOFJC6KFVVBXEVT4QBUIWN257D3YSDDRAGZVXH7ZXHTBA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.482376147 (377h14m24.408670821s) [noise=true , address=45.91.116.8:4045 , piece_id=KM6ZYTHR6JGTARKZQZYIESPKBEU5H2HQFA7OARIKD2ZXDHPFSMGA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.482439192 (377h14m24.408733868s) [noise=true , address=45.91.116.6:4973 , piece_id=Z7HQQ2O5QUSJKT4EMIDMEI5PIAEHNVRYX7MALUR2GQULRFIBUOKA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.483047970 (377h14m24.409342644s) [noise=true , address=77.90.31.13:4627 , piece_id=HULEKFVZNDNKS4SWWGTJUHYZWWYPDCER5W4GFCVLLDL65PAHULFQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.497788632 (377h14m24.424083312s) [noise=true , address=77.90.31.11:4553 , piece_id=TYW7NBJWACIT7OQS7A4WGXUPR3MB2FKAN2OMGAZJERB4X2DLAO2Q, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:16.688921171 (377h14m43.615215851s) [noise=true , address=104.36.82.6:4875 , piece_id=C275F2JPVAF52E34CEHT5QXCHUFGIQZ7XGMOAOUOT4RA7YOHRQLQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:27.177893955 (377h14m54.104188635s) [noise=true , address=104.36.82.2:4732 , piece_id=DJIGHJMJRAULH2L4YT2WGRXPRQJMNKZG5LICHPSKOCHUL6QDV6BQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:27.177985584 (377h14m54.104280281s) [seg_pos={0 0}, upload=302109]: Segment reached optimal threshold of 13 pieces.
Jul 9 02:26:27.178189789 (377h14m54.104484461s) [noise=true , address=104.36.82.5:4838 , piece_id=WK3ALBOEMWWTVEA4FKCS352SZ6YYB35LELURNWXQZK32K4HYDRNA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178252042 (377h14m54.104546733s) [noise=true , address=104.36.82.7:4909 , piece_id=ZRHLWDQ5KIDRWPJ5FXAANQTWERKXCR2PPEGN442IOSYSRFAG7VZA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178273922 (377h14m54.104568608s) [noise=true , address=104.36.82.10:4078 , piece_id=ZFVUQAEV6DDFLPKCZA2XJ2BZHMMM745LGB4JRZJZQPG72WECGMDQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178305983 (377h14m54.104600682s) [seg_pos={0 0}, upload=302109]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 02:26:27.178428768 (377h14m54.104723465s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:26:27.178455810 (377h14m54.104750501s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:26:27.232166184 (377h14m54.158460883s) [upload=302109]: Done uploading segments...
Jul 9 02:26:27.232188444 (377h14m54.158483123s) [upload=302109]: Upload finished. err: <nil>
Duration: 418.040948ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
25 out of 60 is failed (threshold=3) [..XX.XX..X.X.X.....XXX.XX.X...XXX....XX...X..X..X.XX.X..X...] duration:
19.02768603s
FAIL (2025-07-09 02:57:04 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 19.02770673s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
38 out of 60 is failed (threshold=3) [...XX.XXX.XXX.XX.XXXXXX.XXXXX.XX.....X..XXXXX.XX..XXX.XX.XX.] duration:
40.053878627s
FAIL (2025-07-09 02:57:01 UTC)
context deadline exceeded
Duration: 40.053896651s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............X.............................................] duration:
420.415099ms
FAIL (2025-07-09 02:56:50 UTC)
orbiter warning:
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
02:25:27.135824454 (377h13m54.062119141s) [noise=true , address=104.36.82.12:4150 , piece_id=DRG5SBA3EE4KVQPEQOJZ6SS537SFWYFUYQXOAQMWYTT6FPVSJIPQ, seg_pos={0 0}, upload=302099]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: context canceled; context canceled
Jul 9 02:25:27.135860079 (377h13m54.062154769s) [seg_pos={0 0}, upload=302099]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 02:25:27.135957283 (377h13m54.062251971s) [upload=302099]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:25:27.135964211 (377h13m54.062258897s) [upload=302099]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:25:27.199410713 (377h13m54.125705389s) [upload=302099]: Done uploading segments...
Jul 9 02:25:27.199427901 (377h13m54.125722577s) [upload=302099]: Upload finished. err: <nil>
Jul 9 02:25:57.289077655 (377h14m24.215372329s) [upload=302109]: Starting upload
Jul 9 02:25:57.289186750 (377h14m24.215481424s) [upload=302109]: Done starting upload
Jul 9 02:25:57.289444851 (377h14m24.215739548s) [upload=302109]: Uploading segments...
Jul 9 02:25:57.289623846 (377h14m24.215918535s) [upload=302109]: Got next segment. Inline: false
Jul 9 02:25:57.289696741 (377h14m24.215991432s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:25:57.289703375 (377h14m24.215998067s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:25:57.318313152 (377h14m24.244607838s) [seg_pos={0 0}, upload=302109]: Begin upload segment...
Jul 9 02:25:57.318363286 (377h14m24.244657964s) [seg_pos={0 0}, upload=302109]: Done begin upload segment.
Jul 9 02:25:57.318386772 (377h14m24.244681463s) [noise=true , address=77.90.31.11:4553 , piece_id=TYW7NBJWACIT7OQS7A4WGXUPR3MB2FKAN2OMGAZJERB4X2DLAO2Q, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318412127 (377h14m24.244706825s) [noise=true , address=77.90.31.10:4518 , piece_id=VJ4HWM2X4VSRSUCQL5QLMBYPFBYV4JYS5ZXCBCB6MTMB76EHZFGQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318822149 (377h14m24.245116841s) [noise=true , address=45.91.116.9:4263 , piece_id=5YILAJ5BZBGOURSZE343ST4RMOTWM6ZOB3EEQQWL5WUNYWOGSAPA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318965862 (377h14m24.24526056s) [noise=true , address=45.91.116.8:4045 , piece_id=KM6ZYTHR6JGTARKZQZYIESPKBEU5H2HQFA7OARIKD2ZXDHPFSMGA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.319187516 (377h14m24.245482203s) [noise=true , address=104.36.82.7:4909 , piece_id=ZRHLWDQ5KIDRWPJ5FXAANQTWERKXCR2PPEGN442IOSYSRFAG7VZA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.319523815 (377h14m24.245818512s) [noise=true , address=45.91.116.14:4191 , piece_id=GTR6T4OTGZTEZ76FWQZ46OCH5WB3O5X27EN4AV33ZJ7D5HX4535A, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.320556632 (377h14m24.24685133s) [noise=true , address=104.36.82.2:4732 , piece_id=DJIGHJMJRAULH2L4YT2WGRXPRQJMNKZG5LICHPSKOCHUL6QDV6BQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.321019141 (377h14m24.247313835s) [noise=true , address=77.90.31.17:4770 , piece_id=V36UBBIYOFJC6KFVVBXEVT4QBUIWN257D3YSDDRAGZVXH7ZXHTBA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.322591797 (377h14m24.248886493s) [noise=true , address=77.90.31.13:4627 , piece_id=HULEKFVZNDNKS4SWWGTJUHYZWWYPDCER5W4GFCVLLDL65PAHULFQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323120451 (377h14m24.249415138s) [noise=true , address=45.91.116.4:4939 , piece_id=ANA3EZU7OZWELNG4QEMTEAYZY5H3C3H4IJFT4XKPUB7WIF44FEVQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323544591 (377h14m24.249839288s) [noise=true , address=45.91.116.7:4011 , piece_id=VDOJRN6ACK4LO44BQTULO6BB3CLAVQI7YIC6XLR32CZV4UJSPE6Q, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323950189 (377h14m24.250244889s) [noise=true , address=104.36.82.10:4078 , piece_id=ZFVUQAEV6DDFLPKCZA2XJ2BZHMMM745LGB4JRZJZQPG72WECGMDQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.326585810 (377h14m24.252880507s) [noise=true , address=45.91.116.5:4297 , piece_id=MSDWDLY5PJHQVI5565EKANJVSTMZFEUD2OTYZ52C65XQUICNMC3A, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.327871214 (377h14m24.254165904s) [noise=true , address=104.36.82.6:4875 , piece_id=C275F2JPVAF52E34CEHT5QXCHUFGIQZ7XGMOAOUOT4RA7YOHRQLQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.328605288 (377h14m24.254899986s) [noise=true , address=104.36.82.5:4838 , piece_id=WK3ALBOEMWWTVEA4FKCS352SZ6YYB35LELURNWXQZK32K4HYDRNA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.335409087 (377h14m24.261703784s) [noise=true , address=45.91.116.6:4973 , piece_id=Z7HQQ2O5QUSJKT4EMIDMEI5PIAEHNVRYX7MALUR2GQULRFIBUOKA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.475344637 (377h14m24.401639326s) [upload=302109]: Next returned nil segment
Jul 9 02:25:57.475376181 (377h14m24.401670879s) [upload=302109]: Waiting for error group managing segments...
Jul 9 02:25:57.477269278 (377h14m24.403563958s) [noise=true , address=45.91.116.7:4011 , piece_id=VDOJRN6ACK4LO44BQTULO6BB3CLAVQI7YIC6XLR32CZV4UJSPE6Q, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.477736179 (377h14m24.40403086s) [noise=true , address=45.91.116.9:4263 , piece_id=5YILAJ5BZBGOURSZE343ST4RMOTWM6ZOB3EEQQWL5WUNYWOGSAPA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.478396652 (377h14m24.404691332s) [noise=true , address=77.90.31.10:4518 , piece_id=VJ4HWM2X4VSRSUCQL5QLMBYPFBYV4JYS5ZXCBCB6MTMB76EHZFGQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.478919538 (377h14m24.405214224s) [noise=true , address=45.91.116.14:4191 , piece_id=GTR6T4OTGZTEZ76FWQZ46OCH5WB3O5X27EN4AV33ZJ7D5HX4535A, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481426716 (377h14m24.407721396s) [noise=true , address=45.91.116.4:4939 , piece_id=ANA3EZU7OZWELNG4QEMTEAYZY5H3C3H4IJFT4XKPUB7WIF44FEVQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481718373 (377h14m24.408013051s) [noise=true , address=45.91.116.5:4297 , piece_id=MSDWDLY5PJHQVI5565EKANJVSTMZFEUD2OTYZ52C65XQUICNMC3A, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481930356 (377h14m24.408225033s) [noise=true , address=77.90.31.17:4770 , piece_id=V36UBBIYOFJC6KFVVBXEVT4QBUIWN257D3YSDDRAGZVXH7ZXHTBA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.482376147 (377h14m24.408670821s) [noise=true , address=45.91.116.8:4045 , piece_id=KM6ZYTHR6JGTARKZQZYIESPKBEU5H2HQFA7OARIKD2ZXDHPFSMGA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.482439192 (377h14m24.408733868s) [noise=true , address=45.91.116.6:4973 , piece_id=Z7HQQ2O5QUSJKT4EMIDMEI5PIAEHNVRYX7MALUR2GQULRFIBUOKA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.483047970 (377h14m24.409342644s) [noise=true , address=77.90.31.13:4627 , piece_id=HULEKFVZNDNKS4SWWGTJUHYZWWYPDCER5W4GFCVLLDL65PAHULFQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.497788632 (377h14m24.424083312s) [noise=true , address=77.90.31.11:4553 , piece_id=TYW7NBJWACIT7OQS7A4WGXUPR3MB2FKAN2OMGAZJERB4X2DLAO2Q, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:16.688921171 (377h14m43.615215851s) [noise=true , address=104.36.82.6:4875 , piece_id=C275F2JPVAF52E34CEHT5QXCHUFGIQZ7XGMOAOUOT4RA7YOHRQLQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:27.177893955 (377h14m54.104188635s) [noise=true , address=104.36.82.2:4732 , piece_id=DJIGHJMJRAULH2L4YT2WGRXPRQJMNKZG5LICHPSKOCHUL6QDV6BQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:27.177985584 (377h14m54.104280281s) [seg_pos={0 0}, upload=302109]: Segment reached optimal threshold of 13 pieces.
Jul 9 02:26:27.178189789 (377h14m54.104484461s) [noise=true , address=104.36.82.5:4838 , piece_id=WK3ALBOEMWWTVEA4FKCS352SZ6YYB35LELURNWXQZK32K4HYDRNA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178252042 (377h14m54.104546733s) [noise=true , address=104.36.82.7:4909 , piece_id=ZRHLWDQ5KIDRWPJ5FXAANQTWERKXCR2PPEGN442IOSYSRFAG7VZA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178273922 (377h14m54.104568608s) [noise=true , address=104.36.82.10:4078 , piece_id=ZFVUQAEV6DDFLPKCZA2XJ2BZHMMM745LGB4JRZJZQPG72WECGMDQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178305983 (377h14m54.104600682s) [seg_pos={0 0}, upload=302109]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 02:26:27.178428768 (377h14m54.104723465s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:26:27.178455810 (377h14m54.104750501s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:26:27.232166184 (377h14m54.158460883s) [upload=302109]: Done uploading segments...
Jul 9 02:26:27.232188444 (377h14m54.158483123s) [upload=302109]: Upload finished. err: <nil>
Duration: 420.44456ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [......X.....................................................] duration:
8.241841233s
FAIL (2025-07-09 02:56:42 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
3:56.263910643 (377h2m23.19020535s) [upload=301954]: Got next segment. Inline: false
Jul 9 02:13:56.263954358 (377h2m23.190249034s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:13:56.263959891 (377h2m23.190254572s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:13:56.302204249 (377h2m23.228498944s) [seg_pos={0 0}, upload=301954]: Begin upload segment...
Jul 9 02:13:56.302291587 (377h2m23.22858628s) [seg_pos={0 0}, upload=301954]: Done begin upload segment.
Jul 9 02:13:56.302338947 (377h2m23.228633682s) [noise=true , address=77.90.31.14:4689 , piece_id=LW7OT74AJCCC3TEKY6OZI553YK6YQFQ6C7PORQB53E7FUS24KQKQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.302352599 (377h2m23.228647272s) [noise=true , address=104.36.82.8:4986 , piece_id=KPY5JVLSAEB5GUKUJJSIF5M6IIHTJISEYYAXCGUSJ7OLI5QCIBNQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.302450837 (377h2m23.228745531s) [noise=true , address=77.90.31.16:4739 , piece_id=H6UCV6RURPXKC2H3YGAT2OXV2VZMW2DWIRG6TFD6CVXKOVMDP2VQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.302958774 (377h2m23.229253667s) [noise=true , address=77.90.31.13:4640 , piece_id=TPPN75DX3UCBM346CSVJN3GSOA47QV4FVPYSSAHWCJYEM42OCVLQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303069086 (377h2m23.229363782s) [noise=true , address=104.36.82.17:4345 , piece_id=YI27O5WLBYO3TPCX4MVUATFORY6U3EQMDKQHGKRHIZMTG7WGFB2A, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303495994 (377h2m23.22979069s) [noise=true , address=104.36.82.36:4644 , piece_id=K2G3MAPD7TWYZPKNHQBMEPJRIQDLE75FYVPAZFX4I3GJ4IUAA7XQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303575840 (377h2m23.229870536s) [noise=true , address=77.90.31.21:4898 , piece_id=75XXVNT4EV4D5UTX7O5XFL44IZOYY5LZSI2VPFQZPRERL6W5M23A, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303949062 (377h2m23.230245641s) [noise=true , address=79.127.201.215:4339 , piece_id=XDCTYE2SVI6RAPCXTRVANFCZYWJDZXLBHHJ47NQY6V7D36DEFPGA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304060410 (377h2m23.230355106s) [noise=true , address=104.36.82.41:4768 , piece_id=BGMF33OKUZSZKGIHJYI3RNXEBDE54TX67DYLJ7JCZMRNK6ZAU22Q, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304407993 (377h2m23.230702685s) [noise=true , address=77.90.31.10:4525 , piece_id=GPVSIZIBZWTUCP5RFYVPFYIYIL2LWU5PU3UASQVRQLY7SSSFMQBA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304548115 (377h2m23.230842812s) [noise=true , address=104.36.82.23:4464 , piece_id=JCJ4MGEZIIZ7YCTM6QXC5ZJ7PI4OEJBOMHZYBU57D2SNGWJWB5CQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304795164 (377h2m23.231089862s) [noise=true , address=77.90.31.18:4838 , piece_id=TGWYXA3AF7LDG4KPVCRNEYFDADAG6TQ5DAX7JJVW5BLVVVNECILA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305012510 (377h2m23.231307211s) [noise=true , address=104.36.82.34:4554 , piece_id=57K5SDV3YEGLVMSJGSLXNKEWOLETXE6CWBM2NPVCNKLFHPVQCJBQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305113737 (377h2m23.231408427s) [noise=true , address=104.36.82.4:4834 , piece_id=YLYIASUPNRTK7KOMRYLN7OQLQBPWC7WFGCPDGUVIOEQJWQQ7A7GA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305383755 (377h2m23.231678434s) [noise=true , address=104.36.82.38:4698 , piece_id=TSOXJNQEWODCWFEITH4IYKZI7A75UZEXRQPIQO7LMP6LT3WLNXUA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305483822 (377h2m23.231778517s) [noise=true , address=104.36.82.42:4830 , piece_id=ENYQTUCIF7OWQPCXTYS24OC5QT63ZJJXOY652GLIG7OQPJEQCGGA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305954019 (377h2m23.232248718s) [noise=true , address=77.90.31.21:4907 , piece_id=2EUWATHUQNZ77BJV65ZFKR5WL4D3AUURLBQ32XR2IVBA7P4TITOQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.306425517 (377h2m23.232720213s) [noise=true , address=104.36.82.22:4541 , piece_id=XN4HDFDNZ4YUJTDPOTMMDFX4FE2YVZDGCIMHEMJGWXRPNHQ6EU7A, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.375470790 (377h2m23.301765481s) [upload=301954]: Next returned nil segment
Jul 9 02:13:56.375489934 (377h2m23.30178463s) [upload=301954]: Waiting for error group managing segments...
Jul 9 02:13:56.381456859 (377h2m23.307751539s) [noise=true , address=77.90.31.10:4525 , piece_id=GPVSIZIBZWTUCP5RFYVPFYIYIL2LWU5PU3UASQVRQLY7SSSFMQBA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.381956907 (377h2m23.308251586s) [noise=true , address=77.90.31.14:4689 , piece_id=LW7OT74AJCCC3TEKY6OZI553YK6YQFQ6C7PORQB53E7FUS24KQKQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.382837088 (377h2m23.309131761s) [noise=true , address=77.90.31.21:4898 , piece_id=75XXVNT4EV4D5UTX7O5XFL44IZOYY5LZSI2VPFQZPRERL6W5M23A, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.393489125 (377h2m23.319783812s) [noise=true , address=77.90.31.21:4907 , piece_id=2EUWATHUQNZ77BJV65ZFKR5WL4D3AUURLBQ32XR2IVBA7P4TITOQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.404692801 (377h2m23.330987481s) [noise=true , address=77.90.31.16:4739 , piece_id=H6UCV6RURPXKC2H3YGAT2OXV2VZMW2DWIRG6TFD6CVXKOVMDP2VQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.404986115 (377h2m23.331280795s) [noise=true , address=77.90.31.18:4838 , piece_id=TGWYXA3AF7LDG4KPVCRNEYFDADAG6TQ5DAX7JJVW5BLVVVNECILA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.433785086 (377h2m23.360079766s) [noise=true , address=77.90.31.13:4640 , piece_id=TPPN75DX3UCBM346CSVJN3GSOA47QV4FVPYSSAHWCJYEM42OCVLQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.598499747 (377h2m23.524794427s) [noise=true , address=79.127.201.215:4339 , piece_id=XDCTYE2SVI6RAPCXTRVANFCZYWJDZXLBHHJ47NQY6V7D36DEFPGA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:11.107785584 (377h2m38.034080273s) [noise=true , address=104.36.82.22:4541 , piece_id=XN4HDFDNZ4YUJTDPOTMMDFX4FE2YVZDGCIMHEMJGWXRPNHQ6EU7A, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:12.515567029 (377h2m39.441861713s) [noise=true , address=104.36.82.17:4345 , piece_id=YI27O5WLBYO3TPCX4MVUATFORY6U3EQMDKQHGKRHIZMTG7WGFB2A, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:12.923090587 (377h2m39.849385267s) [noise=true , address=104.36.82.8:4986 , piece_id=KPY5JVLSAEB5GUKUJJSIF5M6IIHTJISEYYAXCGUSJ7OLI5QCIBNQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:13.453172659 (377h2m40.379467346s) [noise=true , address=104.36.82.38:4698 , piece_id=TSOXJNQEWODCWFEITH4IYKZI7A75UZEXRQPIQO7LMP6LT3WLNXUA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:14.416524856 (377h2m41.342819534s) [noise=true , address=104.36.82.34:4554 , piece_id=57K5SDV3YEGLVMSJGSLXNKEWOLETXE6CWBM2NPVCNKLFHPVQCJBQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:14.846730699 (377h2m41.773025379s) [noise=true , address=104.36.82.23:4464 , piece_id=JCJ4MGEZIIZ7YCTM6QXC5ZJ7PI4OEJBOMHZYBU57D2SNGWJWB5CQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:14.846763777 (377h2m41.773058463s) [seg_pos={0 0}, upload=301954]: Segment reached optimal threshold of 14 pieces.
Jul 9 02:14:14.846894905 (377h2m41.773189582s) [noise=true , address=104.36.82.36:4644 , piece_id=K2G3MAPD7TWYZPKNHQBMEPJRIQDLE75FYVPAZFX4I3GJ4IUAA7XQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1oYHbb73ACitbfkkP5P6cmRyTt6kmaS1y49G4vbsvTvVDwByhT): protocol: expected piece hash; context canceled; EOF
Jul 9 02:14:14.846948435 (377h2m41.773243131s) [noise=true , address=104.36.82.4:4834 , piece_id=YLYIASUPNRTK7KOMRYLN7OQLQBPWC7WFGCPDGUVIOEQJWQQ7A7GA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Z241DCYA5Wig2TdEogxTpY6FzVmFbnKtmHcTwxLxfcZafgooy): protocol: expected piece hash; context canceled; EOF
Jul 9 02:14:14.847094561 (377h2m41.773389249s) [noise=true , address=104.36.82.42:4830 , piece_id=ENYQTUCIF7OWQPCXTYS24OC5QT63ZJJXOY652GLIG7OQPJEQCGGA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:142ocSHHtv8BpSebXFNk5qk1Y23a8pbFAZqE3cq6i3RMX3dPw9): protocol: expected piece hash; context canceled; EOF
Jul 9 02:14:14.847127825 (377h2m41.773422511s) [noise=true , address=104.36.82.41:4768 , piece_id=BGMF33OKUZSZKGIHJYI3RNXEBDE54TX67DYLJ7JCZMRNK6ZAU22Q, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: write tcp 159.223.143.39:43796->104.36.82.41:4768: use of closed network connection; write tcp 159.223.143.39:43796->104.36.82.41:4768: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:43796->104.36.82.41:4768: use of closed network connection
Jul 9 02:14:14.847142999 (377h2m41.773437687s) [seg_pos={0 0}, upload=301954]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jul 9 02:14:14.847204962 (377h2m41.773499652s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:14:14.847209974 (377h2m41.773504671s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:14:14.909581364 (377h2m41.83587606s) [upload=301954]: Done uploading segments...
Jul 9 02:14:14.909605483 (377h2m41.835900172s) [upload=301954]: Upload finished. err: <nil>
Duration: 8.241870591s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............X............................................] duration:
477.449585ms
FAIL (2025-07-09 02:56:33 UTC)
orbiter warning:
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
02:25:27.135824454 (377h13m54.062119141s) [noise=true , address=104.36.82.12:4150 , piece_id=DRG5SBA3EE4KVQPEQOJZ6SS537SFWYFUYQXOAQMWYTT6FPVSJIPQ, seg_pos={0 0}, upload=302099]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: context canceled; context canceled
Jul 9 02:25:27.135860079 (377h13m54.062154769s) [seg_pos={0 0}, upload=302099]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 02:25:27.135957283 (377h13m54.062251971s) [upload=302099]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:25:27.135964211 (377h13m54.062258897s) [upload=302099]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:25:27.199410713 (377h13m54.125705389s) [upload=302099]: Done uploading segments...
Jul 9 02:25:27.199427901 (377h13m54.125722577s) [upload=302099]: Upload finished. err: <nil>
Jul 9 02:25:57.289077655 (377h14m24.215372329s) [upload=302109]: Starting upload
Jul 9 02:25:57.289186750 (377h14m24.215481424s) [upload=302109]: Done starting upload
Jul 9 02:25:57.289444851 (377h14m24.215739548s) [upload=302109]: Uploading segments...
Jul 9 02:25:57.289623846 (377h14m24.215918535s) [upload=302109]: Got next segment. Inline: false
Jul 9 02:25:57.289696741 (377h14m24.215991432s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:25:57.289703375 (377h14m24.215998067s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:25:57.318313152 (377h14m24.244607838s) [seg_pos={0 0}, upload=302109]: Begin upload segment...
Jul 9 02:25:57.318363286 (377h14m24.244657964s) [seg_pos={0 0}, upload=302109]: Done begin upload segment.
Jul 9 02:25:57.318386772 (377h14m24.244681463s) [noise=true , address=77.90.31.11:4553 , piece_id=TYW7NBJWACIT7OQS7A4WGXUPR3MB2FKAN2OMGAZJERB4X2DLAO2Q, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318412127 (377h14m24.244706825s) [noise=true , address=77.90.31.10:4518 , piece_id=VJ4HWM2X4VSRSUCQL5QLMBYPFBYV4JYS5ZXCBCB6MTMB76EHZFGQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318822149 (377h14m24.245116841s) [noise=true , address=45.91.116.9:4263 , piece_id=5YILAJ5BZBGOURSZE343ST4RMOTWM6ZOB3EEQQWL5WUNYWOGSAPA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318965862 (377h14m24.24526056s) [noise=true , address=45.91.116.8:4045 , piece_id=KM6ZYTHR6JGTARKZQZYIESPKBEU5H2HQFA7OARIKD2ZXDHPFSMGA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.319187516 (377h14m24.245482203s) [noise=true , address=104.36.82.7:4909 , piece_id=ZRHLWDQ5KIDRWPJ5FXAANQTWERKXCR2PPEGN442IOSYSRFAG7VZA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.319523815 (377h14m24.245818512s) [noise=true , address=45.91.116.14:4191 , piece_id=GTR6T4OTGZTEZ76FWQZ46OCH5WB3O5X27EN4AV33ZJ7D5HX4535A, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.320556632 (377h14m24.24685133s) [noise=true , address=104.36.82.2:4732 , piece_id=DJIGHJMJRAULH2L4YT2WGRXPRQJMNKZG5LICHPSKOCHUL6QDV6BQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.321019141 (377h14m24.247313835s) [noise=true , address=77.90.31.17:4770 , piece_id=V36UBBIYOFJC6KFVVBXEVT4QBUIWN257D3YSDDRAGZVXH7ZXHTBA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.322591797 (377h14m24.248886493s) [noise=true , address=77.90.31.13:4627 , piece_id=HULEKFVZNDNKS4SWWGTJUHYZWWYPDCER5W4GFCVLLDL65PAHULFQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323120451 (377h14m24.249415138s) [noise=true , address=45.91.116.4:4939 , piece_id=ANA3EZU7OZWELNG4QEMTEAYZY5H3C3H4IJFT4XKPUB7WIF44FEVQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323544591 (377h14m24.249839288s) [noise=true , address=45.91.116.7:4011 , piece_id=VDOJRN6ACK4LO44BQTULO6BB3CLAVQI7YIC6XLR32CZV4UJSPE6Q, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323950189 (377h14m24.250244889s) [noise=true , address=104.36.82.10:4078 , piece_id=ZFVUQAEV6DDFLPKCZA2XJ2BZHMMM745LGB4JRZJZQPG72WECGMDQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.326585810 (377h14m24.252880507s) [noise=true , address=45.91.116.5:4297 , piece_id=MSDWDLY5PJHQVI5565EKANJVSTMZFEUD2OTYZ52C65XQUICNMC3A, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.327871214 (377h14m24.254165904s) [noise=true , address=104.36.82.6:4875 , piece_id=C275F2JPVAF52E34CEHT5QXCHUFGIQZ7XGMOAOUOT4RA7YOHRQLQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.328605288 (377h14m24.254899986s) [noise=true , address=104.36.82.5:4838 , piece_id=WK3ALBOEMWWTVEA4FKCS352SZ6YYB35LELURNWXQZK32K4HYDRNA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.335409087 (377h14m24.261703784s) [noise=true , address=45.91.116.6:4973 , piece_id=Z7HQQ2O5QUSJKT4EMIDMEI5PIAEHNVRYX7MALUR2GQULRFIBUOKA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.475344637 (377h14m24.401639326s) [upload=302109]: Next returned nil segment
Jul 9 02:25:57.475376181 (377h14m24.401670879s) [upload=302109]: Waiting for error group managing segments...
Jul 9 02:25:57.477269278 (377h14m24.403563958s) [noise=true , address=45.91.116.7:4011 , piece_id=VDOJRN6ACK4LO44BQTULO6BB3CLAVQI7YIC6XLR32CZV4UJSPE6Q, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.477736179 (377h14m24.40403086s) [noise=true , address=45.91.116.9:4263 , piece_id=5YILAJ5BZBGOURSZE343ST4RMOTWM6ZOB3EEQQWL5WUNYWOGSAPA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.478396652 (377h14m24.404691332s) [noise=true , address=77.90.31.10:4518 , piece_id=VJ4HWM2X4VSRSUCQL5QLMBYPFBYV4JYS5ZXCBCB6MTMB76EHZFGQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.478919538 (377h14m24.405214224s) [noise=true , address=45.91.116.14:4191 , piece_id=GTR6T4OTGZTEZ76FWQZ46OCH5WB3O5X27EN4AV33ZJ7D5HX4535A, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481426716 (377h14m24.407721396s) [noise=true , address=45.91.116.4:4939 , piece_id=ANA3EZU7OZWELNG4QEMTEAYZY5H3C3H4IJFT4XKPUB7WIF44FEVQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481718373 (377h14m24.408013051s) [noise=true , address=45.91.116.5:4297 , piece_id=MSDWDLY5PJHQVI5565EKANJVSTMZFEUD2OTYZ52C65XQUICNMC3A, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481930356 (377h14m24.408225033s) [noise=true , address=77.90.31.17:4770 , piece_id=V36UBBIYOFJC6KFVVBXEVT4QBUIWN257D3YSDDRAGZVXH7ZXHTBA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.482376147 (377h14m24.408670821s) [noise=true , address=45.91.116.8:4045 , piece_id=KM6ZYTHR6JGTARKZQZYIESPKBEU5H2HQFA7OARIKD2ZXDHPFSMGA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.482439192 (377h14m24.408733868s) [noise=true , address=45.91.116.6:4973 , piece_id=Z7HQQ2O5QUSJKT4EMIDMEI5PIAEHNVRYX7MALUR2GQULRFIBUOKA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.483047970 (377h14m24.409342644s) [noise=true , address=77.90.31.13:4627 , piece_id=HULEKFVZNDNKS4SWWGTJUHYZWWYPDCER5W4GFCVLLDL65PAHULFQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.497788632 (377h14m24.424083312s) [noise=true , address=77.90.31.11:4553 , piece_id=TYW7NBJWACIT7OQS7A4WGXUPR3MB2FKAN2OMGAZJERB4X2DLAO2Q, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:16.688921171 (377h14m43.615215851s) [noise=true , address=104.36.82.6:4875 , piece_id=C275F2JPVAF52E34CEHT5QXCHUFGIQZ7XGMOAOUOT4RA7YOHRQLQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:27.177893955 (377h14m54.104188635s) [noise=true , address=104.36.82.2:4732 , piece_id=DJIGHJMJRAULH2L4YT2WGRXPRQJMNKZG5LICHPSKOCHUL6QDV6BQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:27.177985584 (377h14m54.104280281s) [seg_pos={0 0}, upload=302109]: Segment reached optimal threshold of 13 pieces.
Jul 9 02:26:27.178189789 (377h14m54.104484461s) [noise=true , address=104.36.82.5:4838 , piece_id=WK3ALBOEMWWTVEA4FKCS352SZ6YYB35LELURNWXQZK32K4HYDRNA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178252042 (377h14m54.104546733s) [noise=true , address=104.36.82.7:4909 , piece_id=ZRHLWDQ5KIDRWPJ5FXAANQTWERKXCR2PPEGN442IOSYSRFAG7VZA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178273922 (377h14m54.104568608s) [noise=true , address=104.36.82.10:4078 , piece_id=ZFVUQAEV6DDFLPKCZA2XJ2BZHMMM745LGB4JRZJZQPG72WECGMDQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178305983 (377h14m54.104600682s) [seg_pos={0 0}, upload=302109]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 02:26:27.178428768 (377h14m54.104723465s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:26:27.178455810 (377h14m54.104750501s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:26:27.232166184 (377h14m54.158460883s) [upload=302109]: Done uploading segments...
Jul 9 02:26:27.232188444 (377h14m54.158483123s) [upload=302109]: Upload finished. err: <nil>
Duration: 477.478322ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
26 out of 60 is failed (threshold=3) [X..XX.XX..X.X.X.....XXX.XX.X...XXX....XX...X..X..X.XX.X..X..] duration:
20.240178247s
FAIL (2025-07-09 02:56:25 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 20.241096208s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
38 out of 60 is failed (threshold=3) [....XX.XXX.XXX.XX.XXXXXX.XXXXX.XX.....X..XXXXX.XX..XXX.XX.XX] duration:
1m0.000421584s
FAIL (2025-07-09 02:56:21 UTC)
context deadline exceeded
Duration: 1m0.000440723s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.......X....................................................] duration:
18.739530282s
FAIL (2025-07-09 02:56:12 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
3:56.263910643 (377h2m23.19020535s) [upload=301954]: Got next segment. Inline: false
Jul 9 02:13:56.263954358 (377h2m23.190249034s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:13:56.263959891 (377h2m23.190254572s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:13:56.302204249 (377h2m23.228498944s) [seg_pos={0 0}, upload=301954]: Begin upload segment...
Jul 9 02:13:56.302291587 (377h2m23.22858628s) [seg_pos={0 0}, upload=301954]: Done begin upload segment.
Jul 9 02:13:56.302338947 (377h2m23.228633682s) [noise=true , address=77.90.31.14:4689 , piece_id=LW7OT74AJCCC3TEKY6OZI553YK6YQFQ6C7PORQB53E7FUS24KQKQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.302352599 (377h2m23.228647272s) [noise=true , address=104.36.82.8:4986 , piece_id=KPY5JVLSAEB5GUKUJJSIF5M6IIHTJISEYYAXCGUSJ7OLI5QCIBNQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.302450837 (377h2m23.228745531s) [noise=true , address=77.90.31.16:4739 , piece_id=H6UCV6RURPXKC2H3YGAT2OXV2VZMW2DWIRG6TFD6CVXKOVMDP2VQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.302958774 (377h2m23.229253667s) [noise=true , address=77.90.31.13:4640 , piece_id=TPPN75DX3UCBM346CSVJN3GSOA47QV4FVPYSSAHWCJYEM42OCVLQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303069086 (377h2m23.229363782s) [noise=true , address=104.36.82.17:4345 , piece_id=YI27O5WLBYO3TPCX4MVUATFORY6U3EQMDKQHGKRHIZMTG7WGFB2A, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303495994 (377h2m23.22979069s) [noise=true , address=104.36.82.36:4644 , piece_id=K2G3MAPD7TWYZPKNHQBMEPJRIQDLE75FYVPAZFX4I3GJ4IUAA7XQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303575840 (377h2m23.229870536s) [noise=true , address=77.90.31.21:4898 , piece_id=75XXVNT4EV4D5UTX7O5XFL44IZOYY5LZSI2VPFQZPRERL6W5M23A, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303949062 (377h2m23.230245641s) [noise=true , address=79.127.201.215:4339 , piece_id=XDCTYE2SVI6RAPCXTRVANFCZYWJDZXLBHHJ47NQY6V7D36DEFPGA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304060410 (377h2m23.230355106s) [noise=true , address=104.36.82.41:4768 , piece_id=BGMF33OKUZSZKGIHJYI3RNXEBDE54TX67DYLJ7JCZMRNK6ZAU22Q, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304407993 (377h2m23.230702685s) [noise=true , address=77.90.31.10:4525 , piece_id=GPVSIZIBZWTUCP5RFYVPFYIYIL2LWU5PU3UASQVRQLY7SSSFMQBA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304548115 (377h2m23.230842812s) [noise=true , address=104.36.82.23:4464 , piece_id=JCJ4MGEZIIZ7YCTM6QXC5ZJ7PI4OEJBOMHZYBU57D2SNGWJWB5CQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304795164 (377h2m23.231089862s) [noise=true , address=77.90.31.18:4838 , piece_id=TGWYXA3AF7LDG4KPVCRNEYFDADAG6TQ5DAX7JJVW5BLVVVNECILA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305012510 (377h2m23.231307211s) [noise=true , address=104.36.82.34:4554 , piece_id=57K5SDV3YEGLVMSJGSLXNKEWOLETXE6CWBM2NPVCNKLFHPVQCJBQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305113737 (377h2m23.231408427s) [noise=true , address=104.36.82.4:4834 , piece_id=YLYIASUPNRTK7KOMRYLN7OQLQBPWC7WFGCPDGUVIOEQJWQQ7A7GA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305383755 (377h2m23.231678434s) [noise=true , address=104.36.82.38:4698 , piece_id=TSOXJNQEWODCWFEITH4IYKZI7A75UZEXRQPIQO7LMP6LT3WLNXUA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305483822 (377h2m23.231778517s) [noise=true , address=104.36.82.42:4830 , piece_id=ENYQTUCIF7OWQPCXTYS24OC5QT63ZJJXOY652GLIG7OQPJEQCGGA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305954019 (377h2m23.232248718s) [noise=true , address=77.90.31.21:4907 , piece_id=2EUWATHUQNZ77BJV65ZFKR5WL4D3AUURLBQ32XR2IVBA7P4TITOQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.306425517 (377h2m23.232720213s) [noise=true , address=104.36.82.22:4541 , piece_id=XN4HDFDNZ4YUJTDPOTMMDFX4FE2YVZDGCIMHEMJGWXRPNHQ6EU7A, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.375470790 (377h2m23.301765481s) [upload=301954]: Next returned nil segment
Jul 9 02:13:56.375489934 (377h2m23.30178463s) [upload=301954]: Waiting for error group managing segments...
Jul 9 02:13:56.381456859 (377h2m23.307751539s) [noise=true , address=77.90.31.10:4525 , piece_id=GPVSIZIBZWTUCP5RFYVPFYIYIL2LWU5PU3UASQVRQLY7SSSFMQBA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.381956907 (377h2m23.308251586s) [noise=true , address=77.90.31.14:4689 , piece_id=LW7OT74AJCCC3TEKY6OZI553YK6YQFQ6C7PORQB53E7FUS24KQKQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.382837088 (377h2m23.309131761s) [noise=true , address=77.90.31.21:4898 , piece_id=75XXVNT4EV4D5UTX7O5XFL44IZOYY5LZSI2VPFQZPRERL6W5M23A, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.393489125 (377h2m23.319783812s) [noise=true , address=77.90.31.21:4907 , piece_id=2EUWATHUQNZ77BJV65ZFKR5WL4D3AUURLBQ32XR2IVBA7P4TITOQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.404692801 (377h2m23.330987481s) [noise=true , address=77.90.31.16:4739 , piece_id=H6UCV6RURPXKC2H3YGAT2OXV2VZMW2DWIRG6TFD6CVXKOVMDP2VQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.404986115 (377h2m23.331280795s) [noise=true , address=77.90.31.18:4838 , piece_id=TGWYXA3AF7LDG4KPVCRNEYFDADAG6TQ5DAX7JJVW5BLVVVNECILA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.433785086 (377h2m23.360079766s) [noise=true , address=77.90.31.13:4640 , piece_id=TPPN75DX3UCBM346CSVJN3GSOA47QV4FVPYSSAHWCJYEM42OCVLQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.598499747 (377h2m23.524794427s) [noise=true , address=79.127.201.215:4339 , piece_id=XDCTYE2SVI6RAPCXTRVANFCZYWJDZXLBHHJ47NQY6V7D36DEFPGA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:11.107785584 (377h2m38.034080273s) [noise=true , address=104.36.82.22:4541 , piece_id=XN4HDFDNZ4YUJTDPOTMMDFX4FE2YVZDGCIMHEMJGWXRPNHQ6EU7A, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:12.515567029 (377h2m39.441861713s) [noise=true , address=104.36.82.17:4345 , piece_id=YI27O5WLBYO3TPCX4MVUATFORY6U3EQMDKQHGKRHIZMTG7WGFB2A, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:12.923090587 (377h2m39.849385267s) [noise=true , address=104.36.82.8:4986 , piece_id=KPY5JVLSAEB5GUKUJJSIF5M6IIHTJISEYYAXCGUSJ7OLI5QCIBNQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:13.453172659 (377h2m40.379467346s) [noise=true , address=104.36.82.38:4698 , piece_id=TSOXJNQEWODCWFEITH4IYKZI7A75UZEXRQPIQO7LMP6LT3WLNXUA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:14.416524856 (377h2m41.342819534s) [noise=true , address=104.36.82.34:4554 , piece_id=57K5SDV3YEGLVMSJGSLXNKEWOLETXE6CWBM2NPVCNKLFHPVQCJBQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:14.846730699 (377h2m41.773025379s) [noise=true , address=104.36.82.23:4464 , piece_id=JCJ4MGEZIIZ7YCTM6QXC5ZJ7PI4OEJBOMHZYBU57D2SNGWJWB5CQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:14.846763777 (377h2m41.773058463s) [seg_pos={0 0}, upload=301954]: Segment reached optimal threshold of 14 pieces.
Jul 9 02:14:14.846894905 (377h2m41.773189582s) [noise=true , address=104.36.82.36:4644 , piece_id=K2G3MAPD7TWYZPKNHQBMEPJRIQDLE75FYVPAZFX4I3GJ4IUAA7XQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1oYHbb73ACitbfkkP5P6cmRyTt6kmaS1y49G4vbsvTvVDwByhT): protocol: expected piece hash; context canceled; EOF
Jul 9 02:14:14.846948435 (377h2m41.773243131s) [noise=true , address=104.36.82.4:4834 , piece_id=YLYIASUPNRTK7KOMRYLN7OQLQBPWC7WFGCPDGUVIOEQJWQQ7A7GA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Z241DCYA5Wig2TdEogxTpY6FzVmFbnKtmHcTwxLxfcZafgooy): protocol: expected piece hash; context canceled; EOF
Jul 9 02:14:14.847094561 (377h2m41.773389249s) [noise=true , address=104.36.82.42:4830 , piece_id=ENYQTUCIF7OWQPCXTYS24OC5QT63ZJJXOY652GLIG7OQPJEQCGGA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:142ocSHHtv8BpSebXFNk5qk1Y23a8pbFAZqE3cq6i3RMX3dPw9): protocol: expected piece hash; context canceled; EOF
Jul 9 02:14:14.847127825 (377h2m41.773422511s) [noise=true , address=104.36.82.41:4768 , piece_id=BGMF33OKUZSZKGIHJYI3RNXEBDE54TX67DYLJ7JCZMRNK6ZAU22Q, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: write tcp 159.223.143.39:43796->104.36.82.41:4768: use of closed network connection; write tcp 159.223.143.39:43796->104.36.82.41:4768: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:43796->104.36.82.41:4768: use of closed network connection
Jul 9 02:14:14.847142999 (377h2m41.773437687s) [seg_pos={0 0}, upload=301954]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jul 9 02:14:14.847204962 (377h2m41.773499652s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:14:14.847209974 (377h2m41.773504671s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:14:14.909581364 (377h2m41.83587606s) [upload=301954]: Done uploading segments...
Jul 9 02:14:14.909605483 (377h2m41.835900172s) [upload=301954]: Upload finished. err: <nil>
Duration: 18.73968529s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [................X...........................................] duration:
9.496234198s
FAIL (2025-07-09 02:56:00 UTC)
orbiter warning:
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
02:25:27.135824454 (377h13m54.062119141s) [noise=true , address=104.36.82.12:4150 , piece_id=DRG5SBA3EE4KVQPEQOJZ6SS537SFWYFUYQXOAQMWYTT6FPVSJIPQ, seg_pos={0 0}, upload=302099]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: context canceled; context canceled
Jul 9 02:25:27.135860079 (377h13m54.062154769s) [seg_pos={0 0}, upload=302099]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 02:25:27.135957283 (377h13m54.062251971s) [upload=302099]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:25:27.135964211 (377h13m54.062258897s) [upload=302099]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:25:27.199410713 (377h13m54.125705389s) [upload=302099]: Done uploading segments...
Jul 9 02:25:27.199427901 (377h13m54.125722577s) [upload=302099]: Upload finished. err: <nil>
Jul 9 02:25:57.289077655 (377h14m24.215372329s) [upload=302109]: Starting upload
Jul 9 02:25:57.289186750 (377h14m24.215481424s) [upload=302109]: Done starting upload
Jul 9 02:25:57.289444851 (377h14m24.215739548s) [upload=302109]: Uploading segments...
Jul 9 02:25:57.289623846 (377h14m24.215918535s) [upload=302109]: Got next segment. Inline: false
Jul 9 02:25:57.289696741 (377h14m24.215991432s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:25:57.289703375 (377h14m24.215998067s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:25:57.318313152 (377h14m24.244607838s) [seg_pos={0 0}, upload=302109]: Begin upload segment...
Jul 9 02:25:57.318363286 (377h14m24.244657964s) [seg_pos={0 0}, upload=302109]: Done begin upload segment.
Jul 9 02:25:57.318386772 (377h14m24.244681463s) [noise=true , address=77.90.31.11:4553 , piece_id=TYW7NBJWACIT7OQS7A4WGXUPR3MB2FKAN2OMGAZJERB4X2DLAO2Q, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318412127 (377h14m24.244706825s) [noise=true , address=77.90.31.10:4518 , piece_id=VJ4HWM2X4VSRSUCQL5QLMBYPFBYV4JYS5ZXCBCB6MTMB76EHZFGQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318822149 (377h14m24.245116841s) [noise=true , address=45.91.116.9:4263 , piece_id=5YILAJ5BZBGOURSZE343ST4RMOTWM6ZOB3EEQQWL5WUNYWOGSAPA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318965862 (377h14m24.24526056s) [noise=true , address=45.91.116.8:4045 , piece_id=KM6ZYTHR6JGTARKZQZYIESPKBEU5H2HQFA7OARIKD2ZXDHPFSMGA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.319187516 (377h14m24.245482203s) [noise=true , address=104.36.82.7:4909 , piece_id=ZRHLWDQ5KIDRWPJ5FXAANQTWERKXCR2PPEGN442IOSYSRFAG7VZA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.319523815 (377h14m24.245818512s) [noise=true , address=45.91.116.14:4191 , piece_id=GTR6T4OTGZTEZ76FWQZ46OCH5WB3O5X27EN4AV33ZJ7D5HX4535A, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.320556632 (377h14m24.24685133s) [noise=true , address=104.36.82.2:4732 , piece_id=DJIGHJMJRAULH2L4YT2WGRXPRQJMNKZG5LICHPSKOCHUL6QDV6BQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.321019141 (377h14m24.247313835s) [noise=true , address=77.90.31.17:4770 , piece_id=V36UBBIYOFJC6KFVVBXEVT4QBUIWN257D3YSDDRAGZVXH7ZXHTBA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.322591797 (377h14m24.248886493s) [noise=true , address=77.90.31.13:4627 , piece_id=HULEKFVZNDNKS4SWWGTJUHYZWWYPDCER5W4GFCVLLDL65PAHULFQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323120451 (377h14m24.249415138s) [noise=true , address=45.91.116.4:4939 , piece_id=ANA3EZU7OZWELNG4QEMTEAYZY5H3C3H4IJFT4XKPUB7WIF44FEVQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323544591 (377h14m24.249839288s) [noise=true , address=45.91.116.7:4011 , piece_id=VDOJRN6ACK4LO44BQTULO6BB3CLAVQI7YIC6XLR32CZV4UJSPE6Q, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323950189 (377h14m24.250244889s) [noise=true , address=104.36.82.10:4078 , piece_id=ZFVUQAEV6DDFLPKCZA2XJ2BZHMMM745LGB4JRZJZQPG72WECGMDQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.326585810 (377h14m24.252880507s) [noise=true , address=45.91.116.5:4297 , piece_id=MSDWDLY5PJHQVI5565EKANJVSTMZFEUD2OTYZ52C65XQUICNMC3A, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.327871214 (377h14m24.254165904s) [noise=true , address=104.36.82.6:4875 , piece_id=C275F2JPVAF52E34CEHT5QXCHUFGIQZ7XGMOAOUOT4RA7YOHRQLQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.328605288 (377h14m24.254899986s) [noise=true , address=104.36.82.5:4838 , piece_id=WK3ALBOEMWWTVEA4FKCS352SZ6YYB35LELURNWXQZK32K4HYDRNA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.335409087 (377h14m24.261703784s) [noise=true , address=45.91.116.6:4973 , piece_id=Z7HQQ2O5QUSJKT4EMIDMEI5PIAEHNVRYX7MALUR2GQULRFIBUOKA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.475344637 (377h14m24.401639326s) [upload=302109]: Next returned nil segment
Jul 9 02:25:57.475376181 (377h14m24.401670879s) [upload=302109]: Waiting for error group managing segments...
Jul 9 02:25:57.477269278 (377h14m24.403563958s) [noise=true , address=45.91.116.7:4011 , piece_id=VDOJRN6ACK4LO44BQTULO6BB3CLAVQI7YIC6XLR32CZV4UJSPE6Q, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.477736179 (377h14m24.40403086s) [noise=true , address=45.91.116.9:4263 , piece_id=5YILAJ5BZBGOURSZE343ST4RMOTWM6ZOB3EEQQWL5WUNYWOGSAPA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.478396652 (377h14m24.404691332s) [noise=true , address=77.90.31.10:4518 , piece_id=VJ4HWM2X4VSRSUCQL5QLMBYPFBYV4JYS5ZXCBCB6MTMB76EHZFGQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.478919538 (377h14m24.405214224s) [noise=true , address=45.91.116.14:4191 , piece_id=GTR6T4OTGZTEZ76FWQZ46OCH5WB3O5X27EN4AV33ZJ7D5HX4535A, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481426716 (377h14m24.407721396s) [noise=true , address=45.91.116.4:4939 , piece_id=ANA3EZU7OZWELNG4QEMTEAYZY5H3C3H4IJFT4XKPUB7WIF44FEVQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481718373 (377h14m24.408013051s) [noise=true , address=45.91.116.5:4297 , piece_id=MSDWDLY5PJHQVI5565EKANJVSTMZFEUD2OTYZ52C65XQUICNMC3A, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481930356 (377h14m24.408225033s) [noise=true , address=77.90.31.17:4770 , piece_id=V36UBBIYOFJC6KFVVBXEVT4QBUIWN257D3YSDDRAGZVXH7ZXHTBA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.482376147 (377h14m24.408670821s) [noise=true , address=45.91.116.8:4045 , piece_id=KM6ZYTHR6JGTARKZQZYIESPKBEU5H2HQFA7OARIKD2ZXDHPFSMGA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.482439192 (377h14m24.408733868s) [noise=true , address=45.91.116.6:4973 , piece_id=Z7HQQ2O5QUSJKT4EMIDMEI5PIAEHNVRYX7MALUR2GQULRFIBUOKA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.483047970 (377h14m24.409342644s) [noise=true , address=77.90.31.13:4627 , piece_id=HULEKFVZNDNKS4SWWGTJUHYZWWYPDCER5W4GFCVLLDL65PAHULFQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.497788632 (377h14m24.424083312s) [noise=true , address=77.90.31.11:4553 , piece_id=TYW7NBJWACIT7OQS7A4WGXUPR3MB2FKAN2OMGAZJERB4X2DLAO2Q, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:16.688921171 (377h14m43.615215851s) [noise=true , address=104.36.82.6:4875 , piece_id=C275F2JPVAF52E34CEHT5QXCHUFGIQZ7XGMOAOUOT4RA7YOHRQLQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:27.177893955 (377h14m54.104188635s) [noise=true , address=104.36.82.2:4732 , piece_id=DJIGHJMJRAULH2L4YT2WGRXPRQJMNKZG5LICHPSKOCHUL6QDV6BQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:27.177985584 (377h14m54.104280281s) [seg_pos={0 0}, upload=302109]: Segment reached optimal threshold of 13 pieces.
Jul 9 02:26:27.178189789 (377h14m54.104484461s) [noise=true , address=104.36.82.5:4838 , piece_id=WK3ALBOEMWWTVEA4FKCS352SZ6YYB35LELURNWXQZK32K4HYDRNA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178252042 (377h14m54.104546733s) [noise=true , address=104.36.82.7:4909 , piece_id=ZRHLWDQ5KIDRWPJ5FXAANQTWERKXCR2PPEGN442IOSYSRFAG7VZA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178273922 (377h14m54.104568608s) [noise=true , address=104.36.82.10:4078 , piece_id=ZFVUQAEV6DDFLPKCZA2XJ2BZHMMM745LGB4JRZJZQPG72WECGMDQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178305983 (377h14m54.104600682s) [seg_pos={0 0}, upload=302109]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 02:26:27.178428768 (377h14m54.104723465s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:26:27.178455810 (377h14m54.104750501s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:26:27.232166184 (377h14m54.158460883s) [upload=302109]: Done uploading segments...
Jul 9 02:26:27.232188444 (377h14m54.158483123s) [upload=302109]: Upload finished. err: <nil>
Duration: 9.49628598s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [........X...................................................] duration:
31.539654397s
FAIL (2025-07-09 02:55:37 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
3:56.263910643 (377h2m23.19020535s) [upload=301954]: Got next segment. Inline: false
Jul 9 02:13:56.263954358 (377h2m23.190249034s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:13:56.263959891 (377h2m23.190254572s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:13:56.302204249 (377h2m23.228498944s) [seg_pos={0 0}, upload=301954]: Begin upload segment...
Jul 9 02:13:56.302291587 (377h2m23.22858628s) [seg_pos={0 0}, upload=301954]: Done begin upload segment.
Jul 9 02:13:56.302338947 (377h2m23.228633682s) [noise=true , address=77.90.31.14:4689 , piece_id=LW7OT74AJCCC3TEKY6OZI553YK6YQFQ6C7PORQB53E7FUS24KQKQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.302352599 (377h2m23.228647272s) [noise=true , address=104.36.82.8:4986 , piece_id=KPY5JVLSAEB5GUKUJJSIF5M6IIHTJISEYYAXCGUSJ7OLI5QCIBNQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.302450837 (377h2m23.228745531s) [noise=true , address=77.90.31.16:4739 , piece_id=H6UCV6RURPXKC2H3YGAT2OXV2VZMW2DWIRG6TFD6CVXKOVMDP2VQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.302958774 (377h2m23.229253667s) [noise=true , address=77.90.31.13:4640 , piece_id=TPPN75DX3UCBM346CSVJN3GSOA47QV4FVPYSSAHWCJYEM42OCVLQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303069086 (377h2m23.229363782s) [noise=true , address=104.36.82.17:4345 , piece_id=YI27O5WLBYO3TPCX4MVUATFORY6U3EQMDKQHGKRHIZMTG7WGFB2A, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303495994 (377h2m23.22979069s) [noise=true , address=104.36.82.36:4644 , piece_id=K2G3MAPD7TWYZPKNHQBMEPJRIQDLE75FYVPAZFX4I3GJ4IUAA7XQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303575840 (377h2m23.229870536s) [noise=true , address=77.90.31.21:4898 , piece_id=75XXVNT4EV4D5UTX7O5XFL44IZOYY5LZSI2VPFQZPRERL6W5M23A, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303949062 (377h2m23.230245641s) [noise=true , address=79.127.201.215:4339 , piece_id=XDCTYE2SVI6RAPCXTRVANFCZYWJDZXLBHHJ47NQY6V7D36DEFPGA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304060410 (377h2m23.230355106s) [noise=true , address=104.36.82.41:4768 , piece_id=BGMF33OKUZSZKGIHJYI3RNXEBDE54TX67DYLJ7JCZMRNK6ZAU22Q, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304407993 (377h2m23.230702685s) [noise=true , address=77.90.31.10:4525 , piece_id=GPVSIZIBZWTUCP5RFYVPFYIYIL2LWU5PU3UASQVRQLY7SSSFMQBA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304548115 (377h2m23.230842812s) [noise=true , address=104.36.82.23:4464 , piece_id=JCJ4MGEZIIZ7YCTM6QXC5ZJ7PI4OEJBOMHZYBU57D2SNGWJWB5CQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304795164 (377h2m23.231089862s) [noise=true , address=77.90.31.18:4838 , piece_id=TGWYXA3AF7LDG4KPVCRNEYFDADAG6TQ5DAX7JJVW5BLVVVNECILA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305012510 (377h2m23.231307211s) [noise=true , address=104.36.82.34:4554 , piece_id=57K5SDV3YEGLVMSJGSLXNKEWOLETXE6CWBM2NPVCNKLFHPVQCJBQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305113737 (377h2m23.231408427s) [noise=true , address=104.36.82.4:4834 , piece_id=YLYIASUPNRTK7KOMRYLN7OQLQBPWC7WFGCPDGUVIOEQJWQQ7A7GA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305383755 (377h2m23.231678434s) [noise=true , address=104.36.82.38:4698 , piece_id=TSOXJNQEWODCWFEITH4IYKZI7A75UZEXRQPIQO7LMP6LT3WLNXUA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305483822 (377h2m23.231778517s) [noise=true , address=104.36.82.42:4830 , piece_id=ENYQTUCIF7OWQPCXTYS24OC5QT63ZJJXOY652GLIG7OQPJEQCGGA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305954019 (377h2m23.232248718s) [noise=true , address=77.90.31.21:4907 , piece_id=2EUWATHUQNZ77BJV65ZFKR5WL4D3AUURLBQ32XR2IVBA7P4TITOQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.306425517 (377h2m23.232720213s) [noise=true , address=104.36.82.22:4541 , piece_id=XN4HDFDNZ4YUJTDPOTMMDFX4FE2YVZDGCIMHEMJGWXRPNHQ6EU7A, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.375470790 (377h2m23.301765481s) [upload=301954]: Next returned nil segment
Jul 9 02:13:56.375489934 (377h2m23.30178463s) [upload=301954]: Waiting for error group managing segments...
Jul 9 02:13:56.381456859 (377h2m23.307751539s) [noise=true , address=77.90.31.10:4525 , piece_id=GPVSIZIBZWTUCP5RFYVPFYIYIL2LWU5PU3UASQVRQLY7SSSFMQBA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.381956907 (377h2m23.308251586s) [noise=true , address=77.90.31.14:4689 , piece_id=LW7OT74AJCCC3TEKY6OZI553YK6YQFQ6C7PORQB53E7FUS24KQKQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.382837088 (377h2m23.309131761s) [noise=true , address=77.90.31.21:4898 , piece_id=75XXVNT4EV4D5UTX7O5XFL44IZOYY5LZSI2VPFQZPRERL6W5M23A, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.393489125 (377h2m23.319783812s) [noise=true , address=77.90.31.21:4907 , piece_id=2EUWATHUQNZ77BJV65ZFKR5WL4D3AUURLBQ32XR2IVBA7P4TITOQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.404692801 (377h2m23.330987481s) [noise=true , address=77.90.31.16:4739 , piece_id=H6UCV6RURPXKC2H3YGAT2OXV2VZMW2DWIRG6TFD6CVXKOVMDP2VQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.404986115 (377h2m23.331280795s) [noise=true , address=77.90.31.18:4838 , piece_id=TGWYXA3AF7LDG4KPVCRNEYFDADAG6TQ5DAX7JJVW5BLVVVNECILA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.433785086 (377h2m23.360079766s) [noise=true , address=77.90.31.13:4640 , piece_id=TPPN75DX3UCBM346CSVJN3GSOA47QV4FVPYSSAHWCJYEM42OCVLQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.598499747 (377h2m23.524794427s) [noise=true , address=79.127.201.215:4339 , piece_id=XDCTYE2SVI6RAPCXTRVANFCZYWJDZXLBHHJ47NQY6V7D36DEFPGA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:11.107785584 (377h2m38.034080273s) [noise=true , address=104.36.82.22:4541 , piece_id=XN4HDFDNZ4YUJTDPOTMMDFX4FE2YVZDGCIMHEMJGWXRPNHQ6EU7A, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:12.515567029 (377h2m39.441861713s) [noise=true , address=104.36.82.17:4345 , piece_id=YI27O5WLBYO3TPCX4MVUATFORY6U3EQMDKQHGKRHIZMTG7WGFB2A, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:12.923090587 (377h2m39.849385267s) [noise=true , address=104.36.82.8:4986 , piece_id=KPY5JVLSAEB5GUKUJJSIF5M6IIHTJISEYYAXCGUSJ7OLI5QCIBNQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:13.453172659 (377h2m40.379467346s) [noise=true , address=104.36.82.38:4698 , piece_id=TSOXJNQEWODCWFEITH4IYKZI7A75UZEXRQPIQO7LMP6LT3WLNXUA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:14.416524856 (377h2m41.342819534s) [noise=true , address=104.36.82.34:4554 , piece_id=57K5SDV3YEGLVMSJGSLXNKEWOLETXE6CWBM2NPVCNKLFHPVQCJBQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:14.846730699 (377h2m41.773025379s) [noise=true , address=104.36.82.23:4464 , piece_id=JCJ4MGEZIIZ7YCTM6QXC5ZJ7PI4OEJBOMHZYBU57D2SNGWJWB5CQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:14.846763777 (377h2m41.773058463s) [seg_pos={0 0}, upload=301954]: Segment reached optimal threshold of 14 pieces.
Jul 9 02:14:14.846894905 (377h2m41.773189582s) [noise=true , address=104.36.82.36:4644 , piece_id=K2G3MAPD7TWYZPKNHQBMEPJRIQDLE75FYVPAZFX4I3GJ4IUAA7XQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1oYHbb73ACitbfkkP5P6cmRyTt6kmaS1y49G4vbsvTvVDwByhT): protocol: expected piece hash; context canceled; EOF
Jul 9 02:14:14.846948435 (377h2m41.773243131s) [noise=true , address=104.36.82.4:4834 , piece_id=YLYIASUPNRTK7KOMRYLN7OQLQBPWC7WFGCPDGUVIOEQJWQQ7A7GA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Z241DCYA5Wig2TdEogxTpY6FzVmFbnKtmHcTwxLxfcZafgooy): protocol: expected piece hash; context canceled; EOF
Jul 9 02:14:14.847094561 (377h2m41.773389249s) [noise=true , address=104.36.82.42:4830 , piece_id=ENYQTUCIF7OWQPCXTYS24OC5QT63ZJJXOY652GLIG7OQPJEQCGGA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:142ocSHHtv8BpSebXFNk5qk1Y23a8pbFAZqE3cq6i3RMX3dPw9): protocol: expected piece hash; context canceled; EOF
Jul 9 02:14:14.847127825 (377h2m41.773422511s) [noise=true , address=104.36.82.41:4768 , piece_id=BGMF33OKUZSZKGIHJYI3RNXEBDE54TX67DYLJ7JCZMRNK6ZAU22Q, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: write tcp 159.223.143.39:43796->104.36.82.41:4768: use of closed network connection; write tcp 159.223.143.39:43796->104.36.82.41:4768: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:43796->104.36.82.41:4768: use of closed network connection
Jul 9 02:14:14.847142999 (377h2m41.773437687s) [seg_pos={0 0}, upload=301954]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jul 9 02:14:14.847204962 (377h2m41.773499652s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:14:14.847209974 (377h2m41.773504671s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:14:14.909581364 (377h2m41.83587606s) [upload=301954]: Done uploading segments...
Jul 9 02:14:14.909605483 (377h2m41.835900172s) [upload=301954]: Upload finished. err: <nil>
Duration: 31.539770038s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
26 out of 60 is failed (threshold=3) [.X..XX.XX..X.X.X.....XXX.XX.X...XXX....XX...X..X..X.XX.X..X.] duration:
7.660812132s
FAIL (2025-07-09 02:55:32 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 7.660834982s
edge
gateway
us-select-1
|