Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
22 out of 60 is failed (threshold=3) [..X.........X....X.X..XX.XX..X.X.X.....XXX.XX.X...XXX....XX.] duration:
21.023291626s
FAIL (2025-07-09 02:41:58 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 21.023305719s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
22 out of 60 is failed (threshold=3) [...X.........X....X.X..XX.XX..X.X.X.....XXX.XX.X...XXX....XX] duration:
30.001458242s
FAIL (2025-07-09 02:41:37 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 30.001482331s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
4 out of 60 is failed (threshold=3) [......X.....X...X....................X......................] duration:
16.424606915s
FAIL (2025-07-09 02:41:27 UTC)
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
02:25:27.135824454 (377h13m54.062119141s) [noise=true , address=104.36.82.12:4150 , piece_id=DRG5SBA3EE4KVQPEQOJZ6SS537SFWYFUYQXOAQMWYTT6FPVSJIPQ, seg_pos={0 0}, upload=302099]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: context canceled; context canceled
Jul 9 02:25:27.135860079 (377h13m54.062154769s) [seg_pos={0 0}, upload=302099]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 02:25:27.135957283 (377h13m54.062251971s) [upload=302099]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:25:27.135964211 (377h13m54.062258897s) [upload=302099]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:25:27.199410713 (377h13m54.125705389s) [upload=302099]: Done uploading segments...
Jul 9 02:25:27.199427901 (377h13m54.125722577s) [upload=302099]: Upload finished. err: <nil>
Jul 9 02:25:57.289077655 (377h14m24.215372329s) [upload=302109]: Starting upload
Jul 9 02:25:57.289186750 (377h14m24.215481424s) [upload=302109]: Done starting upload
Jul 9 02:25:57.289444851 (377h14m24.215739548s) [upload=302109]: Uploading segments...
Jul 9 02:25:57.289623846 (377h14m24.215918535s) [upload=302109]: Got next segment. Inline: false
Jul 9 02:25:57.289696741 (377h14m24.215991432s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:25:57.289703375 (377h14m24.215998067s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:25:57.318313152 (377h14m24.244607838s) [seg_pos={0 0}, upload=302109]: Begin upload segment...
Jul 9 02:25:57.318363286 (377h14m24.244657964s) [seg_pos={0 0}, upload=302109]: Done begin upload segment.
Jul 9 02:25:57.318386772 (377h14m24.244681463s) [noise=true , address=77.90.31.11:4553 , piece_id=TYW7NBJWACIT7OQS7A4WGXUPR3MB2FKAN2OMGAZJERB4X2DLAO2Q, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318412127 (377h14m24.244706825s) [noise=true , address=77.90.31.10:4518 , piece_id=VJ4HWM2X4VSRSUCQL5QLMBYPFBYV4JYS5ZXCBCB6MTMB76EHZFGQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318822149 (377h14m24.245116841s) [noise=true , address=45.91.116.9:4263 , piece_id=5YILAJ5BZBGOURSZE343ST4RMOTWM6ZOB3EEQQWL5WUNYWOGSAPA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318965862 (377h14m24.24526056s) [noise=true , address=45.91.116.8:4045 , piece_id=KM6ZYTHR6JGTARKZQZYIESPKBEU5H2HQFA7OARIKD2ZXDHPFSMGA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.319187516 (377h14m24.245482203s) [noise=true , address=104.36.82.7:4909 , piece_id=ZRHLWDQ5KIDRWPJ5FXAANQTWERKXCR2PPEGN442IOSYSRFAG7VZA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.319523815 (377h14m24.245818512s) [noise=true , address=45.91.116.14:4191 , piece_id=GTR6T4OTGZTEZ76FWQZ46OCH5WB3O5X27EN4AV33ZJ7D5HX4535A, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.320556632 (377h14m24.24685133s) [noise=true , address=104.36.82.2:4732 , piece_id=DJIGHJMJRAULH2L4YT2WGRXPRQJMNKZG5LICHPSKOCHUL6QDV6BQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.321019141 (377h14m24.247313835s) [noise=true , address=77.90.31.17:4770 , piece_id=V36UBBIYOFJC6KFVVBXEVT4QBUIWN257D3YSDDRAGZVXH7ZXHTBA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.322591797 (377h14m24.248886493s) [noise=true , address=77.90.31.13:4627 , piece_id=HULEKFVZNDNKS4SWWGTJUHYZWWYPDCER5W4GFCVLLDL65PAHULFQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323120451 (377h14m24.249415138s) [noise=true , address=45.91.116.4:4939 , piece_id=ANA3EZU7OZWELNG4QEMTEAYZY5H3C3H4IJFT4XKPUB7WIF44FEVQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323544591 (377h14m24.249839288s) [noise=true , address=45.91.116.7:4011 , piece_id=VDOJRN6ACK4LO44BQTULO6BB3CLAVQI7YIC6XLR32CZV4UJSPE6Q, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323950189 (377h14m24.250244889s) [noise=true , address=104.36.82.10:4078 , piece_id=ZFVUQAEV6DDFLPKCZA2XJ2BZHMMM745LGB4JRZJZQPG72WECGMDQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.326585810 (377h14m24.252880507s) [noise=true , address=45.91.116.5:4297 , piece_id=MSDWDLY5PJHQVI5565EKANJVSTMZFEUD2OTYZ52C65XQUICNMC3A, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.327871214 (377h14m24.254165904s) [noise=true , address=104.36.82.6:4875 , piece_id=C275F2JPVAF52E34CEHT5QXCHUFGIQZ7XGMOAOUOT4RA7YOHRQLQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.328605288 (377h14m24.254899986s) [noise=true , address=104.36.82.5:4838 , piece_id=WK3ALBOEMWWTVEA4FKCS352SZ6YYB35LELURNWXQZK32K4HYDRNA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.335409087 (377h14m24.261703784s) [noise=true , address=45.91.116.6:4973 , piece_id=Z7HQQ2O5QUSJKT4EMIDMEI5PIAEHNVRYX7MALUR2GQULRFIBUOKA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.475344637 (377h14m24.401639326s) [upload=302109]: Next returned nil segment
Jul 9 02:25:57.475376181 (377h14m24.401670879s) [upload=302109]: Waiting for error group managing segments...
Jul 9 02:25:57.477269278 (377h14m24.403563958s) [noise=true , address=45.91.116.7:4011 , piece_id=VDOJRN6ACK4LO44BQTULO6BB3CLAVQI7YIC6XLR32CZV4UJSPE6Q, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.477736179 (377h14m24.40403086s) [noise=true , address=45.91.116.9:4263 , piece_id=5YILAJ5BZBGOURSZE343ST4RMOTWM6ZOB3EEQQWL5WUNYWOGSAPA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.478396652 (377h14m24.404691332s) [noise=true , address=77.90.31.10:4518 , piece_id=VJ4HWM2X4VSRSUCQL5QLMBYPFBYV4JYS5ZXCBCB6MTMB76EHZFGQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.478919538 (377h14m24.405214224s) [noise=true , address=45.91.116.14:4191 , piece_id=GTR6T4OTGZTEZ76FWQZ46OCH5WB3O5X27EN4AV33ZJ7D5HX4535A, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481426716 (377h14m24.407721396s) [noise=true , address=45.91.116.4:4939 , piece_id=ANA3EZU7OZWELNG4QEMTEAYZY5H3C3H4IJFT4XKPUB7WIF44FEVQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481718373 (377h14m24.408013051s) [noise=true , address=45.91.116.5:4297 , piece_id=MSDWDLY5PJHQVI5565EKANJVSTMZFEUD2OTYZ52C65XQUICNMC3A, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481930356 (377h14m24.408225033s) [noise=true , address=77.90.31.17:4770 , piece_id=V36UBBIYOFJC6KFVVBXEVT4QBUIWN257D3YSDDRAGZVXH7ZXHTBA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.482376147 (377h14m24.408670821s) [noise=true , address=45.91.116.8:4045 , piece_id=KM6ZYTHR6JGTARKZQZYIESPKBEU5H2HQFA7OARIKD2ZXDHPFSMGA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.482439192 (377h14m24.408733868s) [noise=true , address=45.91.116.6:4973 , piece_id=Z7HQQ2O5QUSJKT4EMIDMEI5PIAEHNVRYX7MALUR2GQULRFIBUOKA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.483047970 (377h14m24.409342644s) [noise=true , address=77.90.31.13:4627 , piece_id=HULEKFVZNDNKS4SWWGTJUHYZWWYPDCER5W4GFCVLLDL65PAHULFQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.497788632 (377h14m24.424083312s) [noise=true , address=77.90.31.11:4553 , piece_id=TYW7NBJWACIT7OQS7A4WGXUPR3MB2FKAN2OMGAZJERB4X2DLAO2Q, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:16.688921171 (377h14m43.615215851s) [noise=true , address=104.36.82.6:4875 , piece_id=C275F2JPVAF52E34CEHT5QXCHUFGIQZ7XGMOAOUOT4RA7YOHRQLQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:27.177893955 (377h14m54.104188635s) [noise=true , address=104.36.82.2:4732 , piece_id=DJIGHJMJRAULH2L4YT2WGRXPRQJMNKZG5LICHPSKOCHUL6QDV6BQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:27.177985584 (377h14m54.104280281s) [seg_pos={0 0}, upload=302109]: Segment reached optimal threshold of 13 pieces.
Jul 9 02:26:27.178189789 (377h14m54.104484461s) [noise=true , address=104.36.82.5:4838 , piece_id=WK3ALBOEMWWTVEA4FKCS352SZ6YYB35LELURNWXQZK32K4HYDRNA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178252042 (377h14m54.104546733s) [noise=true , address=104.36.82.7:4909 , piece_id=ZRHLWDQ5KIDRWPJ5FXAANQTWERKXCR2PPEGN442IOSYSRFAG7VZA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178273922 (377h14m54.104568608s) [noise=true , address=104.36.82.10:4078 , piece_id=ZFVUQAEV6DDFLPKCZA2XJ2BZHMMM745LGB4JRZJZQPG72WECGMDQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178305983 (377h14m54.104600682s) [seg_pos={0 0}, upload=302109]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 02:26:27.178428768 (377h14m54.104723465s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:26:27.178455810 (377h14m54.104750501s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:26:27.232166184 (377h14m54.158460883s) [upload=302109]: Done uploading segments...
Jul 9 02:26:27.232188444 (377h14m54.158483123s) [upload=302109]: Upload finished. err: <nil>
Duration: 16.424638507s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
39 out of 60 is failed (threshold=3) [X.X.XXXXXXX..XX....XX.XXX.XXX.XX.XXXXXX.XXXXX.XX.....X..XXXX] duration:
1m0.000500527s
FAIL (2025-07-09 02:41:22 UTC)
operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1m0.000523097s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
21 out of 60 is failed (threshold=3) [....X.........X....X.X..XX.XX..X.X.X.....XXX.XX.X...XXX....X] duration:
30.001169204s
FAIL (2025-07-09 02:40:56 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 30.001246027s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [...........................X................................] duration:
21.676042706s
FAIL (2025-07-09 02:40:50 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
3:56.263910643 (377h2m23.19020535s) [upload=301954]: Got next segment. Inline: false
Jul 9 02:13:56.263954358 (377h2m23.190249034s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:13:56.263959891 (377h2m23.190254572s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:13:56.302204249 (377h2m23.228498944s) [seg_pos={0 0}, upload=301954]: Begin upload segment...
Jul 9 02:13:56.302291587 (377h2m23.22858628s) [seg_pos={0 0}, upload=301954]: Done begin upload segment.
Jul 9 02:13:56.302338947 (377h2m23.228633682s) [noise=true , address=77.90.31.14:4689 , piece_id=LW7OT74AJCCC3TEKY6OZI553YK6YQFQ6C7PORQB53E7FUS24KQKQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.302352599 (377h2m23.228647272s) [noise=true , address=104.36.82.8:4986 , piece_id=KPY5JVLSAEB5GUKUJJSIF5M6IIHTJISEYYAXCGUSJ7OLI5QCIBNQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.302450837 (377h2m23.228745531s) [noise=true , address=77.90.31.16:4739 , piece_id=H6UCV6RURPXKC2H3YGAT2OXV2VZMW2DWIRG6TFD6CVXKOVMDP2VQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.302958774 (377h2m23.229253667s) [noise=true , address=77.90.31.13:4640 , piece_id=TPPN75DX3UCBM346CSVJN3GSOA47QV4FVPYSSAHWCJYEM42OCVLQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303069086 (377h2m23.229363782s) [noise=true , address=104.36.82.17:4345 , piece_id=YI27O5WLBYO3TPCX4MVUATFORY6U3EQMDKQHGKRHIZMTG7WGFB2A, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303495994 (377h2m23.22979069s) [noise=true , address=104.36.82.36:4644 , piece_id=K2G3MAPD7TWYZPKNHQBMEPJRIQDLE75FYVPAZFX4I3GJ4IUAA7XQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303575840 (377h2m23.229870536s) [noise=true , address=77.90.31.21:4898 , piece_id=75XXVNT4EV4D5UTX7O5XFL44IZOYY5LZSI2VPFQZPRERL6W5M23A, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303949062 (377h2m23.230245641s) [noise=true , address=79.127.201.215:4339 , piece_id=XDCTYE2SVI6RAPCXTRVANFCZYWJDZXLBHHJ47NQY6V7D36DEFPGA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304060410 (377h2m23.230355106s) [noise=true , address=104.36.82.41:4768 , piece_id=BGMF33OKUZSZKGIHJYI3RNXEBDE54TX67DYLJ7JCZMRNK6ZAU22Q, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304407993 (377h2m23.230702685s) [noise=true , address=77.90.31.10:4525 , piece_id=GPVSIZIBZWTUCP5RFYVPFYIYIL2LWU5PU3UASQVRQLY7SSSFMQBA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304548115 (377h2m23.230842812s) [noise=true , address=104.36.82.23:4464 , piece_id=JCJ4MGEZIIZ7YCTM6QXC5ZJ7PI4OEJBOMHZYBU57D2SNGWJWB5CQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304795164 (377h2m23.231089862s) [noise=true , address=77.90.31.18:4838 , piece_id=TGWYXA3AF7LDG4KPVCRNEYFDADAG6TQ5DAX7JJVW5BLVVVNECILA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305012510 (377h2m23.231307211s) [noise=true , address=104.36.82.34:4554 , piece_id=57K5SDV3YEGLVMSJGSLXNKEWOLETXE6CWBM2NPVCNKLFHPVQCJBQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305113737 (377h2m23.231408427s) [noise=true , address=104.36.82.4:4834 , piece_id=YLYIASUPNRTK7KOMRYLN7OQLQBPWC7WFGCPDGUVIOEQJWQQ7A7GA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305383755 (377h2m23.231678434s) [noise=true , address=104.36.82.38:4698 , piece_id=TSOXJNQEWODCWFEITH4IYKZI7A75UZEXRQPIQO7LMP6LT3WLNXUA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305483822 (377h2m23.231778517s) [noise=true , address=104.36.82.42:4830 , piece_id=ENYQTUCIF7OWQPCXTYS24OC5QT63ZJJXOY652GLIG7OQPJEQCGGA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305954019 (377h2m23.232248718s) [noise=true , address=77.90.31.21:4907 , piece_id=2EUWATHUQNZ77BJV65ZFKR5WL4D3AUURLBQ32XR2IVBA7P4TITOQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.306425517 (377h2m23.232720213s) [noise=true , address=104.36.82.22:4541 , piece_id=XN4HDFDNZ4YUJTDPOTMMDFX4FE2YVZDGCIMHEMJGWXRPNHQ6EU7A, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.375470790 (377h2m23.301765481s) [upload=301954]: Next returned nil segment
Jul 9 02:13:56.375489934 (377h2m23.30178463s) [upload=301954]: Waiting for error group managing segments...
Jul 9 02:13:56.381456859 (377h2m23.307751539s) [noise=true , address=77.90.31.10:4525 , piece_id=GPVSIZIBZWTUCP5RFYVPFYIYIL2LWU5PU3UASQVRQLY7SSSFMQBA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.381956907 (377h2m23.308251586s) [noise=true , address=77.90.31.14:4689 , piece_id=LW7OT74AJCCC3TEKY6OZI553YK6YQFQ6C7PORQB53E7FUS24KQKQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.382837088 (377h2m23.309131761s) [noise=true , address=77.90.31.21:4898 , piece_id=75XXVNT4EV4D5UTX7O5XFL44IZOYY5LZSI2VPFQZPRERL6W5M23A, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.393489125 (377h2m23.319783812s) [noise=true , address=77.90.31.21:4907 , piece_id=2EUWATHUQNZ77BJV65ZFKR5WL4D3AUURLBQ32XR2IVBA7P4TITOQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.404692801 (377h2m23.330987481s) [noise=true , address=77.90.31.16:4739 , piece_id=H6UCV6RURPXKC2H3YGAT2OXV2VZMW2DWIRG6TFD6CVXKOVMDP2VQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.404986115 (377h2m23.331280795s) [noise=true , address=77.90.31.18:4838 , piece_id=TGWYXA3AF7LDG4KPVCRNEYFDADAG6TQ5DAX7JJVW5BLVVVNECILA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.433785086 (377h2m23.360079766s) [noise=true , address=77.90.31.13:4640 , piece_id=TPPN75DX3UCBM346CSVJN3GSOA47QV4FVPYSSAHWCJYEM42OCVLQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.598499747 (377h2m23.524794427s) [noise=true , address=79.127.201.215:4339 , piece_id=XDCTYE2SVI6RAPCXTRVANFCZYWJDZXLBHHJ47NQY6V7D36DEFPGA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:11.107785584 (377h2m38.034080273s) [noise=true , address=104.36.82.22:4541 , piece_id=XN4HDFDNZ4YUJTDPOTMMDFX4FE2YVZDGCIMHEMJGWXRPNHQ6EU7A, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:12.515567029 (377h2m39.441861713s) [noise=true , address=104.36.82.17:4345 , piece_id=YI27O5WLBYO3TPCX4MVUATFORY6U3EQMDKQHGKRHIZMTG7WGFB2A, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:12.923090587 (377h2m39.849385267s) [noise=true , address=104.36.82.8:4986 , piece_id=KPY5JVLSAEB5GUKUJJSIF5M6IIHTJISEYYAXCGUSJ7OLI5QCIBNQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:13.453172659 (377h2m40.379467346s) [noise=true , address=104.36.82.38:4698 , piece_id=TSOXJNQEWODCWFEITH4IYKZI7A75UZEXRQPIQO7LMP6LT3WLNXUA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:14.416524856 (377h2m41.342819534s) [noise=true , address=104.36.82.34:4554 , piece_id=57K5SDV3YEGLVMSJGSLXNKEWOLETXE6CWBM2NPVCNKLFHPVQCJBQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:14.846730699 (377h2m41.773025379s) [noise=true , address=104.36.82.23:4464 , piece_id=JCJ4MGEZIIZ7YCTM6QXC5ZJ7PI4OEJBOMHZYBU57D2SNGWJWB5CQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:14.846763777 (377h2m41.773058463s) [seg_pos={0 0}, upload=301954]: Segment reached optimal threshold of 14 pieces.
Jul 9 02:14:14.846894905 (377h2m41.773189582s) [noise=true , address=104.36.82.36:4644 , piece_id=K2G3MAPD7TWYZPKNHQBMEPJRIQDLE75FYVPAZFX4I3GJ4IUAA7XQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1oYHbb73ACitbfkkP5P6cmRyTt6kmaS1y49G4vbsvTvVDwByhT): protocol: expected piece hash; context canceled; EOF
Jul 9 02:14:14.846948435 (377h2m41.773243131s) [noise=true , address=104.36.82.4:4834 , piece_id=YLYIASUPNRTK7KOMRYLN7OQLQBPWC7WFGCPDGUVIOEQJWQQ7A7GA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Z241DCYA5Wig2TdEogxTpY6FzVmFbnKtmHcTwxLxfcZafgooy): protocol: expected piece hash; context canceled; EOF
Jul 9 02:14:14.847094561 (377h2m41.773389249s) [noise=true , address=104.36.82.42:4830 , piece_id=ENYQTUCIF7OWQPCXTYS24OC5QT63ZJJXOY652GLIG7OQPJEQCGGA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:142ocSHHtv8BpSebXFNk5qk1Y23a8pbFAZqE3cq6i3RMX3dPw9): protocol: expected piece hash; context canceled; EOF
Jul 9 02:14:14.847127825 (377h2m41.773422511s) [noise=true , address=104.36.82.41:4768 , piece_id=BGMF33OKUZSZKGIHJYI3RNXEBDE54TX67DYLJ7JCZMRNK6ZAU22Q, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: write tcp 159.223.143.39:43796->104.36.82.41:4768: use of closed network connection; write tcp 159.223.143.39:43796->104.36.82.41:4768: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:43796->104.36.82.41:4768: use of closed network connection
Jul 9 02:14:14.847142999 (377h2m41.773437687s) [seg_pos={0 0}, upload=301954]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jul 9 02:14:14.847204962 (377h2m41.773499652s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:14:14.847209974 (377h2m41.773504671s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:14:14.909581364 (377h2m41.83587606s) [upload=301954]: Done uploading segments...
Jul 9 02:14:14.909605483 (377h2m41.835900172s) [upload=301954]: Upload finished. err: <nil>
Duration: 21.676065546s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
4 out of 60 is failed (threshold=3) [.......X.....X...X....................X.....................] duration:
445.543042ms
FAIL (2025-07-09 02:40:34 UTC)
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
02:25:27.135824454 (377h13m54.062119141s) [noise=true , address=104.36.82.12:4150 , piece_id=DRG5SBA3EE4KVQPEQOJZ6SS537SFWYFUYQXOAQMWYTT6FPVSJIPQ, seg_pos={0 0}, upload=302099]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: context canceled; context canceled
Jul 9 02:25:27.135860079 (377h13m54.062154769s) [seg_pos={0 0}, upload=302099]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 02:25:27.135957283 (377h13m54.062251971s) [upload=302099]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:25:27.135964211 (377h13m54.062258897s) [upload=302099]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:25:27.199410713 (377h13m54.125705389s) [upload=302099]: Done uploading segments...
Jul 9 02:25:27.199427901 (377h13m54.125722577s) [upload=302099]: Upload finished. err: <nil>
Jul 9 02:25:57.289077655 (377h14m24.215372329s) [upload=302109]: Starting upload
Jul 9 02:25:57.289186750 (377h14m24.215481424s) [upload=302109]: Done starting upload
Jul 9 02:25:57.289444851 (377h14m24.215739548s) [upload=302109]: Uploading segments...
Jul 9 02:25:57.289623846 (377h14m24.215918535s) [upload=302109]: Got next segment. Inline: false
Jul 9 02:25:57.289696741 (377h14m24.215991432s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:25:57.289703375 (377h14m24.215998067s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:25:57.318313152 (377h14m24.244607838s) [seg_pos={0 0}, upload=302109]: Begin upload segment...
Jul 9 02:25:57.318363286 (377h14m24.244657964s) [seg_pos={0 0}, upload=302109]: Done begin upload segment.
Jul 9 02:25:57.318386772 (377h14m24.244681463s) [noise=true , address=77.90.31.11:4553 , piece_id=TYW7NBJWACIT7OQS7A4WGXUPR3MB2FKAN2OMGAZJERB4X2DLAO2Q, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318412127 (377h14m24.244706825s) [noise=true , address=77.90.31.10:4518 , piece_id=VJ4HWM2X4VSRSUCQL5QLMBYPFBYV4JYS5ZXCBCB6MTMB76EHZFGQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318822149 (377h14m24.245116841s) [noise=true , address=45.91.116.9:4263 , piece_id=5YILAJ5BZBGOURSZE343ST4RMOTWM6ZOB3EEQQWL5WUNYWOGSAPA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318965862 (377h14m24.24526056s) [noise=true , address=45.91.116.8:4045 , piece_id=KM6ZYTHR6JGTARKZQZYIESPKBEU5H2HQFA7OARIKD2ZXDHPFSMGA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.319187516 (377h14m24.245482203s) [noise=true , address=104.36.82.7:4909 , piece_id=ZRHLWDQ5KIDRWPJ5FXAANQTWERKXCR2PPEGN442IOSYSRFAG7VZA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.319523815 (377h14m24.245818512s) [noise=true , address=45.91.116.14:4191 , piece_id=GTR6T4OTGZTEZ76FWQZ46OCH5WB3O5X27EN4AV33ZJ7D5HX4535A, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.320556632 (377h14m24.24685133s) [noise=true , address=104.36.82.2:4732 , piece_id=DJIGHJMJRAULH2L4YT2WGRXPRQJMNKZG5LICHPSKOCHUL6QDV6BQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.321019141 (377h14m24.247313835s) [noise=true , address=77.90.31.17:4770 , piece_id=V36UBBIYOFJC6KFVVBXEVT4QBUIWN257D3YSDDRAGZVXH7ZXHTBA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.322591797 (377h14m24.248886493s) [noise=true , address=77.90.31.13:4627 , piece_id=HULEKFVZNDNKS4SWWGTJUHYZWWYPDCER5W4GFCVLLDL65PAHULFQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323120451 (377h14m24.249415138s) [noise=true , address=45.91.116.4:4939 , piece_id=ANA3EZU7OZWELNG4QEMTEAYZY5H3C3H4IJFT4XKPUB7WIF44FEVQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323544591 (377h14m24.249839288s) [noise=true , address=45.91.116.7:4011 , piece_id=VDOJRN6ACK4LO44BQTULO6BB3CLAVQI7YIC6XLR32CZV4UJSPE6Q, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323950189 (377h14m24.250244889s) [noise=true , address=104.36.82.10:4078 , piece_id=ZFVUQAEV6DDFLPKCZA2XJ2BZHMMM745LGB4JRZJZQPG72WECGMDQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.326585810 (377h14m24.252880507s) [noise=true , address=45.91.116.5:4297 , piece_id=MSDWDLY5PJHQVI5565EKANJVSTMZFEUD2OTYZ52C65XQUICNMC3A, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.327871214 (377h14m24.254165904s) [noise=true , address=104.36.82.6:4875 , piece_id=C275F2JPVAF52E34CEHT5QXCHUFGIQZ7XGMOAOUOT4RA7YOHRQLQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.328605288 (377h14m24.254899986s) [noise=true , address=104.36.82.5:4838 , piece_id=WK3ALBOEMWWTVEA4FKCS352SZ6YYB35LELURNWXQZK32K4HYDRNA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.335409087 (377h14m24.261703784s) [noise=true , address=45.91.116.6:4973 , piece_id=Z7HQQ2O5QUSJKT4EMIDMEI5PIAEHNVRYX7MALUR2GQULRFIBUOKA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.475344637 (377h14m24.401639326s) [upload=302109]: Next returned nil segment
Jul 9 02:25:57.475376181 (377h14m24.401670879s) [upload=302109]: Waiting for error group managing segments...
Jul 9 02:25:57.477269278 (377h14m24.403563958s) [noise=true , address=45.91.116.7:4011 , piece_id=VDOJRN6ACK4LO44BQTULO6BB3CLAVQI7YIC6XLR32CZV4UJSPE6Q, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.477736179 (377h14m24.40403086s) [noise=true , address=45.91.116.9:4263 , piece_id=5YILAJ5BZBGOURSZE343ST4RMOTWM6ZOB3EEQQWL5WUNYWOGSAPA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.478396652 (377h14m24.404691332s) [noise=true , address=77.90.31.10:4518 , piece_id=VJ4HWM2X4VSRSUCQL5QLMBYPFBYV4JYS5ZXCBCB6MTMB76EHZFGQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.478919538 (377h14m24.405214224s) [noise=true , address=45.91.116.14:4191 , piece_id=GTR6T4OTGZTEZ76FWQZ46OCH5WB3O5X27EN4AV33ZJ7D5HX4535A, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481426716 (377h14m24.407721396s) [noise=true , address=45.91.116.4:4939 , piece_id=ANA3EZU7OZWELNG4QEMTEAYZY5H3C3H4IJFT4XKPUB7WIF44FEVQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481718373 (377h14m24.408013051s) [noise=true , address=45.91.116.5:4297 , piece_id=MSDWDLY5PJHQVI5565EKANJVSTMZFEUD2OTYZ52C65XQUICNMC3A, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481930356 (377h14m24.408225033s) [noise=true , address=77.90.31.17:4770 , piece_id=V36UBBIYOFJC6KFVVBXEVT4QBUIWN257D3YSDDRAGZVXH7ZXHTBA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.482376147 (377h14m24.408670821s) [noise=true , address=45.91.116.8:4045 , piece_id=KM6ZYTHR6JGTARKZQZYIESPKBEU5H2HQFA7OARIKD2ZXDHPFSMGA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.482439192 (377h14m24.408733868s) [noise=true , address=45.91.116.6:4973 , piece_id=Z7HQQ2O5QUSJKT4EMIDMEI5PIAEHNVRYX7MALUR2GQULRFIBUOKA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.483047970 (377h14m24.409342644s) [noise=true , address=77.90.31.13:4627 , piece_id=HULEKFVZNDNKS4SWWGTJUHYZWWYPDCER5W4GFCVLLDL65PAHULFQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.497788632 (377h14m24.424083312s) [noise=true , address=77.90.31.11:4553 , piece_id=TYW7NBJWACIT7OQS7A4WGXUPR3MB2FKAN2OMGAZJERB4X2DLAO2Q, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:16.688921171 (377h14m43.615215851s) [noise=true , address=104.36.82.6:4875 , piece_id=C275F2JPVAF52E34CEHT5QXCHUFGIQZ7XGMOAOUOT4RA7YOHRQLQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:27.177893955 (377h14m54.104188635s) [noise=true , address=104.36.82.2:4732 , piece_id=DJIGHJMJRAULH2L4YT2WGRXPRQJMNKZG5LICHPSKOCHUL6QDV6BQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:27.177985584 (377h14m54.104280281s) [seg_pos={0 0}, upload=302109]: Segment reached optimal threshold of 13 pieces.
Jul 9 02:26:27.178189789 (377h14m54.104484461s) [noise=true , address=104.36.82.5:4838 , piece_id=WK3ALBOEMWWTVEA4FKCS352SZ6YYB35LELURNWXQZK32K4HYDRNA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178252042 (377h14m54.104546733s) [noise=true , address=104.36.82.7:4909 , piece_id=ZRHLWDQ5KIDRWPJ5FXAANQTWERKXCR2PPEGN442IOSYSRFAG7VZA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178273922 (377h14m54.104568608s) [noise=true , address=104.36.82.10:4078 , piece_id=ZFVUQAEV6DDFLPKCZA2XJ2BZHMMM745LGB4JRZJZQPG72WECGMDQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178305983 (377h14m54.104600682s) [seg_pos={0 0}, upload=302109]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 02:26:27.178428768 (377h14m54.104723465s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:26:27.178455810 (377h14m54.104750501s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:26:27.232166184 (377h14m54.158460883s) [upload=302109]: Done uploading segments...
Jul 9 02:26:27.232188444 (377h14m54.158483123s) [upload=302109]: Upload finished. err: <nil>
Duration: 445.572028ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
38 out of 60 is failed (threshold=3) [.X.X.XXXXXXX..XX....XX.XXX.XXX.XX.XXXXXX.XXXXX.XX.....X..XXX] duration:
1m0.001192523s
FAIL (2025-07-09 02:40:10 UTC)
operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1m0.001212895s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [............................X...............................] duration:
28.085422544s
FAIL (2025-07-09 02:39:42 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
3:56.263910643 (377h2m23.19020535s) [upload=301954]: Got next segment. Inline: false
Jul 9 02:13:56.263954358 (377h2m23.190249034s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:13:56.263959891 (377h2m23.190254572s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:13:56.302204249 (377h2m23.228498944s) [seg_pos={0 0}, upload=301954]: Begin upload segment...
Jul 9 02:13:56.302291587 (377h2m23.22858628s) [seg_pos={0 0}, upload=301954]: Done begin upload segment.
Jul 9 02:13:56.302338947 (377h2m23.228633682s) [noise=true , address=77.90.31.14:4689 , piece_id=LW7OT74AJCCC3TEKY6OZI553YK6YQFQ6C7PORQB53E7FUS24KQKQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.302352599 (377h2m23.228647272s) [noise=true , address=104.36.82.8:4986 , piece_id=KPY5JVLSAEB5GUKUJJSIF5M6IIHTJISEYYAXCGUSJ7OLI5QCIBNQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.302450837 (377h2m23.228745531s) [noise=true , address=77.90.31.16:4739 , piece_id=H6UCV6RURPXKC2H3YGAT2OXV2VZMW2DWIRG6TFD6CVXKOVMDP2VQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.302958774 (377h2m23.229253667s) [noise=true , address=77.90.31.13:4640 , piece_id=TPPN75DX3UCBM346CSVJN3GSOA47QV4FVPYSSAHWCJYEM42OCVLQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303069086 (377h2m23.229363782s) [noise=true , address=104.36.82.17:4345 , piece_id=YI27O5WLBYO3TPCX4MVUATFORY6U3EQMDKQHGKRHIZMTG7WGFB2A, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303495994 (377h2m23.22979069s) [noise=true , address=104.36.82.36:4644 , piece_id=K2G3MAPD7TWYZPKNHQBMEPJRIQDLE75FYVPAZFX4I3GJ4IUAA7XQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303575840 (377h2m23.229870536s) [noise=true , address=77.90.31.21:4898 , piece_id=75XXVNT4EV4D5UTX7O5XFL44IZOYY5LZSI2VPFQZPRERL6W5M23A, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303949062 (377h2m23.230245641s) [noise=true , address=79.127.201.215:4339 , piece_id=XDCTYE2SVI6RAPCXTRVANFCZYWJDZXLBHHJ47NQY6V7D36DEFPGA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304060410 (377h2m23.230355106s) [noise=true , address=104.36.82.41:4768 , piece_id=BGMF33OKUZSZKGIHJYI3RNXEBDE54TX67DYLJ7JCZMRNK6ZAU22Q, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304407993 (377h2m23.230702685s) [noise=true , address=77.90.31.10:4525 , piece_id=GPVSIZIBZWTUCP5RFYVPFYIYIL2LWU5PU3UASQVRQLY7SSSFMQBA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304548115 (377h2m23.230842812s) [noise=true , address=104.36.82.23:4464 , piece_id=JCJ4MGEZIIZ7YCTM6QXC5ZJ7PI4OEJBOMHZYBU57D2SNGWJWB5CQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304795164 (377h2m23.231089862s) [noise=true , address=77.90.31.18:4838 , piece_id=TGWYXA3AF7LDG4KPVCRNEYFDADAG6TQ5DAX7JJVW5BLVVVNECILA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305012510 (377h2m23.231307211s) [noise=true , address=104.36.82.34:4554 , piece_id=57K5SDV3YEGLVMSJGSLXNKEWOLETXE6CWBM2NPVCNKLFHPVQCJBQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305113737 (377h2m23.231408427s) [noise=true , address=104.36.82.4:4834 , piece_id=YLYIASUPNRTK7KOMRYLN7OQLQBPWC7WFGCPDGUVIOEQJWQQ7A7GA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305383755 (377h2m23.231678434s) [noise=true , address=104.36.82.38:4698 , piece_id=TSOXJNQEWODCWFEITH4IYKZI7A75UZEXRQPIQO7LMP6LT3WLNXUA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305483822 (377h2m23.231778517s) [noise=true , address=104.36.82.42:4830 , piece_id=ENYQTUCIF7OWQPCXTYS24OC5QT63ZJJXOY652GLIG7OQPJEQCGGA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305954019 (377h2m23.232248718s) [noise=true , address=77.90.31.21:4907 , piece_id=2EUWATHUQNZ77BJV65ZFKR5WL4D3AUURLBQ32XR2IVBA7P4TITOQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.306425517 (377h2m23.232720213s) [noise=true , address=104.36.82.22:4541 , piece_id=XN4HDFDNZ4YUJTDPOTMMDFX4FE2YVZDGCIMHEMJGWXRPNHQ6EU7A, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.375470790 (377h2m23.301765481s) [upload=301954]: Next returned nil segment
Jul 9 02:13:56.375489934 (377h2m23.30178463s) [upload=301954]: Waiting for error group managing segments...
Jul 9 02:13:56.381456859 (377h2m23.307751539s) [noise=true , address=77.90.31.10:4525 , piece_id=GPVSIZIBZWTUCP5RFYVPFYIYIL2LWU5PU3UASQVRQLY7SSSFMQBA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.381956907 (377h2m23.308251586s) [noise=true , address=77.90.31.14:4689 , piece_id=LW7OT74AJCCC3TEKY6OZI553YK6YQFQ6C7PORQB53E7FUS24KQKQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.382837088 (377h2m23.309131761s) [noise=true , address=77.90.31.21:4898 , piece_id=75XXVNT4EV4D5UTX7O5XFL44IZOYY5LZSI2VPFQZPRERL6W5M23A, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.393489125 (377h2m23.319783812s) [noise=true , address=77.90.31.21:4907 , piece_id=2EUWATHUQNZ77BJV65ZFKR5WL4D3AUURLBQ32XR2IVBA7P4TITOQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.404692801 (377h2m23.330987481s) [noise=true , address=77.90.31.16:4739 , piece_id=H6UCV6RURPXKC2H3YGAT2OXV2VZMW2DWIRG6TFD6CVXKOVMDP2VQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.404986115 (377h2m23.331280795s) [noise=true , address=77.90.31.18:4838 , piece_id=TGWYXA3AF7LDG4KPVCRNEYFDADAG6TQ5DAX7JJVW5BLVVVNECILA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.433785086 (377h2m23.360079766s) [noise=true , address=77.90.31.13:4640 , piece_id=TPPN75DX3UCBM346CSVJN3GSOA47QV4FVPYSSAHWCJYEM42OCVLQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.598499747 (377h2m23.524794427s) [noise=true , address=79.127.201.215:4339 , piece_id=XDCTYE2SVI6RAPCXTRVANFCZYWJDZXLBHHJ47NQY6V7D36DEFPGA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:11.107785584 (377h2m38.034080273s) [noise=true , address=104.36.82.22:4541 , piece_id=XN4HDFDNZ4YUJTDPOTMMDFX4FE2YVZDGCIMHEMJGWXRPNHQ6EU7A, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:12.515567029 (377h2m39.441861713s) [noise=true , address=104.36.82.17:4345 , piece_id=YI27O5WLBYO3TPCX4MVUATFORY6U3EQMDKQHGKRHIZMTG7WGFB2A, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:12.923090587 (377h2m39.849385267s) [noise=true , address=104.36.82.8:4986 , piece_id=KPY5JVLSAEB5GUKUJJSIF5M6IIHTJISEYYAXCGUSJ7OLI5QCIBNQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:13.453172659 (377h2m40.379467346s) [noise=true , address=104.36.82.38:4698 , piece_id=TSOXJNQEWODCWFEITH4IYKZI7A75UZEXRQPIQO7LMP6LT3WLNXUA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:14.416524856 (377h2m41.342819534s) [noise=true , address=104.36.82.34:4554 , piece_id=57K5SDV3YEGLVMSJGSLXNKEWOLETXE6CWBM2NPVCNKLFHPVQCJBQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:14.846730699 (377h2m41.773025379s) [noise=true , address=104.36.82.23:4464 , piece_id=JCJ4MGEZIIZ7YCTM6QXC5ZJ7PI4OEJBOMHZYBU57D2SNGWJWB5CQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:14.846763777 (377h2m41.773058463s) [seg_pos={0 0}, upload=301954]: Segment reached optimal threshold of 14 pieces.
Jul 9 02:14:14.846894905 (377h2m41.773189582s) [noise=true , address=104.36.82.36:4644 , piece_id=K2G3MAPD7TWYZPKNHQBMEPJRIQDLE75FYVPAZFX4I3GJ4IUAA7XQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1oYHbb73ACitbfkkP5P6cmRyTt6kmaS1y49G4vbsvTvVDwByhT): protocol: expected piece hash; context canceled; EOF
Jul 9 02:14:14.846948435 (377h2m41.773243131s) [noise=true , address=104.36.82.4:4834 , piece_id=YLYIASUPNRTK7KOMRYLN7OQLQBPWC7WFGCPDGUVIOEQJWQQ7A7GA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Z241DCYA5Wig2TdEogxTpY6FzVmFbnKtmHcTwxLxfcZafgooy): protocol: expected piece hash; context canceled; EOF
Jul 9 02:14:14.847094561 (377h2m41.773389249s) [noise=true , address=104.36.82.42:4830 , piece_id=ENYQTUCIF7OWQPCXTYS24OC5QT63ZJJXOY652GLIG7OQPJEQCGGA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:142ocSHHtv8BpSebXFNk5qk1Y23a8pbFAZqE3cq6i3RMX3dPw9): protocol: expected piece hash; context canceled; EOF
Jul 9 02:14:14.847127825 (377h2m41.773422511s) [noise=true , address=104.36.82.41:4768 , piece_id=BGMF33OKUZSZKGIHJYI3RNXEBDE54TX67DYLJ7JCZMRNK6ZAU22Q, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: write tcp 159.223.143.39:43796->104.36.82.41:4768: use of closed network connection; write tcp 159.223.143.39:43796->104.36.82.41:4768: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:43796->104.36.82.41:4768: use of closed network connection
Jul 9 02:14:14.847142999 (377h2m41.773437687s) [seg_pos={0 0}, upload=301954]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jul 9 02:14:14.847204962 (377h2m41.773499652s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:14:14.847209974 (377h2m41.773504671s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:14:14.909581364 (377h2m41.83587606s) [upload=301954]: Done uploading segments...
Jul 9 02:14:14.909605483 (377h2m41.835900172s) [upload=301954]: Upload finished. err: <nil>
Duration: 28.085455736s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
4 out of 60 is failed (threshold=3) [........X.....X...X....................X....................] duration:
529.528378ms
FAIL (2025-07-09 02:39:40 UTC)
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
02:25:27.135824454 (377h13m54.062119141s) [noise=true , address=104.36.82.12:4150 , piece_id=DRG5SBA3EE4KVQPEQOJZ6SS537SFWYFUYQXOAQMWYTT6FPVSJIPQ, seg_pos={0 0}, upload=302099]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: context canceled; context canceled
Jul 9 02:25:27.135860079 (377h13m54.062154769s) [seg_pos={0 0}, upload=302099]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 02:25:27.135957283 (377h13m54.062251971s) [upload=302099]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:25:27.135964211 (377h13m54.062258897s) [upload=302099]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:25:27.199410713 (377h13m54.125705389s) [upload=302099]: Done uploading segments...
Jul 9 02:25:27.199427901 (377h13m54.125722577s) [upload=302099]: Upload finished. err: <nil>
Jul 9 02:25:57.289077655 (377h14m24.215372329s) [upload=302109]: Starting upload
Jul 9 02:25:57.289186750 (377h14m24.215481424s) [upload=302109]: Done starting upload
Jul 9 02:25:57.289444851 (377h14m24.215739548s) [upload=302109]: Uploading segments...
Jul 9 02:25:57.289623846 (377h14m24.215918535s) [upload=302109]: Got next segment. Inline: false
Jul 9 02:25:57.289696741 (377h14m24.215991432s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:25:57.289703375 (377h14m24.215998067s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:25:57.318313152 (377h14m24.244607838s) [seg_pos={0 0}, upload=302109]: Begin upload segment...
Jul 9 02:25:57.318363286 (377h14m24.244657964s) [seg_pos={0 0}, upload=302109]: Done begin upload segment.
Jul 9 02:25:57.318386772 (377h14m24.244681463s) [noise=true , address=77.90.31.11:4553 , piece_id=TYW7NBJWACIT7OQS7A4WGXUPR3MB2FKAN2OMGAZJERB4X2DLAO2Q, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318412127 (377h14m24.244706825s) [noise=true , address=77.90.31.10:4518 , piece_id=VJ4HWM2X4VSRSUCQL5QLMBYPFBYV4JYS5ZXCBCB6MTMB76EHZFGQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318822149 (377h14m24.245116841s) [noise=true , address=45.91.116.9:4263 , piece_id=5YILAJ5BZBGOURSZE343ST4RMOTWM6ZOB3EEQQWL5WUNYWOGSAPA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318965862 (377h14m24.24526056s) [noise=true , address=45.91.116.8:4045 , piece_id=KM6ZYTHR6JGTARKZQZYIESPKBEU5H2HQFA7OARIKD2ZXDHPFSMGA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.319187516 (377h14m24.245482203s) [noise=true , address=104.36.82.7:4909 , piece_id=ZRHLWDQ5KIDRWPJ5FXAANQTWERKXCR2PPEGN442IOSYSRFAG7VZA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.319523815 (377h14m24.245818512s) [noise=true , address=45.91.116.14:4191 , piece_id=GTR6T4OTGZTEZ76FWQZ46OCH5WB3O5X27EN4AV33ZJ7D5HX4535A, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.320556632 (377h14m24.24685133s) [noise=true , address=104.36.82.2:4732 , piece_id=DJIGHJMJRAULH2L4YT2WGRXPRQJMNKZG5LICHPSKOCHUL6QDV6BQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.321019141 (377h14m24.247313835s) [noise=true , address=77.90.31.17:4770 , piece_id=V36UBBIYOFJC6KFVVBXEVT4QBUIWN257D3YSDDRAGZVXH7ZXHTBA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.322591797 (377h14m24.248886493s) [noise=true , address=77.90.31.13:4627 , piece_id=HULEKFVZNDNKS4SWWGTJUHYZWWYPDCER5W4GFCVLLDL65PAHULFQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323120451 (377h14m24.249415138s) [noise=true , address=45.91.116.4:4939 , piece_id=ANA3EZU7OZWELNG4QEMTEAYZY5H3C3H4IJFT4XKPUB7WIF44FEVQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323544591 (377h14m24.249839288s) [noise=true , address=45.91.116.7:4011 , piece_id=VDOJRN6ACK4LO44BQTULO6BB3CLAVQI7YIC6XLR32CZV4UJSPE6Q, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323950189 (377h14m24.250244889s) [noise=true , address=104.36.82.10:4078 , piece_id=ZFVUQAEV6DDFLPKCZA2XJ2BZHMMM745LGB4JRZJZQPG72WECGMDQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.326585810 (377h14m24.252880507s) [noise=true , address=45.91.116.5:4297 , piece_id=MSDWDLY5PJHQVI5565EKANJVSTMZFEUD2OTYZ52C65XQUICNMC3A, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.327871214 (377h14m24.254165904s) [noise=true , address=104.36.82.6:4875 , piece_id=C275F2JPVAF52E34CEHT5QXCHUFGIQZ7XGMOAOUOT4RA7YOHRQLQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.328605288 (377h14m24.254899986s) [noise=true , address=104.36.82.5:4838 , piece_id=WK3ALBOEMWWTVEA4FKCS352SZ6YYB35LELURNWXQZK32K4HYDRNA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.335409087 (377h14m24.261703784s) [noise=true , address=45.91.116.6:4973 , piece_id=Z7HQQ2O5QUSJKT4EMIDMEI5PIAEHNVRYX7MALUR2GQULRFIBUOKA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.475344637 (377h14m24.401639326s) [upload=302109]: Next returned nil segment
Jul 9 02:25:57.475376181 (377h14m24.401670879s) [upload=302109]: Waiting for error group managing segments...
Jul 9 02:25:57.477269278 (377h14m24.403563958s) [noise=true , address=45.91.116.7:4011 , piece_id=VDOJRN6ACK4LO44BQTULO6BB3CLAVQI7YIC6XLR32CZV4UJSPE6Q, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.477736179 (377h14m24.40403086s) [noise=true , address=45.91.116.9:4263 , piece_id=5YILAJ5BZBGOURSZE343ST4RMOTWM6ZOB3EEQQWL5WUNYWOGSAPA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.478396652 (377h14m24.404691332s) [noise=true , address=77.90.31.10:4518 , piece_id=VJ4HWM2X4VSRSUCQL5QLMBYPFBYV4JYS5ZXCBCB6MTMB76EHZFGQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.478919538 (377h14m24.405214224s) [noise=true , address=45.91.116.14:4191 , piece_id=GTR6T4OTGZTEZ76FWQZ46OCH5WB3O5X27EN4AV33ZJ7D5HX4535A, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481426716 (377h14m24.407721396s) [noise=true , address=45.91.116.4:4939 , piece_id=ANA3EZU7OZWELNG4QEMTEAYZY5H3C3H4IJFT4XKPUB7WIF44FEVQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481718373 (377h14m24.408013051s) [noise=true , address=45.91.116.5:4297 , piece_id=MSDWDLY5PJHQVI5565EKANJVSTMZFEUD2OTYZ52C65XQUICNMC3A, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481930356 (377h14m24.408225033s) [noise=true , address=77.90.31.17:4770 , piece_id=V36UBBIYOFJC6KFVVBXEVT4QBUIWN257D3YSDDRAGZVXH7ZXHTBA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.482376147 (377h14m24.408670821s) [noise=true , address=45.91.116.8:4045 , piece_id=KM6ZYTHR6JGTARKZQZYIESPKBEU5H2HQFA7OARIKD2ZXDHPFSMGA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.482439192 (377h14m24.408733868s) [noise=true , address=45.91.116.6:4973 , piece_id=Z7HQQ2O5QUSJKT4EMIDMEI5PIAEHNVRYX7MALUR2GQULRFIBUOKA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.483047970 (377h14m24.409342644s) [noise=true , address=77.90.31.13:4627 , piece_id=HULEKFVZNDNKS4SWWGTJUHYZWWYPDCER5W4GFCVLLDL65PAHULFQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.497788632 (377h14m24.424083312s) [noise=true , address=77.90.31.11:4553 , piece_id=TYW7NBJWACIT7OQS7A4WGXUPR3MB2FKAN2OMGAZJERB4X2DLAO2Q, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:16.688921171 (377h14m43.615215851s) [noise=true , address=104.36.82.6:4875 , piece_id=C275F2JPVAF52E34CEHT5QXCHUFGIQZ7XGMOAOUOT4RA7YOHRQLQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:27.177893955 (377h14m54.104188635s) [noise=true , address=104.36.82.2:4732 , piece_id=DJIGHJMJRAULH2L4YT2WGRXPRQJMNKZG5LICHPSKOCHUL6QDV6BQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:27.177985584 (377h14m54.104280281s) [seg_pos={0 0}, upload=302109]: Segment reached optimal threshold of 13 pieces.
Jul 9 02:26:27.178189789 (377h14m54.104484461s) [noise=true , address=104.36.82.5:4838 , piece_id=WK3ALBOEMWWTVEA4FKCS352SZ6YYB35LELURNWXQZK32K4HYDRNA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178252042 (377h14m54.104546733s) [noise=true , address=104.36.82.7:4909 , piece_id=ZRHLWDQ5KIDRWPJ5FXAANQTWERKXCR2PPEGN442IOSYSRFAG7VZA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178273922 (377h14m54.104568608s) [noise=true , address=104.36.82.10:4078 , piece_id=ZFVUQAEV6DDFLPKCZA2XJ2BZHMMM745LGB4JRZJZQPG72WECGMDQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178305983 (377h14m54.104600682s) [seg_pos={0 0}, upload=302109]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 02:26:27.178428768 (377h14m54.104723465s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:26:27.178455810 (377h14m54.104750501s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:26:27.232166184 (377h14m54.158460883s) [upload=302109]: Done uploading segments...
Jul 9 02:26:27.232188444 (377h14m54.158483123s) [upload=302109]: Upload finished. err: <nil>
Duration: 529.557307ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
21 out of 60 is failed (threshold=3) [X....X.........X....X.X..XX.XX..X.X.X.....XXX.XX.X...XXX....] duration:
14.576315642s
FAIL (2025-07-09 02:39:32 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 14.576335843s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
38 out of 60 is failed (threshold=3) [X.X.X.XXXXXXX..XX....XX.XXX.XXX.XX.XXXXXX.XXXXX.XX.....X..XX] duration:
1m0.00057032s
FAIL (2025-07-09 02:39:10 UTC)
operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1m0.000609373s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.............................X..............................] duration:
16.175500316s
FAIL (2025-07-09 02:39:04 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
3:56.263910643 (377h2m23.19020535s) [upload=301954]: Got next segment. Inline: false
Jul 9 02:13:56.263954358 (377h2m23.190249034s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:13:56.263959891 (377h2m23.190254572s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:13:56.302204249 (377h2m23.228498944s) [seg_pos={0 0}, upload=301954]: Begin upload segment...
Jul 9 02:13:56.302291587 (377h2m23.22858628s) [seg_pos={0 0}, upload=301954]: Done begin upload segment.
Jul 9 02:13:56.302338947 (377h2m23.228633682s) [noise=true , address=77.90.31.14:4689 , piece_id=LW7OT74AJCCC3TEKY6OZI553YK6YQFQ6C7PORQB53E7FUS24KQKQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.302352599 (377h2m23.228647272s) [noise=true , address=104.36.82.8:4986 , piece_id=KPY5JVLSAEB5GUKUJJSIF5M6IIHTJISEYYAXCGUSJ7OLI5QCIBNQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.302450837 (377h2m23.228745531s) [noise=true , address=77.90.31.16:4739 , piece_id=H6UCV6RURPXKC2H3YGAT2OXV2VZMW2DWIRG6TFD6CVXKOVMDP2VQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.302958774 (377h2m23.229253667s) [noise=true , address=77.90.31.13:4640 , piece_id=TPPN75DX3UCBM346CSVJN3GSOA47QV4FVPYSSAHWCJYEM42OCVLQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303069086 (377h2m23.229363782s) [noise=true , address=104.36.82.17:4345 , piece_id=YI27O5WLBYO3TPCX4MVUATFORY6U3EQMDKQHGKRHIZMTG7WGFB2A, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303495994 (377h2m23.22979069s) [noise=true , address=104.36.82.36:4644 , piece_id=K2G3MAPD7TWYZPKNHQBMEPJRIQDLE75FYVPAZFX4I3GJ4IUAA7XQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303575840 (377h2m23.229870536s) [noise=true , address=77.90.31.21:4898 , piece_id=75XXVNT4EV4D5UTX7O5XFL44IZOYY5LZSI2VPFQZPRERL6W5M23A, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303949062 (377h2m23.230245641s) [noise=true , address=79.127.201.215:4339 , piece_id=XDCTYE2SVI6RAPCXTRVANFCZYWJDZXLBHHJ47NQY6V7D36DEFPGA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304060410 (377h2m23.230355106s) [noise=true , address=104.36.82.41:4768 , piece_id=BGMF33OKUZSZKGIHJYI3RNXEBDE54TX67DYLJ7JCZMRNK6ZAU22Q, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304407993 (377h2m23.230702685s) [noise=true , address=77.90.31.10:4525 , piece_id=GPVSIZIBZWTUCP5RFYVPFYIYIL2LWU5PU3UASQVRQLY7SSSFMQBA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304548115 (377h2m23.230842812s) [noise=true , address=104.36.82.23:4464 , piece_id=JCJ4MGEZIIZ7YCTM6QXC5ZJ7PI4OEJBOMHZYBU57D2SNGWJWB5CQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304795164 (377h2m23.231089862s) [noise=true , address=77.90.31.18:4838 , piece_id=TGWYXA3AF7LDG4KPVCRNEYFDADAG6TQ5DAX7JJVW5BLVVVNECILA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305012510 (377h2m23.231307211s) [noise=true , address=104.36.82.34:4554 , piece_id=57K5SDV3YEGLVMSJGSLXNKEWOLETXE6CWBM2NPVCNKLFHPVQCJBQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305113737 (377h2m23.231408427s) [noise=true , address=104.36.82.4:4834 , piece_id=YLYIASUPNRTK7KOMRYLN7OQLQBPWC7WFGCPDGUVIOEQJWQQ7A7GA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305383755 (377h2m23.231678434s) [noise=true , address=104.36.82.38:4698 , piece_id=TSOXJNQEWODCWFEITH4IYKZI7A75UZEXRQPIQO7LMP6LT3WLNXUA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305483822 (377h2m23.231778517s) [noise=true , address=104.36.82.42:4830 , piece_id=ENYQTUCIF7OWQPCXTYS24OC5QT63ZJJXOY652GLIG7OQPJEQCGGA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305954019 (377h2m23.232248718s) [noise=true , address=77.90.31.21:4907 , piece_id=2EUWATHUQNZ77BJV65ZFKR5WL4D3AUURLBQ32XR2IVBA7P4TITOQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.306425517 (377h2m23.232720213s) [noise=true , address=104.36.82.22:4541 , piece_id=XN4HDFDNZ4YUJTDPOTMMDFX4FE2YVZDGCIMHEMJGWXRPNHQ6EU7A, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.375470790 (377h2m23.301765481s) [upload=301954]: Next returned nil segment
Jul 9 02:13:56.375489934 (377h2m23.30178463s) [upload=301954]: Waiting for error group managing segments...
Jul 9 02:13:56.381456859 (377h2m23.307751539s) [noise=true , address=77.90.31.10:4525 , piece_id=GPVSIZIBZWTUCP5RFYVPFYIYIL2LWU5PU3UASQVRQLY7SSSFMQBA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.381956907 (377h2m23.308251586s) [noise=true , address=77.90.31.14:4689 , piece_id=LW7OT74AJCCC3TEKY6OZI553YK6YQFQ6C7PORQB53E7FUS24KQKQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.382837088 (377h2m23.309131761s) [noise=true , address=77.90.31.21:4898 , piece_id=75XXVNT4EV4D5UTX7O5XFL44IZOYY5LZSI2VPFQZPRERL6W5M23A, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.393489125 (377h2m23.319783812s) [noise=true , address=77.90.31.21:4907 , piece_id=2EUWATHUQNZ77BJV65ZFKR5WL4D3AUURLBQ32XR2IVBA7P4TITOQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.404692801 (377h2m23.330987481s) [noise=true , address=77.90.31.16:4739 , piece_id=H6UCV6RURPXKC2H3YGAT2OXV2VZMW2DWIRG6TFD6CVXKOVMDP2VQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.404986115 (377h2m23.331280795s) [noise=true , address=77.90.31.18:4838 , piece_id=TGWYXA3AF7LDG4KPVCRNEYFDADAG6TQ5DAX7JJVW5BLVVVNECILA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.433785086 (377h2m23.360079766s) [noise=true , address=77.90.31.13:4640 , piece_id=TPPN75DX3UCBM346CSVJN3GSOA47QV4FVPYSSAHWCJYEM42OCVLQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.598499747 (377h2m23.524794427s) [noise=true , address=79.127.201.215:4339 , piece_id=XDCTYE2SVI6RAPCXTRVANFCZYWJDZXLBHHJ47NQY6V7D36DEFPGA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:11.107785584 (377h2m38.034080273s) [noise=true , address=104.36.82.22:4541 , piece_id=XN4HDFDNZ4YUJTDPOTMMDFX4FE2YVZDGCIMHEMJGWXRPNHQ6EU7A, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:12.515567029 (377h2m39.441861713s) [noise=true , address=104.36.82.17:4345 , piece_id=YI27O5WLBYO3TPCX4MVUATFORY6U3EQMDKQHGKRHIZMTG7WGFB2A, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:12.923090587 (377h2m39.849385267s) [noise=true , address=104.36.82.8:4986 , piece_id=KPY5JVLSAEB5GUKUJJSIF5M6IIHTJISEYYAXCGUSJ7OLI5QCIBNQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:13.453172659 (377h2m40.379467346s) [noise=true , address=104.36.82.38:4698 , piece_id=TSOXJNQEWODCWFEITH4IYKZI7A75UZEXRQPIQO7LMP6LT3WLNXUA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:14.416524856 (377h2m41.342819534s) [noise=true , address=104.36.82.34:4554 , piece_id=57K5SDV3YEGLVMSJGSLXNKEWOLETXE6CWBM2NPVCNKLFHPVQCJBQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:14.846730699 (377h2m41.773025379s) [noise=true , address=104.36.82.23:4464 , piece_id=JCJ4MGEZIIZ7YCTM6QXC5ZJ7PI4OEJBOMHZYBU57D2SNGWJWB5CQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:14.846763777 (377h2m41.773058463s) [seg_pos={0 0}, upload=301954]: Segment reached optimal threshold of 14 pieces.
Jul 9 02:14:14.846894905 (377h2m41.773189582s) [noise=true , address=104.36.82.36:4644 , piece_id=K2G3MAPD7TWYZPKNHQBMEPJRIQDLE75FYVPAZFX4I3GJ4IUAA7XQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1oYHbb73ACitbfkkP5P6cmRyTt6kmaS1y49G4vbsvTvVDwByhT): protocol: expected piece hash; context canceled; EOF
Jul 9 02:14:14.846948435 (377h2m41.773243131s) [noise=true , address=104.36.82.4:4834 , piece_id=YLYIASUPNRTK7KOMRYLN7OQLQBPWC7WFGCPDGUVIOEQJWQQ7A7GA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Z241DCYA5Wig2TdEogxTpY6FzVmFbnKtmHcTwxLxfcZafgooy): protocol: expected piece hash; context canceled; EOF
Jul 9 02:14:14.847094561 (377h2m41.773389249s) [noise=true , address=104.36.82.42:4830 , piece_id=ENYQTUCIF7OWQPCXTYS24OC5QT63ZJJXOY652GLIG7OQPJEQCGGA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:142ocSHHtv8BpSebXFNk5qk1Y23a8pbFAZqE3cq6i3RMX3dPw9): protocol: expected piece hash; context canceled; EOF
Jul 9 02:14:14.847127825 (377h2m41.773422511s) [noise=true , address=104.36.82.41:4768 , piece_id=BGMF33OKUZSZKGIHJYI3RNXEBDE54TX67DYLJ7JCZMRNK6ZAU22Q, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: write tcp 159.223.143.39:43796->104.36.82.41:4768: use of closed network connection; write tcp 159.223.143.39:43796->104.36.82.41:4768: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:43796->104.36.82.41:4768: use of closed network connection
Jul 9 02:14:14.847142999 (377h2m41.773437687s) [seg_pos={0 0}, upload=301954]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jul 9 02:14:14.847204962 (377h2m41.773499652s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:14:14.847209974 (377h2m41.773504671s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:14:14.909581364 (377h2m41.83587606s) [upload=301954]: Done uploading segments...
Jul 9 02:14:14.909605483 (377h2m41.835900172s) [upload=301954]: Upload finished. err: <nil>
Duration: 16.175544439s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
21 out of 60 is failed (threshold=3) [.X....X.........X....X.X..XX.XX..X.X.X.....XXX.XX.X...XXX...] duration:
13.388267528s
FAIL (2025-07-09 02:39:04 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 13.388291244s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [..............................X.............................] duration:
27.166481738s
FAIL (2025-07-09 02:38:40 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
3:56.263910643 (377h2m23.19020535s) [upload=301954]: Got next segment. Inline: false
Jul 9 02:13:56.263954358 (377h2m23.190249034s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:13:56.263959891 (377h2m23.190254572s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:13:56.302204249 (377h2m23.228498944s) [seg_pos={0 0}, upload=301954]: Begin upload segment...
Jul 9 02:13:56.302291587 (377h2m23.22858628s) [seg_pos={0 0}, upload=301954]: Done begin upload segment.
Jul 9 02:13:56.302338947 (377h2m23.228633682s) [noise=true , address=77.90.31.14:4689 , piece_id=LW7OT74AJCCC3TEKY6OZI553YK6YQFQ6C7PORQB53E7FUS24KQKQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.302352599 (377h2m23.228647272s) [noise=true , address=104.36.82.8:4986 , piece_id=KPY5JVLSAEB5GUKUJJSIF5M6IIHTJISEYYAXCGUSJ7OLI5QCIBNQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.302450837 (377h2m23.228745531s) [noise=true , address=77.90.31.16:4739 , piece_id=H6UCV6RURPXKC2H3YGAT2OXV2VZMW2DWIRG6TFD6CVXKOVMDP2VQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.302958774 (377h2m23.229253667s) [noise=true , address=77.90.31.13:4640 , piece_id=TPPN75DX3UCBM346CSVJN3GSOA47QV4FVPYSSAHWCJYEM42OCVLQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303069086 (377h2m23.229363782s) [noise=true , address=104.36.82.17:4345 , piece_id=YI27O5WLBYO3TPCX4MVUATFORY6U3EQMDKQHGKRHIZMTG7WGFB2A, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303495994 (377h2m23.22979069s) [noise=true , address=104.36.82.36:4644 , piece_id=K2G3MAPD7TWYZPKNHQBMEPJRIQDLE75FYVPAZFX4I3GJ4IUAA7XQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303575840 (377h2m23.229870536s) [noise=true , address=77.90.31.21:4898 , piece_id=75XXVNT4EV4D5UTX7O5XFL44IZOYY5LZSI2VPFQZPRERL6W5M23A, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303949062 (377h2m23.230245641s) [noise=true , address=79.127.201.215:4339 , piece_id=XDCTYE2SVI6RAPCXTRVANFCZYWJDZXLBHHJ47NQY6V7D36DEFPGA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304060410 (377h2m23.230355106s) [noise=true , address=104.36.82.41:4768 , piece_id=BGMF33OKUZSZKGIHJYI3RNXEBDE54TX67DYLJ7JCZMRNK6ZAU22Q, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304407993 (377h2m23.230702685s) [noise=true , address=77.90.31.10:4525 , piece_id=GPVSIZIBZWTUCP5RFYVPFYIYIL2LWU5PU3UASQVRQLY7SSSFMQBA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304548115 (377h2m23.230842812s) [noise=true , address=104.36.82.23:4464 , piece_id=JCJ4MGEZIIZ7YCTM6QXC5ZJ7PI4OEJBOMHZYBU57D2SNGWJWB5CQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304795164 (377h2m23.231089862s) [noise=true , address=77.90.31.18:4838 , piece_id=TGWYXA3AF7LDG4KPVCRNEYFDADAG6TQ5DAX7JJVW5BLVVVNECILA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305012510 (377h2m23.231307211s) [noise=true , address=104.36.82.34:4554 , piece_id=57K5SDV3YEGLVMSJGSLXNKEWOLETXE6CWBM2NPVCNKLFHPVQCJBQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305113737 (377h2m23.231408427s) [noise=true , address=104.36.82.4:4834 , piece_id=YLYIASUPNRTK7KOMRYLN7OQLQBPWC7WFGCPDGUVIOEQJWQQ7A7GA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305383755 (377h2m23.231678434s) [noise=true , address=104.36.82.38:4698 , piece_id=TSOXJNQEWODCWFEITH4IYKZI7A75UZEXRQPIQO7LMP6LT3WLNXUA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305483822 (377h2m23.231778517s) [noise=true , address=104.36.82.42:4830 , piece_id=ENYQTUCIF7OWQPCXTYS24OC5QT63ZJJXOY652GLIG7OQPJEQCGGA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305954019 (377h2m23.232248718s) [noise=true , address=77.90.31.21:4907 , piece_id=2EUWATHUQNZ77BJV65ZFKR5WL4D3AUURLBQ32XR2IVBA7P4TITOQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.306425517 (377h2m23.232720213s) [noise=true , address=104.36.82.22:4541 , piece_id=XN4HDFDNZ4YUJTDPOTMMDFX4FE2YVZDGCIMHEMJGWXRPNHQ6EU7A, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.375470790 (377h2m23.301765481s) [upload=301954]: Next returned nil segment
Jul 9 02:13:56.375489934 (377h2m23.30178463s) [upload=301954]: Waiting for error group managing segments...
Jul 9 02:13:56.381456859 (377h2m23.307751539s) [noise=true , address=77.90.31.10:4525 , piece_id=GPVSIZIBZWTUCP5RFYVPFYIYIL2LWU5PU3UASQVRQLY7SSSFMQBA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.381956907 (377h2m23.308251586s) [noise=true , address=77.90.31.14:4689 , piece_id=LW7OT74AJCCC3TEKY6OZI553YK6YQFQ6C7PORQB53E7FUS24KQKQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.382837088 (377h2m23.309131761s) [noise=true , address=77.90.31.21:4898 , piece_id=75XXVNT4EV4D5UTX7O5XFL44IZOYY5LZSI2VPFQZPRERL6W5M23A, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.393489125 (377h2m23.319783812s) [noise=true , address=77.90.31.21:4907 , piece_id=2EUWATHUQNZ77BJV65ZFKR5WL4D3AUURLBQ32XR2IVBA7P4TITOQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.404692801 (377h2m23.330987481s) [noise=true , address=77.90.31.16:4739 , piece_id=H6UCV6RURPXKC2H3YGAT2OXV2VZMW2DWIRG6TFD6CVXKOVMDP2VQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.404986115 (377h2m23.331280795s) [noise=true , address=77.90.31.18:4838 , piece_id=TGWYXA3AF7LDG4KPVCRNEYFDADAG6TQ5DAX7JJVW5BLVVVNECILA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.433785086 (377h2m23.360079766s) [noise=true , address=77.90.31.13:4640 , piece_id=TPPN75DX3UCBM346CSVJN3GSOA47QV4FVPYSSAHWCJYEM42OCVLQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.598499747 (377h2m23.524794427s) [noise=true , address=79.127.201.215:4339 , piece_id=XDCTYE2SVI6RAPCXTRVANFCZYWJDZXLBHHJ47NQY6V7D36DEFPGA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:11.107785584 (377h2m38.034080273s) [noise=true , address=104.36.82.22:4541 , piece_id=XN4HDFDNZ4YUJTDPOTMMDFX4FE2YVZDGCIMHEMJGWXRPNHQ6EU7A, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:12.515567029 (377h2m39.441861713s) [noise=true , address=104.36.82.17:4345 , piece_id=YI27O5WLBYO3TPCX4MVUATFORY6U3EQMDKQHGKRHIZMTG7WGFB2A, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:12.923090587 (377h2m39.849385267s) [noise=true , address=104.36.82.8:4986 , piece_id=KPY5JVLSAEB5GUKUJJSIF5M6IIHTJISEYYAXCGUSJ7OLI5QCIBNQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:13.453172659 (377h2m40.379467346s) [noise=true , address=104.36.82.38:4698 , piece_id=TSOXJNQEWODCWFEITH4IYKZI7A75UZEXRQPIQO7LMP6LT3WLNXUA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:14.416524856 (377h2m41.342819534s) [noise=true , address=104.36.82.34:4554 , piece_id=57K5SDV3YEGLVMSJGSLXNKEWOLETXE6CWBM2NPVCNKLFHPVQCJBQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:14.846730699 (377h2m41.773025379s) [noise=true , address=104.36.82.23:4464 , piece_id=JCJ4MGEZIIZ7YCTM6QXC5ZJ7PI4OEJBOMHZYBU57D2SNGWJWB5CQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:14.846763777 (377h2m41.773058463s) [seg_pos={0 0}, upload=301954]: Segment reached optimal threshold of 14 pieces.
Jul 9 02:14:14.846894905 (377h2m41.773189582s) [noise=true , address=104.36.82.36:4644 , piece_id=K2G3MAPD7TWYZPKNHQBMEPJRIQDLE75FYVPAZFX4I3GJ4IUAA7XQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1oYHbb73ACitbfkkP5P6cmRyTt6kmaS1y49G4vbsvTvVDwByhT): protocol: expected piece hash; context canceled; EOF
Jul 9 02:14:14.846948435 (377h2m41.773243131s) [noise=true , address=104.36.82.4:4834 , piece_id=YLYIASUPNRTK7KOMRYLN7OQLQBPWC7WFGCPDGUVIOEQJWQQ7A7GA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Z241DCYA5Wig2TdEogxTpY6FzVmFbnKtmHcTwxLxfcZafgooy): protocol: expected piece hash; context canceled; EOF
Jul 9 02:14:14.847094561 (377h2m41.773389249s) [noise=true , address=104.36.82.42:4830 , piece_id=ENYQTUCIF7OWQPCXTYS24OC5QT63ZJJXOY652GLIG7OQPJEQCGGA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:142ocSHHtv8BpSebXFNk5qk1Y23a8pbFAZqE3cq6i3RMX3dPw9): protocol: expected piece hash; context canceled; EOF
Jul 9 02:14:14.847127825 (377h2m41.773422511s) [noise=true , address=104.36.82.41:4768 , piece_id=BGMF33OKUZSZKGIHJYI3RNXEBDE54TX67DYLJ7JCZMRNK6ZAU22Q, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: write tcp 159.223.143.39:43796->104.36.82.41:4768: use of closed network connection; write tcp 159.223.143.39:43796->104.36.82.41:4768: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:43796->104.36.82.41:4768: use of closed network connection
Jul 9 02:14:14.847142999 (377h2m41.773437687s) [seg_pos={0 0}, upload=301954]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jul 9 02:14:14.847204962 (377h2m41.773499652s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:14:14.847209974 (377h2m41.773504671s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:14:14.909581364 (377h2m41.83587606s) [upload=301954]: Done uploading segments...
Jul 9 02:14:14.909605483 (377h2m41.835900172s) [upload=301954]: Upload finished. err: <nil>
Duration: 27.166504624s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
4 out of 60 is failed (threshold=3) [.........X.....X...X....................X...................] duration:
438.896146ms
FAIL (2025-07-09 02:38:28 UTC)
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
02:25:27.135824454 (377h13m54.062119141s) [noise=true , address=104.36.82.12:4150 , piece_id=DRG5SBA3EE4KVQPEQOJZ6SS537SFWYFUYQXOAQMWYTT6FPVSJIPQ, seg_pos={0 0}, upload=302099]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: context canceled; context canceled
Jul 9 02:25:27.135860079 (377h13m54.062154769s) [seg_pos={0 0}, upload=302099]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 02:25:27.135957283 (377h13m54.062251971s) [upload=302099]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:25:27.135964211 (377h13m54.062258897s) [upload=302099]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:25:27.199410713 (377h13m54.125705389s) [upload=302099]: Done uploading segments...
Jul 9 02:25:27.199427901 (377h13m54.125722577s) [upload=302099]: Upload finished. err: <nil>
Jul 9 02:25:57.289077655 (377h14m24.215372329s) [upload=302109]: Starting upload
Jul 9 02:25:57.289186750 (377h14m24.215481424s) [upload=302109]: Done starting upload
Jul 9 02:25:57.289444851 (377h14m24.215739548s) [upload=302109]: Uploading segments...
Jul 9 02:25:57.289623846 (377h14m24.215918535s) [upload=302109]: Got next segment. Inline: false
Jul 9 02:25:57.289696741 (377h14m24.215991432s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:25:57.289703375 (377h14m24.215998067s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:25:57.318313152 (377h14m24.244607838s) [seg_pos={0 0}, upload=302109]: Begin upload segment...
Jul 9 02:25:57.318363286 (377h14m24.244657964s) [seg_pos={0 0}, upload=302109]: Done begin upload segment.
Jul 9 02:25:57.318386772 (377h14m24.244681463s) [noise=true , address=77.90.31.11:4553 , piece_id=TYW7NBJWACIT7OQS7A4WGXUPR3MB2FKAN2OMGAZJERB4X2DLAO2Q, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318412127 (377h14m24.244706825s) [noise=true , address=77.90.31.10:4518 , piece_id=VJ4HWM2X4VSRSUCQL5QLMBYPFBYV4JYS5ZXCBCB6MTMB76EHZFGQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318822149 (377h14m24.245116841s) [noise=true , address=45.91.116.9:4263 , piece_id=5YILAJ5BZBGOURSZE343ST4RMOTWM6ZOB3EEQQWL5WUNYWOGSAPA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318965862 (377h14m24.24526056s) [noise=true , address=45.91.116.8:4045 , piece_id=KM6ZYTHR6JGTARKZQZYIESPKBEU5H2HQFA7OARIKD2ZXDHPFSMGA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.319187516 (377h14m24.245482203s) [noise=true , address=104.36.82.7:4909 , piece_id=ZRHLWDQ5KIDRWPJ5FXAANQTWERKXCR2PPEGN442IOSYSRFAG7VZA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.319523815 (377h14m24.245818512s) [noise=true , address=45.91.116.14:4191 , piece_id=GTR6T4OTGZTEZ76FWQZ46OCH5WB3O5X27EN4AV33ZJ7D5HX4535A, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.320556632 (377h14m24.24685133s) [noise=true , address=104.36.82.2:4732 , piece_id=DJIGHJMJRAULH2L4YT2WGRXPRQJMNKZG5LICHPSKOCHUL6QDV6BQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.321019141 (377h14m24.247313835s) [noise=true , address=77.90.31.17:4770 , piece_id=V36UBBIYOFJC6KFVVBXEVT4QBUIWN257D3YSDDRAGZVXH7ZXHTBA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.322591797 (377h14m24.248886493s) [noise=true , address=77.90.31.13:4627 , piece_id=HULEKFVZNDNKS4SWWGTJUHYZWWYPDCER5W4GFCVLLDL65PAHULFQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323120451 (377h14m24.249415138s) [noise=true , address=45.91.116.4:4939 , piece_id=ANA3EZU7OZWELNG4QEMTEAYZY5H3C3H4IJFT4XKPUB7WIF44FEVQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323544591 (377h14m24.249839288s) [noise=true , address=45.91.116.7:4011 , piece_id=VDOJRN6ACK4LO44BQTULO6BB3CLAVQI7YIC6XLR32CZV4UJSPE6Q, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323950189 (377h14m24.250244889s) [noise=true , address=104.36.82.10:4078 , piece_id=ZFVUQAEV6DDFLPKCZA2XJ2BZHMMM745LGB4JRZJZQPG72WECGMDQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.326585810 (377h14m24.252880507s) [noise=true , address=45.91.116.5:4297 , piece_id=MSDWDLY5PJHQVI5565EKANJVSTMZFEUD2OTYZ52C65XQUICNMC3A, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.327871214 (377h14m24.254165904s) [noise=true , address=104.36.82.6:4875 , piece_id=C275F2JPVAF52E34CEHT5QXCHUFGIQZ7XGMOAOUOT4RA7YOHRQLQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.328605288 (377h14m24.254899986s) [noise=true , address=104.36.82.5:4838 , piece_id=WK3ALBOEMWWTVEA4FKCS352SZ6YYB35LELURNWXQZK32K4HYDRNA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.335409087 (377h14m24.261703784s) [noise=true , address=45.91.116.6:4973 , piece_id=Z7HQQ2O5QUSJKT4EMIDMEI5PIAEHNVRYX7MALUR2GQULRFIBUOKA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.475344637 (377h14m24.401639326s) [upload=302109]: Next returned nil segment
Jul 9 02:25:57.475376181 (377h14m24.401670879s) [upload=302109]: Waiting for error group managing segments...
Jul 9 02:25:57.477269278 (377h14m24.403563958s) [noise=true , address=45.91.116.7:4011 , piece_id=VDOJRN6ACK4LO44BQTULO6BB3CLAVQI7YIC6XLR32CZV4UJSPE6Q, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.477736179 (377h14m24.40403086s) [noise=true , address=45.91.116.9:4263 , piece_id=5YILAJ5BZBGOURSZE343ST4RMOTWM6ZOB3EEQQWL5WUNYWOGSAPA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.478396652 (377h14m24.404691332s) [noise=true , address=77.90.31.10:4518 , piece_id=VJ4HWM2X4VSRSUCQL5QLMBYPFBYV4JYS5ZXCBCB6MTMB76EHZFGQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.478919538 (377h14m24.405214224s) [noise=true , address=45.91.116.14:4191 , piece_id=GTR6T4OTGZTEZ76FWQZ46OCH5WB3O5X27EN4AV33ZJ7D5HX4535A, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481426716 (377h14m24.407721396s) [noise=true , address=45.91.116.4:4939 , piece_id=ANA3EZU7OZWELNG4QEMTEAYZY5H3C3H4IJFT4XKPUB7WIF44FEVQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481718373 (377h14m24.408013051s) [noise=true , address=45.91.116.5:4297 , piece_id=MSDWDLY5PJHQVI5565EKANJVSTMZFEUD2OTYZ52C65XQUICNMC3A, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481930356 (377h14m24.408225033s) [noise=true , address=77.90.31.17:4770 , piece_id=V36UBBIYOFJC6KFVVBXEVT4QBUIWN257D3YSDDRAGZVXH7ZXHTBA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.482376147 (377h14m24.408670821s) [noise=true , address=45.91.116.8:4045 , piece_id=KM6ZYTHR6JGTARKZQZYIESPKBEU5H2HQFA7OARIKD2ZXDHPFSMGA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.482439192 (377h14m24.408733868s) [noise=true , address=45.91.116.6:4973 , piece_id=Z7HQQ2O5QUSJKT4EMIDMEI5PIAEHNVRYX7MALUR2GQULRFIBUOKA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.483047970 (377h14m24.409342644s) [noise=true , address=77.90.31.13:4627 , piece_id=HULEKFVZNDNKS4SWWGTJUHYZWWYPDCER5W4GFCVLLDL65PAHULFQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.497788632 (377h14m24.424083312s) [noise=true , address=77.90.31.11:4553 , piece_id=TYW7NBJWACIT7OQS7A4WGXUPR3MB2FKAN2OMGAZJERB4X2DLAO2Q, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:16.688921171 (377h14m43.615215851s) [noise=true , address=104.36.82.6:4875 , piece_id=C275F2JPVAF52E34CEHT5QXCHUFGIQZ7XGMOAOUOT4RA7YOHRQLQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:27.177893955 (377h14m54.104188635s) [noise=true , address=104.36.82.2:4732 , piece_id=DJIGHJMJRAULH2L4YT2WGRXPRQJMNKZG5LICHPSKOCHUL6QDV6BQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:27.177985584 (377h14m54.104280281s) [seg_pos={0 0}, upload=302109]: Segment reached optimal threshold of 13 pieces.
Jul 9 02:26:27.178189789 (377h14m54.104484461s) [noise=true , address=104.36.82.5:4838 , piece_id=WK3ALBOEMWWTVEA4FKCS352SZ6YYB35LELURNWXQZK32K4HYDRNA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178252042 (377h14m54.104546733s) [noise=true , address=104.36.82.7:4909 , piece_id=ZRHLWDQ5KIDRWPJ5FXAANQTWERKXCR2PPEGN442IOSYSRFAG7VZA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178273922 (377h14m54.104568608s) [noise=true , address=104.36.82.10:4078 , piece_id=ZFVUQAEV6DDFLPKCZA2XJ2BZHMMM745LGB4JRZJZQPG72WECGMDQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178305983 (377h14m54.104600682s) [seg_pos={0 0}, upload=302109]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 02:26:27.178428768 (377h14m54.104723465s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:26:27.178455810 (377h14m54.104750501s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:26:27.232166184 (377h14m54.158460883s) [upload=302109]: Done uploading segments...
Jul 9 02:26:27.232188444 (377h14m54.158483123s) [upload=302109]: Upload finished. err: <nil>
Duration: 438.997493ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [...............................X............................] duration:
20.376392513s
FAIL (2025-07-09 02:38:13 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
3:56.263910643 (377h2m23.19020535s) [upload=301954]: Got next segment. Inline: false
Jul 9 02:13:56.263954358 (377h2m23.190249034s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:13:56.263959891 (377h2m23.190254572s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:13:56.302204249 (377h2m23.228498944s) [seg_pos={0 0}, upload=301954]: Begin upload segment...
Jul 9 02:13:56.302291587 (377h2m23.22858628s) [seg_pos={0 0}, upload=301954]: Done begin upload segment.
Jul 9 02:13:56.302338947 (377h2m23.228633682s) [noise=true , address=77.90.31.14:4689 , piece_id=LW7OT74AJCCC3TEKY6OZI553YK6YQFQ6C7PORQB53E7FUS24KQKQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.302352599 (377h2m23.228647272s) [noise=true , address=104.36.82.8:4986 , piece_id=KPY5JVLSAEB5GUKUJJSIF5M6IIHTJISEYYAXCGUSJ7OLI5QCIBNQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.302450837 (377h2m23.228745531s) [noise=true , address=77.90.31.16:4739 , piece_id=H6UCV6RURPXKC2H3YGAT2OXV2VZMW2DWIRG6TFD6CVXKOVMDP2VQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.302958774 (377h2m23.229253667s) [noise=true , address=77.90.31.13:4640 , piece_id=TPPN75DX3UCBM346CSVJN3GSOA47QV4FVPYSSAHWCJYEM42OCVLQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303069086 (377h2m23.229363782s) [noise=true , address=104.36.82.17:4345 , piece_id=YI27O5WLBYO3TPCX4MVUATFORY6U3EQMDKQHGKRHIZMTG7WGFB2A, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303495994 (377h2m23.22979069s) [noise=true , address=104.36.82.36:4644 , piece_id=K2G3MAPD7TWYZPKNHQBMEPJRIQDLE75FYVPAZFX4I3GJ4IUAA7XQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303575840 (377h2m23.229870536s) [noise=true , address=77.90.31.21:4898 , piece_id=75XXVNT4EV4D5UTX7O5XFL44IZOYY5LZSI2VPFQZPRERL6W5M23A, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303949062 (377h2m23.230245641s) [noise=true , address=79.127.201.215:4339 , piece_id=XDCTYE2SVI6RAPCXTRVANFCZYWJDZXLBHHJ47NQY6V7D36DEFPGA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304060410 (377h2m23.230355106s) [noise=true , address=104.36.82.41:4768 , piece_id=BGMF33OKUZSZKGIHJYI3RNXEBDE54TX67DYLJ7JCZMRNK6ZAU22Q, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304407993 (377h2m23.230702685s) [noise=true , address=77.90.31.10:4525 , piece_id=GPVSIZIBZWTUCP5RFYVPFYIYIL2LWU5PU3UASQVRQLY7SSSFMQBA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304548115 (377h2m23.230842812s) [noise=true , address=104.36.82.23:4464 , piece_id=JCJ4MGEZIIZ7YCTM6QXC5ZJ7PI4OEJBOMHZYBU57D2SNGWJWB5CQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304795164 (377h2m23.231089862s) [noise=true , address=77.90.31.18:4838 , piece_id=TGWYXA3AF7LDG4KPVCRNEYFDADAG6TQ5DAX7JJVW5BLVVVNECILA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305012510 (377h2m23.231307211s) [noise=true , address=104.36.82.34:4554 , piece_id=57K5SDV3YEGLVMSJGSLXNKEWOLETXE6CWBM2NPVCNKLFHPVQCJBQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305113737 (377h2m23.231408427s) [noise=true , address=104.36.82.4:4834 , piece_id=YLYIASUPNRTK7KOMRYLN7OQLQBPWC7WFGCPDGUVIOEQJWQQ7A7GA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305383755 (377h2m23.231678434s) [noise=true , address=104.36.82.38:4698 , piece_id=TSOXJNQEWODCWFEITH4IYKZI7A75UZEXRQPIQO7LMP6LT3WLNXUA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305483822 (377h2m23.231778517s) [noise=true , address=104.36.82.42:4830 , piece_id=ENYQTUCIF7OWQPCXTYS24OC5QT63ZJJXOY652GLIG7OQPJEQCGGA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305954019 (377h2m23.232248718s) [noise=true , address=77.90.31.21:4907 , piece_id=2EUWATHUQNZ77BJV65ZFKR5WL4D3AUURLBQ32XR2IVBA7P4TITOQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.306425517 (377h2m23.232720213s) [noise=true , address=104.36.82.22:4541 , piece_id=XN4HDFDNZ4YUJTDPOTMMDFX4FE2YVZDGCIMHEMJGWXRPNHQ6EU7A, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.375470790 (377h2m23.301765481s) [upload=301954]: Next returned nil segment
Jul 9 02:13:56.375489934 (377h2m23.30178463s) [upload=301954]: Waiting for error group managing segments...
Jul 9 02:13:56.381456859 (377h2m23.307751539s) [noise=true , address=77.90.31.10:4525 , piece_id=GPVSIZIBZWTUCP5RFYVPFYIYIL2LWU5PU3UASQVRQLY7SSSFMQBA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.381956907 (377h2m23.308251586s) [noise=true , address=77.90.31.14:4689 , piece_id=LW7OT74AJCCC3TEKY6OZI553YK6YQFQ6C7PORQB53E7FUS24KQKQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.382837088 (377h2m23.309131761s) [noise=true , address=77.90.31.21:4898 , piece_id=75XXVNT4EV4D5UTX7O5XFL44IZOYY5LZSI2VPFQZPRERL6W5M23A, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.393489125 (377h2m23.319783812s) [noise=true , address=77.90.31.21:4907 , piece_id=2EUWATHUQNZ77BJV65ZFKR5WL4D3AUURLBQ32XR2IVBA7P4TITOQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.404692801 (377h2m23.330987481s) [noise=true , address=77.90.31.16:4739 , piece_id=H6UCV6RURPXKC2H3YGAT2OXV2VZMW2DWIRG6TFD6CVXKOVMDP2VQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.404986115 (377h2m23.331280795s) [noise=true , address=77.90.31.18:4838 , piece_id=TGWYXA3AF7LDG4KPVCRNEYFDADAG6TQ5DAX7JJVW5BLVVVNECILA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.433785086 (377h2m23.360079766s) [noise=true , address=77.90.31.13:4640 , piece_id=TPPN75DX3UCBM346CSVJN3GSOA47QV4FVPYSSAHWCJYEM42OCVLQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.598499747 (377h2m23.524794427s) [noise=true , address=79.127.201.215:4339 , piece_id=XDCTYE2SVI6RAPCXTRVANFCZYWJDZXLBHHJ47NQY6V7D36DEFPGA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:11.107785584 (377h2m38.034080273s) [noise=true , address=104.36.82.22:4541 , piece_id=XN4HDFDNZ4YUJTDPOTMMDFX4FE2YVZDGCIMHEMJGWXRPNHQ6EU7A, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:12.515567029 (377h2m39.441861713s) [noise=true , address=104.36.82.17:4345 , piece_id=YI27O5WLBYO3TPCX4MVUATFORY6U3EQMDKQHGKRHIZMTG7WGFB2A, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:12.923090587 (377h2m39.849385267s) [noise=true , address=104.36.82.8:4986 , piece_id=KPY5JVLSAEB5GUKUJJSIF5M6IIHTJISEYYAXCGUSJ7OLI5QCIBNQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:13.453172659 (377h2m40.379467346s) [noise=true , address=104.36.82.38:4698 , piece_id=TSOXJNQEWODCWFEITH4IYKZI7A75UZEXRQPIQO7LMP6LT3WLNXUA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:14.416524856 (377h2m41.342819534s) [noise=true , address=104.36.82.34:4554 , piece_id=57K5SDV3YEGLVMSJGSLXNKEWOLETXE6CWBM2NPVCNKLFHPVQCJBQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:14.846730699 (377h2m41.773025379s) [noise=true , address=104.36.82.23:4464 , piece_id=JCJ4MGEZIIZ7YCTM6QXC5ZJ7PI4OEJBOMHZYBU57D2SNGWJWB5CQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:14.846763777 (377h2m41.773058463s) [seg_pos={0 0}, upload=301954]: Segment reached optimal threshold of 14 pieces.
Jul 9 02:14:14.846894905 (377h2m41.773189582s) [noise=true , address=104.36.82.36:4644 , piece_id=K2G3MAPD7TWYZPKNHQBMEPJRIQDLE75FYVPAZFX4I3GJ4IUAA7XQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1oYHbb73ACitbfkkP5P6cmRyTt6kmaS1y49G4vbsvTvVDwByhT): protocol: expected piece hash; context canceled; EOF
Jul 9 02:14:14.846948435 (377h2m41.773243131s) [noise=true , address=104.36.82.4:4834 , piece_id=YLYIASUPNRTK7KOMRYLN7OQLQBPWC7WFGCPDGUVIOEQJWQQ7A7GA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Z241DCYA5Wig2TdEogxTpY6FzVmFbnKtmHcTwxLxfcZafgooy): protocol: expected piece hash; context canceled; EOF
Jul 9 02:14:14.847094561 (377h2m41.773389249s) [noise=true , address=104.36.82.42:4830 , piece_id=ENYQTUCIF7OWQPCXTYS24OC5QT63ZJJXOY652GLIG7OQPJEQCGGA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:142ocSHHtv8BpSebXFNk5qk1Y23a8pbFAZqE3cq6i3RMX3dPw9): protocol: expected piece hash; context canceled; EOF
Jul 9 02:14:14.847127825 (377h2m41.773422511s) [noise=true , address=104.36.82.41:4768 , piece_id=BGMF33OKUZSZKGIHJYI3RNXEBDE54TX67DYLJ7JCZMRNK6ZAU22Q, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: write tcp 159.223.143.39:43796->104.36.82.41:4768: use of closed network connection; write tcp 159.223.143.39:43796->104.36.82.41:4768: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:43796->104.36.82.41:4768: use of closed network connection
Jul 9 02:14:14.847142999 (377h2m41.773437687s) [seg_pos={0 0}, upload=301954]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jul 9 02:14:14.847204962 (377h2m41.773499652s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:14:14.847209974 (377h2m41.773504671s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:14:14.909581364 (377h2m41.83587606s) [upload=301954]: Done uploading segments...
Jul 9 02:14:14.909605483 (377h2m41.835900172s) [upload=301954]: Upload finished. err: <nil>
Duration: 20.376422978s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
21 out of 60 is failed (threshold=3) [..X....X.........X....X.X..XX.XX..X.X.X.....XXX.XX.X...XXX..] duration:
3.380692874s
FAIL (2025-07-09 02:38:02 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.3811698s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
38 out of 60 is failed (threshold=3) [XX.X.X.XXXXXXX..XX....XX.XXX.XXX.XX.XXXXXX.XXXXX.XX.....X..X] duration:
1m0.000918567s
FAIL (2025-07-09 02:37:59 UTC)
operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1m0.000939168s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
4 out of 60 is failed (threshold=3) [..........X.....X...X....................X..................] duration:
588.119633ms
FAIL (2025-07-09 02:37:51 UTC)
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
02:25:27.135824454 (377h13m54.062119141s) [noise=true , address=104.36.82.12:4150 , piece_id=DRG5SBA3EE4KVQPEQOJZ6SS537SFWYFUYQXOAQMWYTT6FPVSJIPQ, seg_pos={0 0}, upload=302099]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: context canceled; context canceled
Jul 9 02:25:27.135860079 (377h13m54.062154769s) [seg_pos={0 0}, upload=302099]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 02:25:27.135957283 (377h13m54.062251971s) [upload=302099]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:25:27.135964211 (377h13m54.062258897s) [upload=302099]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:25:27.199410713 (377h13m54.125705389s) [upload=302099]: Done uploading segments...
Jul 9 02:25:27.199427901 (377h13m54.125722577s) [upload=302099]: Upload finished. err: <nil>
Jul 9 02:25:57.289077655 (377h14m24.215372329s) [upload=302109]: Starting upload
Jul 9 02:25:57.289186750 (377h14m24.215481424s) [upload=302109]: Done starting upload
Jul 9 02:25:57.289444851 (377h14m24.215739548s) [upload=302109]: Uploading segments...
Jul 9 02:25:57.289623846 (377h14m24.215918535s) [upload=302109]: Got next segment. Inline: false
Jul 9 02:25:57.289696741 (377h14m24.215991432s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:25:57.289703375 (377h14m24.215998067s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:25:57.318313152 (377h14m24.244607838s) [seg_pos={0 0}, upload=302109]: Begin upload segment...
Jul 9 02:25:57.318363286 (377h14m24.244657964s) [seg_pos={0 0}, upload=302109]: Done begin upload segment.
Jul 9 02:25:57.318386772 (377h14m24.244681463s) [noise=true , address=77.90.31.11:4553 , piece_id=TYW7NBJWACIT7OQS7A4WGXUPR3MB2FKAN2OMGAZJERB4X2DLAO2Q, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318412127 (377h14m24.244706825s) [noise=true , address=77.90.31.10:4518 , piece_id=VJ4HWM2X4VSRSUCQL5QLMBYPFBYV4JYS5ZXCBCB6MTMB76EHZFGQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318822149 (377h14m24.245116841s) [noise=true , address=45.91.116.9:4263 , piece_id=5YILAJ5BZBGOURSZE343ST4RMOTWM6ZOB3EEQQWL5WUNYWOGSAPA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318965862 (377h14m24.24526056s) [noise=true , address=45.91.116.8:4045 , piece_id=KM6ZYTHR6JGTARKZQZYIESPKBEU5H2HQFA7OARIKD2ZXDHPFSMGA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.319187516 (377h14m24.245482203s) [noise=true , address=104.36.82.7:4909 , piece_id=ZRHLWDQ5KIDRWPJ5FXAANQTWERKXCR2PPEGN442IOSYSRFAG7VZA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.319523815 (377h14m24.245818512s) [noise=true , address=45.91.116.14:4191 , piece_id=GTR6T4OTGZTEZ76FWQZ46OCH5WB3O5X27EN4AV33ZJ7D5HX4535A, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.320556632 (377h14m24.24685133s) [noise=true , address=104.36.82.2:4732 , piece_id=DJIGHJMJRAULH2L4YT2WGRXPRQJMNKZG5LICHPSKOCHUL6QDV6BQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.321019141 (377h14m24.247313835s) [noise=true , address=77.90.31.17:4770 , piece_id=V36UBBIYOFJC6KFVVBXEVT4QBUIWN257D3YSDDRAGZVXH7ZXHTBA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.322591797 (377h14m24.248886493s) [noise=true , address=77.90.31.13:4627 , piece_id=HULEKFVZNDNKS4SWWGTJUHYZWWYPDCER5W4GFCVLLDL65PAHULFQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323120451 (377h14m24.249415138s) [noise=true , address=45.91.116.4:4939 , piece_id=ANA3EZU7OZWELNG4QEMTEAYZY5H3C3H4IJFT4XKPUB7WIF44FEVQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323544591 (377h14m24.249839288s) [noise=true , address=45.91.116.7:4011 , piece_id=VDOJRN6ACK4LO44BQTULO6BB3CLAVQI7YIC6XLR32CZV4UJSPE6Q, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323950189 (377h14m24.250244889s) [noise=true , address=104.36.82.10:4078 , piece_id=ZFVUQAEV6DDFLPKCZA2XJ2BZHMMM745LGB4JRZJZQPG72WECGMDQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.326585810 (377h14m24.252880507s) [noise=true , address=45.91.116.5:4297 , piece_id=MSDWDLY5PJHQVI5565EKANJVSTMZFEUD2OTYZ52C65XQUICNMC3A, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.327871214 (377h14m24.254165904s) [noise=true , address=104.36.82.6:4875 , piece_id=C275F2JPVAF52E34CEHT5QXCHUFGIQZ7XGMOAOUOT4RA7YOHRQLQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.328605288 (377h14m24.254899986s) [noise=true , address=104.36.82.5:4838 , piece_id=WK3ALBOEMWWTVEA4FKCS352SZ6YYB35LELURNWXQZK32K4HYDRNA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.335409087 (377h14m24.261703784s) [noise=true , address=45.91.116.6:4973 , piece_id=Z7HQQ2O5QUSJKT4EMIDMEI5PIAEHNVRYX7MALUR2GQULRFIBUOKA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.475344637 (377h14m24.401639326s) [upload=302109]: Next returned nil segment
Jul 9 02:25:57.475376181 (377h14m24.401670879s) [upload=302109]: Waiting for error group managing segments...
Jul 9 02:25:57.477269278 (377h14m24.403563958s) [noise=true , address=45.91.116.7:4011 , piece_id=VDOJRN6ACK4LO44BQTULO6BB3CLAVQI7YIC6XLR32CZV4UJSPE6Q, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.477736179 (377h14m24.40403086s) [noise=true , address=45.91.116.9:4263 , piece_id=5YILAJ5BZBGOURSZE343ST4RMOTWM6ZOB3EEQQWL5WUNYWOGSAPA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.478396652 (377h14m24.404691332s) [noise=true , address=77.90.31.10:4518 , piece_id=VJ4HWM2X4VSRSUCQL5QLMBYPFBYV4JYS5ZXCBCB6MTMB76EHZFGQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.478919538 (377h14m24.405214224s) [noise=true , address=45.91.116.14:4191 , piece_id=GTR6T4OTGZTEZ76FWQZ46OCH5WB3O5X27EN4AV33ZJ7D5HX4535A, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481426716 (377h14m24.407721396s) [noise=true , address=45.91.116.4:4939 , piece_id=ANA3EZU7OZWELNG4QEMTEAYZY5H3C3H4IJFT4XKPUB7WIF44FEVQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481718373 (377h14m24.408013051s) [noise=true , address=45.91.116.5:4297 , piece_id=MSDWDLY5PJHQVI5565EKANJVSTMZFEUD2OTYZ52C65XQUICNMC3A, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481930356 (377h14m24.408225033s) [noise=true , address=77.90.31.17:4770 , piece_id=V36UBBIYOFJC6KFVVBXEVT4QBUIWN257D3YSDDRAGZVXH7ZXHTBA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.482376147 (377h14m24.408670821s) [noise=true , address=45.91.116.8:4045 , piece_id=KM6ZYTHR6JGTARKZQZYIESPKBEU5H2HQFA7OARIKD2ZXDHPFSMGA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.482439192 (377h14m24.408733868s) [noise=true , address=45.91.116.6:4973 , piece_id=Z7HQQ2O5QUSJKT4EMIDMEI5PIAEHNVRYX7MALUR2GQULRFIBUOKA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.483047970 (377h14m24.409342644s) [noise=true , address=77.90.31.13:4627 , piece_id=HULEKFVZNDNKS4SWWGTJUHYZWWYPDCER5W4GFCVLLDL65PAHULFQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.497788632 (377h14m24.424083312s) [noise=true , address=77.90.31.11:4553 , piece_id=TYW7NBJWACIT7OQS7A4WGXUPR3MB2FKAN2OMGAZJERB4X2DLAO2Q, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:16.688921171 (377h14m43.615215851s) [noise=true , address=104.36.82.6:4875 , piece_id=C275F2JPVAF52E34CEHT5QXCHUFGIQZ7XGMOAOUOT4RA7YOHRQLQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:27.177893955 (377h14m54.104188635s) [noise=true , address=104.36.82.2:4732 , piece_id=DJIGHJMJRAULH2L4YT2WGRXPRQJMNKZG5LICHPSKOCHUL6QDV6BQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:27.177985584 (377h14m54.104280281s) [seg_pos={0 0}, upload=302109]: Segment reached optimal threshold of 13 pieces.
Jul 9 02:26:27.178189789 (377h14m54.104484461s) [noise=true , address=104.36.82.5:4838 , piece_id=WK3ALBOEMWWTVEA4FKCS352SZ6YYB35LELURNWXQZK32K4HYDRNA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178252042 (377h14m54.104546733s) [noise=true , address=104.36.82.7:4909 , piece_id=ZRHLWDQ5KIDRWPJ5FXAANQTWERKXCR2PPEGN442IOSYSRFAG7VZA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178273922 (377h14m54.104568608s) [noise=true , address=104.36.82.10:4078 , piece_id=ZFVUQAEV6DDFLPKCZA2XJ2BZHMMM745LGB4JRZJZQPG72WECGMDQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178305983 (377h14m54.104600682s) [seg_pos={0 0}, upload=302109]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 02:26:27.178428768 (377h14m54.104723465s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:26:27.178455810 (377h14m54.104750501s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:26:27.232166184 (377h14m54.158460883s) [upload=302109]: Done uploading segments...
Jul 9 02:26:27.232188444 (377h14m54.158483123s) [upload=302109]: Upload finished. err: <nil>
Duration: 588.147218ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
21 out of 60 is failed (threshold=3) [...X....X.........X....X.X..XX.XX..X.X.X.....XXX.XX.X...XXX.] duration:
29.799934088s
FAIL (2025-07-09 02:37:22 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 29.800035721s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [................................X...........................] duration:
14.871437157s
FAIL (2025-07-09 02:37:06 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
3:56.263910643 (377h2m23.19020535s) [upload=301954]: Got next segment. Inline: false
Jul 9 02:13:56.263954358 (377h2m23.190249034s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:13:56.263959891 (377h2m23.190254572s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:13:56.302204249 (377h2m23.228498944s) [seg_pos={0 0}, upload=301954]: Begin upload segment...
Jul 9 02:13:56.302291587 (377h2m23.22858628s) [seg_pos={0 0}, upload=301954]: Done begin upload segment.
Jul 9 02:13:56.302338947 (377h2m23.228633682s) [noise=true , address=77.90.31.14:4689 , piece_id=LW7OT74AJCCC3TEKY6OZI553YK6YQFQ6C7PORQB53E7FUS24KQKQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.302352599 (377h2m23.228647272s) [noise=true , address=104.36.82.8:4986 , piece_id=KPY5JVLSAEB5GUKUJJSIF5M6IIHTJISEYYAXCGUSJ7OLI5QCIBNQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.302450837 (377h2m23.228745531s) [noise=true , address=77.90.31.16:4739 , piece_id=H6UCV6RURPXKC2H3YGAT2OXV2VZMW2DWIRG6TFD6CVXKOVMDP2VQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.302958774 (377h2m23.229253667s) [noise=true , address=77.90.31.13:4640 , piece_id=TPPN75DX3UCBM346CSVJN3GSOA47QV4FVPYSSAHWCJYEM42OCVLQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303069086 (377h2m23.229363782s) [noise=true , address=104.36.82.17:4345 , piece_id=YI27O5WLBYO3TPCX4MVUATFORY6U3EQMDKQHGKRHIZMTG7WGFB2A, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303495994 (377h2m23.22979069s) [noise=true , address=104.36.82.36:4644 , piece_id=K2G3MAPD7TWYZPKNHQBMEPJRIQDLE75FYVPAZFX4I3GJ4IUAA7XQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303575840 (377h2m23.229870536s) [noise=true , address=77.90.31.21:4898 , piece_id=75XXVNT4EV4D5UTX7O5XFL44IZOYY5LZSI2VPFQZPRERL6W5M23A, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303949062 (377h2m23.230245641s) [noise=true , address=79.127.201.215:4339 , piece_id=XDCTYE2SVI6RAPCXTRVANFCZYWJDZXLBHHJ47NQY6V7D36DEFPGA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304060410 (377h2m23.230355106s) [noise=true , address=104.36.82.41:4768 , piece_id=BGMF33OKUZSZKGIHJYI3RNXEBDE54TX67DYLJ7JCZMRNK6ZAU22Q, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304407993 (377h2m23.230702685s) [noise=true , address=77.90.31.10:4525 , piece_id=GPVSIZIBZWTUCP5RFYVPFYIYIL2LWU5PU3UASQVRQLY7SSSFMQBA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304548115 (377h2m23.230842812s) [noise=true , address=104.36.82.23:4464 , piece_id=JCJ4MGEZIIZ7YCTM6QXC5ZJ7PI4OEJBOMHZYBU57D2SNGWJWB5CQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304795164 (377h2m23.231089862s) [noise=true , address=77.90.31.18:4838 , piece_id=TGWYXA3AF7LDG4KPVCRNEYFDADAG6TQ5DAX7JJVW5BLVVVNECILA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305012510 (377h2m23.231307211s) [noise=true , address=104.36.82.34:4554 , piece_id=57K5SDV3YEGLVMSJGSLXNKEWOLETXE6CWBM2NPVCNKLFHPVQCJBQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305113737 (377h2m23.231408427s) [noise=true , address=104.36.82.4:4834 , piece_id=YLYIASUPNRTK7KOMRYLN7OQLQBPWC7WFGCPDGUVIOEQJWQQ7A7GA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305383755 (377h2m23.231678434s) [noise=true , address=104.36.82.38:4698 , piece_id=TSOXJNQEWODCWFEITH4IYKZI7A75UZEXRQPIQO7LMP6LT3WLNXUA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305483822 (377h2m23.231778517s) [noise=true , address=104.36.82.42:4830 , piece_id=ENYQTUCIF7OWQPCXTYS24OC5QT63ZJJXOY652GLIG7OQPJEQCGGA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305954019 (377h2m23.232248718s) [noise=true , address=77.90.31.21:4907 , piece_id=2EUWATHUQNZ77BJV65ZFKR5WL4D3AUURLBQ32XR2IVBA7P4TITOQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.306425517 (377h2m23.232720213s) [noise=true , address=104.36.82.22:4541 , piece_id=XN4HDFDNZ4YUJTDPOTMMDFX4FE2YVZDGCIMHEMJGWXRPNHQ6EU7A, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.375470790 (377h2m23.301765481s) [upload=301954]: Next returned nil segment
Jul 9 02:13:56.375489934 (377h2m23.30178463s) [upload=301954]: Waiting for error group managing segments...
Jul 9 02:13:56.381456859 (377h2m23.307751539s) [noise=true , address=77.90.31.10:4525 , piece_id=GPVSIZIBZWTUCP5RFYVPFYIYIL2LWU5PU3UASQVRQLY7SSSFMQBA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.381956907 (377h2m23.308251586s) [noise=true , address=77.90.31.14:4689 , piece_id=LW7OT74AJCCC3TEKY6OZI553YK6YQFQ6C7PORQB53E7FUS24KQKQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.382837088 (377h2m23.309131761s) [noise=true , address=77.90.31.21:4898 , piece_id=75XXVNT4EV4D5UTX7O5XFL44IZOYY5LZSI2VPFQZPRERL6W5M23A, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.393489125 (377h2m23.319783812s) [noise=true , address=77.90.31.21:4907 , piece_id=2EUWATHUQNZ77BJV65ZFKR5WL4D3AUURLBQ32XR2IVBA7P4TITOQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.404692801 (377h2m23.330987481s) [noise=true , address=77.90.31.16:4739 , piece_id=H6UCV6RURPXKC2H3YGAT2OXV2VZMW2DWIRG6TFD6CVXKOVMDP2VQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.404986115 (377h2m23.331280795s) [noise=true , address=77.90.31.18:4838 , piece_id=TGWYXA3AF7LDG4KPVCRNEYFDADAG6TQ5DAX7JJVW5BLVVVNECILA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.433785086 (377h2m23.360079766s) [noise=true , address=77.90.31.13:4640 , piece_id=TPPN75DX3UCBM346CSVJN3GSOA47QV4FVPYSSAHWCJYEM42OCVLQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.598499747 (377h2m23.524794427s) [noise=true , address=79.127.201.215:4339 , piece_id=XDCTYE2SVI6RAPCXTRVANFCZYWJDZXLBHHJ47NQY6V7D36DEFPGA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:11.107785584 (377h2m38.034080273s) [noise=true , address=104.36.82.22:4541 , piece_id=XN4HDFDNZ4YUJTDPOTMMDFX4FE2YVZDGCIMHEMJGWXRPNHQ6EU7A, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:12.515567029 (377h2m39.441861713s) [noise=true , address=104.36.82.17:4345 , piece_id=YI27O5WLBYO3TPCX4MVUATFORY6U3EQMDKQHGKRHIZMTG7WGFB2A, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:12.923090587 (377h2m39.849385267s) [noise=true , address=104.36.82.8:4986 , piece_id=KPY5JVLSAEB5GUKUJJSIF5M6IIHTJISEYYAXCGUSJ7OLI5QCIBNQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:13.453172659 (377h2m40.379467346s) [noise=true , address=104.36.82.38:4698 , piece_id=TSOXJNQEWODCWFEITH4IYKZI7A75UZEXRQPIQO7LMP6LT3WLNXUA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:14.416524856 (377h2m41.342819534s) [noise=true , address=104.36.82.34:4554 , piece_id=57K5SDV3YEGLVMSJGSLXNKEWOLETXE6CWBM2NPVCNKLFHPVQCJBQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:14.846730699 (377h2m41.773025379s) [noise=true , address=104.36.82.23:4464 , piece_id=JCJ4MGEZIIZ7YCTM6QXC5ZJ7PI4OEJBOMHZYBU57D2SNGWJWB5CQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:14.846763777 (377h2m41.773058463s) [seg_pos={0 0}, upload=301954]: Segment reached optimal threshold of 14 pieces.
Jul 9 02:14:14.846894905 (377h2m41.773189582s) [noise=true , address=104.36.82.36:4644 , piece_id=K2G3MAPD7TWYZPKNHQBMEPJRIQDLE75FYVPAZFX4I3GJ4IUAA7XQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1oYHbb73ACitbfkkP5P6cmRyTt6kmaS1y49G4vbsvTvVDwByhT): protocol: expected piece hash; context canceled; EOF
Jul 9 02:14:14.846948435 (377h2m41.773243131s) [noise=true , address=104.36.82.4:4834 , piece_id=YLYIASUPNRTK7KOMRYLN7OQLQBPWC7WFGCPDGUVIOEQJWQQ7A7GA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Z241DCYA5Wig2TdEogxTpY6FzVmFbnKtmHcTwxLxfcZafgooy): protocol: expected piece hash; context canceled; EOF
Jul 9 02:14:14.847094561 (377h2m41.773389249s) [noise=true , address=104.36.82.42:4830 , piece_id=ENYQTUCIF7OWQPCXTYS24OC5QT63ZJJXOY652GLIG7OQPJEQCGGA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:142ocSHHtv8BpSebXFNk5qk1Y23a8pbFAZqE3cq6i3RMX3dPw9): protocol: expected piece hash; context canceled; EOF
Jul 9 02:14:14.847127825 (377h2m41.773422511s) [noise=true , address=104.36.82.41:4768 , piece_id=BGMF33OKUZSZKGIHJYI3RNXEBDE54TX67DYLJ7JCZMRNK6ZAU22Q, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: write tcp 159.223.143.39:43796->104.36.82.41:4768: use of closed network connection; write tcp 159.223.143.39:43796->104.36.82.41:4768: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:43796->104.36.82.41:4768: use of closed network connection
Jul 9 02:14:14.847142999 (377h2m41.773437687s) [seg_pos={0 0}, upload=301954]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jul 9 02:14:14.847204962 (377h2m41.773499652s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:14:14.847209974 (377h2m41.773504671s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:14:14.909581364 (377h2m41.83587606s) [upload=301954]: Done uploading segments...
Jul 9 02:14:14.909605483 (377h2m41.835900172s) [upload=301954]: Upload finished. err: <nil>
Duration: 14.871485276s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
4 out of 60 is failed (threshold=3) [...........X.....X...X....................X.................] duration:
471.964036ms
FAIL (2025-07-09 02:36:56 UTC)
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
02:25:27.135824454 (377h13m54.062119141s) [noise=true , address=104.36.82.12:4150 , piece_id=DRG5SBA3EE4KVQPEQOJZ6SS537SFWYFUYQXOAQMWYTT6FPVSJIPQ, seg_pos={0 0}, upload=302099]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: context canceled; context canceled
Jul 9 02:25:27.135860079 (377h13m54.062154769s) [seg_pos={0 0}, upload=302099]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 02:25:27.135957283 (377h13m54.062251971s) [upload=302099]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:25:27.135964211 (377h13m54.062258897s) [upload=302099]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:25:27.199410713 (377h13m54.125705389s) [upload=302099]: Done uploading segments...
Jul 9 02:25:27.199427901 (377h13m54.125722577s) [upload=302099]: Upload finished. err: <nil>
Jul 9 02:25:57.289077655 (377h14m24.215372329s) [upload=302109]: Starting upload
Jul 9 02:25:57.289186750 (377h14m24.215481424s) [upload=302109]: Done starting upload
Jul 9 02:25:57.289444851 (377h14m24.215739548s) [upload=302109]: Uploading segments...
Jul 9 02:25:57.289623846 (377h14m24.215918535s) [upload=302109]: Got next segment. Inline: false
Jul 9 02:25:57.289696741 (377h14m24.215991432s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:25:57.289703375 (377h14m24.215998067s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:25:57.318313152 (377h14m24.244607838s) [seg_pos={0 0}, upload=302109]: Begin upload segment...
Jul 9 02:25:57.318363286 (377h14m24.244657964s) [seg_pos={0 0}, upload=302109]: Done begin upload segment.
Jul 9 02:25:57.318386772 (377h14m24.244681463s) [noise=true , address=77.90.31.11:4553 , piece_id=TYW7NBJWACIT7OQS7A4WGXUPR3MB2FKAN2OMGAZJERB4X2DLAO2Q, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318412127 (377h14m24.244706825s) [noise=true , address=77.90.31.10:4518 , piece_id=VJ4HWM2X4VSRSUCQL5QLMBYPFBYV4JYS5ZXCBCB6MTMB76EHZFGQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318822149 (377h14m24.245116841s) [noise=true , address=45.91.116.9:4263 , piece_id=5YILAJ5BZBGOURSZE343ST4RMOTWM6ZOB3EEQQWL5WUNYWOGSAPA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318965862 (377h14m24.24526056s) [noise=true , address=45.91.116.8:4045 , piece_id=KM6ZYTHR6JGTARKZQZYIESPKBEU5H2HQFA7OARIKD2ZXDHPFSMGA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.319187516 (377h14m24.245482203s) [noise=true , address=104.36.82.7:4909 , piece_id=ZRHLWDQ5KIDRWPJ5FXAANQTWERKXCR2PPEGN442IOSYSRFAG7VZA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.319523815 (377h14m24.245818512s) [noise=true , address=45.91.116.14:4191 , piece_id=GTR6T4OTGZTEZ76FWQZ46OCH5WB3O5X27EN4AV33ZJ7D5HX4535A, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.320556632 (377h14m24.24685133s) [noise=true , address=104.36.82.2:4732 , piece_id=DJIGHJMJRAULH2L4YT2WGRXPRQJMNKZG5LICHPSKOCHUL6QDV6BQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.321019141 (377h14m24.247313835s) [noise=true , address=77.90.31.17:4770 , piece_id=V36UBBIYOFJC6KFVVBXEVT4QBUIWN257D3YSDDRAGZVXH7ZXHTBA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.322591797 (377h14m24.248886493s) [noise=true , address=77.90.31.13:4627 , piece_id=HULEKFVZNDNKS4SWWGTJUHYZWWYPDCER5W4GFCVLLDL65PAHULFQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323120451 (377h14m24.249415138s) [noise=true , address=45.91.116.4:4939 , piece_id=ANA3EZU7OZWELNG4QEMTEAYZY5H3C3H4IJFT4XKPUB7WIF44FEVQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323544591 (377h14m24.249839288s) [noise=true , address=45.91.116.7:4011 , piece_id=VDOJRN6ACK4LO44BQTULO6BB3CLAVQI7YIC6XLR32CZV4UJSPE6Q, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323950189 (377h14m24.250244889s) [noise=true , address=104.36.82.10:4078 , piece_id=ZFVUQAEV6DDFLPKCZA2XJ2BZHMMM745LGB4JRZJZQPG72WECGMDQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.326585810 (377h14m24.252880507s) [noise=true , address=45.91.116.5:4297 , piece_id=MSDWDLY5PJHQVI5565EKANJVSTMZFEUD2OTYZ52C65XQUICNMC3A, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.327871214 (377h14m24.254165904s) [noise=true , address=104.36.82.6:4875 , piece_id=C275F2JPVAF52E34CEHT5QXCHUFGIQZ7XGMOAOUOT4RA7YOHRQLQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.328605288 (377h14m24.254899986s) [noise=true , address=104.36.82.5:4838 , piece_id=WK3ALBOEMWWTVEA4FKCS352SZ6YYB35LELURNWXQZK32K4HYDRNA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.335409087 (377h14m24.261703784s) [noise=true , address=45.91.116.6:4973 , piece_id=Z7HQQ2O5QUSJKT4EMIDMEI5PIAEHNVRYX7MALUR2GQULRFIBUOKA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.475344637 (377h14m24.401639326s) [upload=302109]: Next returned nil segment
Jul 9 02:25:57.475376181 (377h14m24.401670879s) [upload=302109]: Waiting for error group managing segments...
Jul 9 02:25:57.477269278 (377h14m24.403563958s) [noise=true , address=45.91.116.7:4011 , piece_id=VDOJRN6ACK4LO44BQTULO6BB3CLAVQI7YIC6XLR32CZV4UJSPE6Q, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.477736179 (377h14m24.40403086s) [noise=true , address=45.91.116.9:4263 , piece_id=5YILAJ5BZBGOURSZE343ST4RMOTWM6ZOB3EEQQWL5WUNYWOGSAPA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.478396652 (377h14m24.404691332s) [noise=true , address=77.90.31.10:4518 , piece_id=VJ4HWM2X4VSRSUCQL5QLMBYPFBYV4JYS5ZXCBCB6MTMB76EHZFGQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.478919538 (377h14m24.405214224s) [noise=true , address=45.91.116.14:4191 , piece_id=GTR6T4OTGZTEZ76FWQZ46OCH5WB3O5X27EN4AV33ZJ7D5HX4535A, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481426716 (377h14m24.407721396s) [noise=true , address=45.91.116.4:4939 , piece_id=ANA3EZU7OZWELNG4QEMTEAYZY5H3C3H4IJFT4XKPUB7WIF44FEVQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481718373 (377h14m24.408013051s) [noise=true , address=45.91.116.5:4297 , piece_id=MSDWDLY5PJHQVI5565EKANJVSTMZFEUD2OTYZ52C65XQUICNMC3A, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481930356 (377h14m24.408225033s) [noise=true , address=77.90.31.17:4770 , piece_id=V36UBBIYOFJC6KFVVBXEVT4QBUIWN257D3YSDDRAGZVXH7ZXHTBA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.482376147 (377h14m24.408670821s) [noise=true , address=45.91.116.8:4045 , piece_id=KM6ZYTHR6JGTARKZQZYIESPKBEU5H2HQFA7OARIKD2ZXDHPFSMGA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.482439192 (377h14m24.408733868s) [noise=true , address=45.91.116.6:4973 , piece_id=Z7HQQ2O5QUSJKT4EMIDMEI5PIAEHNVRYX7MALUR2GQULRFIBUOKA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.483047970 (377h14m24.409342644s) [noise=true , address=77.90.31.13:4627 , piece_id=HULEKFVZNDNKS4SWWGTJUHYZWWYPDCER5W4GFCVLLDL65PAHULFQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.497788632 (377h14m24.424083312s) [noise=true , address=77.90.31.11:4553 , piece_id=TYW7NBJWACIT7OQS7A4WGXUPR3MB2FKAN2OMGAZJERB4X2DLAO2Q, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:16.688921171 (377h14m43.615215851s) [noise=true , address=104.36.82.6:4875 , piece_id=C275F2JPVAF52E34CEHT5QXCHUFGIQZ7XGMOAOUOT4RA7YOHRQLQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:27.177893955 (377h14m54.104188635s) [noise=true , address=104.36.82.2:4732 , piece_id=DJIGHJMJRAULH2L4YT2WGRXPRQJMNKZG5LICHPSKOCHUL6QDV6BQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:27.177985584 (377h14m54.104280281s) [seg_pos={0 0}, upload=302109]: Segment reached optimal threshold of 13 pieces.
Jul 9 02:26:27.178189789 (377h14m54.104484461s) [noise=true , address=104.36.82.5:4838 , piece_id=WK3ALBOEMWWTVEA4FKCS352SZ6YYB35LELURNWXQZK32K4HYDRNA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178252042 (377h14m54.104546733s) [noise=true , address=104.36.82.7:4909 , piece_id=ZRHLWDQ5KIDRWPJ5FXAANQTWERKXCR2PPEGN442IOSYSRFAG7VZA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178273922 (377h14m54.104568608s) [noise=true , address=104.36.82.10:4078 , piece_id=ZFVUQAEV6DDFLPKCZA2XJ2BZHMMM745LGB4JRZJZQPG72WECGMDQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178305983 (377h14m54.104600682s) [seg_pos={0 0}, upload=302109]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 02:26:27.178428768 (377h14m54.104723465s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:26:27.178455810 (377h14m54.104750501s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:26:27.232166184 (377h14m54.158460883s) [upload=302109]: Done uploading segments...
Jul 9 02:26:27.232188444 (377h14m54.158483123s) [upload=302109]: Upload finished. err: <nil>
Duration: 471.99724ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
4 out of 60 is failed (threshold=3) [............X.....X...X....................X................] duration:
11.738729677s
FAIL (2025-07-09 02:36:39 UTC)
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
02:25:27.135824454 (377h13m54.062119141s) [noise=true , address=104.36.82.12:4150 , piece_id=DRG5SBA3EE4KVQPEQOJZ6SS537SFWYFUYQXOAQMWYTT6FPVSJIPQ, seg_pos={0 0}, upload=302099]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: context canceled; context canceled
Jul 9 02:25:27.135860079 (377h13m54.062154769s) [seg_pos={0 0}, upload=302099]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 02:25:27.135957283 (377h13m54.062251971s) [upload=302099]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:25:27.135964211 (377h13m54.062258897s) [upload=302099]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:25:27.199410713 (377h13m54.125705389s) [upload=302099]: Done uploading segments...
Jul 9 02:25:27.199427901 (377h13m54.125722577s) [upload=302099]: Upload finished. err: <nil>
Jul 9 02:25:57.289077655 (377h14m24.215372329s) [upload=302109]: Starting upload
Jul 9 02:25:57.289186750 (377h14m24.215481424s) [upload=302109]: Done starting upload
Jul 9 02:25:57.289444851 (377h14m24.215739548s) [upload=302109]: Uploading segments...
Jul 9 02:25:57.289623846 (377h14m24.215918535s) [upload=302109]: Got next segment. Inline: false
Jul 9 02:25:57.289696741 (377h14m24.215991432s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:25:57.289703375 (377h14m24.215998067s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:25:57.318313152 (377h14m24.244607838s) [seg_pos={0 0}, upload=302109]: Begin upload segment...
Jul 9 02:25:57.318363286 (377h14m24.244657964s) [seg_pos={0 0}, upload=302109]: Done begin upload segment.
Jul 9 02:25:57.318386772 (377h14m24.244681463s) [noise=true , address=77.90.31.11:4553 , piece_id=TYW7NBJWACIT7OQS7A4WGXUPR3MB2FKAN2OMGAZJERB4X2DLAO2Q, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318412127 (377h14m24.244706825s) [noise=true , address=77.90.31.10:4518 , piece_id=VJ4HWM2X4VSRSUCQL5QLMBYPFBYV4JYS5ZXCBCB6MTMB76EHZFGQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318822149 (377h14m24.245116841s) [noise=true , address=45.91.116.9:4263 , piece_id=5YILAJ5BZBGOURSZE343ST4RMOTWM6ZOB3EEQQWL5WUNYWOGSAPA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318965862 (377h14m24.24526056s) [noise=true , address=45.91.116.8:4045 , piece_id=KM6ZYTHR6JGTARKZQZYIESPKBEU5H2HQFA7OARIKD2ZXDHPFSMGA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.319187516 (377h14m24.245482203s) [noise=true , address=104.36.82.7:4909 , piece_id=ZRHLWDQ5KIDRWPJ5FXAANQTWERKXCR2PPEGN442IOSYSRFAG7VZA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.319523815 (377h14m24.245818512s) [noise=true , address=45.91.116.14:4191 , piece_id=GTR6T4OTGZTEZ76FWQZ46OCH5WB3O5X27EN4AV33ZJ7D5HX4535A, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.320556632 (377h14m24.24685133s) [noise=true , address=104.36.82.2:4732 , piece_id=DJIGHJMJRAULH2L4YT2WGRXPRQJMNKZG5LICHPSKOCHUL6QDV6BQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.321019141 (377h14m24.247313835s) [noise=true , address=77.90.31.17:4770 , piece_id=V36UBBIYOFJC6KFVVBXEVT4QBUIWN257D3YSDDRAGZVXH7ZXHTBA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.322591797 (377h14m24.248886493s) [noise=true , address=77.90.31.13:4627 , piece_id=HULEKFVZNDNKS4SWWGTJUHYZWWYPDCER5W4GFCVLLDL65PAHULFQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323120451 (377h14m24.249415138s) [noise=true , address=45.91.116.4:4939 , piece_id=ANA3EZU7OZWELNG4QEMTEAYZY5H3C3H4IJFT4XKPUB7WIF44FEVQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323544591 (377h14m24.249839288s) [noise=true , address=45.91.116.7:4011 , piece_id=VDOJRN6ACK4LO44BQTULO6BB3CLAVQI7YIC6XLR32CZV4UJSPE6Q, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323950189 (377h14m24.250244889s) [noise=true , address=104.36.82.10:4078 , piece_id=ZFVUQAEV6DDFLPKCZA2XJ2BZHMMM745LGB4JRZJZQPG72WECGMDQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.326585810 (377h14m24.252880507s) [noise=true , address=45.91.116.5:4297 , piece_id=MSDWDLY5PJHQVI5565EKANJVSTMZFEUD2OTYZ52C65XQUICNMC3A, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.327871214 (377h14m24.254165904s) [noise=true , address=104.36.82.6:4875 , piece_id=C275F2JPVAF52E34CEHT5QXCHUFGIQZ7XGMOAOUOT4RA7YOHRQLQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.328605288 (377h14m24.254899986s) [noise=true , address=104.36.82.5:4838 , piece_id=WK3ALBOEMWWTVEA4FKCS352SZ6YYB35LELURNWXQZK32K4HYDRNA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.335409087 (377h14m24.261703784s) [noise=true , address=45.91.116.6:4973 , piece_id=Z7HQQ2O5QUSJKT4EMIDMEI5PIAEHNVRYX7MALUR2GQULRFIBUOKA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.475344637 (377h14m24.401639326s) [upload=302109]: Next returned nil segment
Jul 9 02:25:57.475376181 (377h14m24.401670879s) [upload=302109]: Waiting for error group managing segments...
Jul 9 02:25:57.477269278 (377h14m24.403563958s) [noise=true , address=45.91.116.7:4011 , piece_id=VDOJRN6ACK4LO44BQTULO6BB3CLAVQI7YIC6XLR32CZV4UJSPE6Q, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.477736179 (377h14m24.40403086s) [noise=true , address=45.91.116.9:4263 , piece_id=5YILAJ5BZBGOURSZE343ST4RMOTWM6ZOB3EEQQWL5WUNYWOGSAPA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.478396652 (377h14m24.404691332s) [noise=true , address=77.90.31.10:4518 , piece_id=VJ4HWM2X4VSRSUCQL5QLMBYPFBYV4JYS5ZXCBCB6MTMB76EHZFGQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.478919538 (377h14m24.405214224s) [noise=true , address=45.91.116.14:4191 , piece_id=GTR6T4OTGZTEZ76FWQZ46OCH5WB3O5X27EN4AV33ZJ7D5HX4535A, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481426716 (377h14m24.407721396s) [noise=true , address=45.91.116.4:4939 , piece_id=ANA3EZU7OZWELNG4QEMTEAYZY5H3C3H4IJFT4XKPUB7WIF44FEVQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481718373 (377h14m24.408013051s) [noise=true , address=45.91.116.5:4297 , piece_id=MSDWDLY5PJHQVI5565EKANJVSTMZFEUD2OTYZ52C65XQUICNMC3A, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481930356 (377h14m24.408225033s) [noise=true , address=77.90.31.17:4770 , piece_id=V36UBBIYOFJC6KFVVBXEVT4QBUIWN257D3YSDDRAGZVXH7ZXHTBA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.482376147 (377h14m24.408670821s) [noise=true , address=45.91.116.8:4045 , piece_id=KM6ZYTHR6JGTARKZQZYIESPKBEU5H2HQFA7OARIKD2ZXDHPFSMGA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.482439192 (377h14m24.408733868s) [noise=true , address=45.91.116.6:4973 , piece_id=Z7HQQ2O5QUSJKT4EMIDMEI5PIAEHNVRYX7MALUR2GQULRFIBUOKA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.483047970 (377h14m24.409342644s) [noise=true , address=77.90.31.13:4627 , piece_id=HULEKFVZNDNKS4SWWGTJUHYZWWYPDCER5W4GFCVLLDL65PAHULFQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.497788632 (377h14m24.424083312s) [noise=true , address=77.90.31.11:4553 , piece_id=TYW7NBJWACIT7OQS7A4WGXUPR3MB2FKAN2OMGAZJERB4X2DLAO2Q, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:16.688921171 (377h14m43.615215851s) [noise=true , address=104.36.82.6:4875 , piece_id=C275F2JPVAF52E34CEHT5QXCHUFGIQZ7XGMOAOUOT4RA7YOHRQLQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:27.177893955 (377h14m54.104188635s) [noise=true , address=104.36.82.2:4732 , piece_id=DJIGHJMJRAULH2L4YT2WGRXPRQJMNKZG5LICHPSKOCHUL6QDV6BQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:27.177985584 (377h14m54.104280281s) [seg_pos={0 0}, upload=302109]: Segment reached optimal threshold of 13 pieces.
Jul 9 02:26:27.178189789 (377h14m54.104484461s) [noise=true , address=104.36.82.5:4838 , piece_id=WK3ALBOEMWWTVEA4FKCS352SZ6YYB35LELURNWXQZK32K4HYDRNA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178252042 (377h14m54.104546733s) [noise=true , address=104.36.82.7:4909 , piece_id=ZRHLWDQ5KIDRWPJ5FXAANQTWERKXCR2PPEGN442IOSYSRFAG7VZA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178273922 (377h14m54.104568608s) [noise=true , address=104.36.82.10:4078 , piece_id=ZFVUQAEV6DDFLPKCZA2XJ2BZHMMM745LGB4JRZJZQPG72WECGMDQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178305983 (377h14m54.104600682s) [seg_pos={0 0}, upload=302109]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 02:26:27.178428768 (377h14m54.104723465s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:26:27.178455810 (377h14m54.104750501s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:26:27.232166184 (377h14m54.158460883s) [upload=302109]: Done uploading segments...
Jul 9 02:26:27.232188444 (377h14m54.158483123s) [upload=302109]: Upload finished. err: <nil>
Duration: 11.738764034s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
38 out of 60 is failed (threshold=3) [XXX.X.X.XXXXXXX..XX....XX.XXX.XXX.XX.XXXXXX.XXXXX.XX.....X..] duration:
42.84864612s
FAIL (2025-07-09 02:36:32 UTC)
operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 42.84867441s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.................................X..........................] duration:
28.708685227s
FAIL (2025-07-09 02:36:31 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
3:56.263910643 (377h2m23.19020535s) [upload=301954]: Got next segment. Inline: false
Jul 9 02:13:56.263954358 (377h2m23.190249034s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:13:56.263959891 (377h2m23.190254572s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:13:56.302204249 (377h2m23.228498944s) [seg_pos={0 0}, upload=301954]: Begin upload segment...
Jul 9 02:13:56.302291587 (377h2m23.22858628s) [seg_pos={0 0}, upload=301954]: Done begin upload segment.
Jul 9 02:13:56.302338947 (377h2m23.228633682s) [noise=true , address=77.90.31.14:4689 , piece_id=LW7OT74AJCCC3TEKY6OZI553YK6YQFQ6C7PORQB53E7FUS24KQKQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.302352599 (377h2m23.228647272s) [noise=true , address=104.36.82.8:4986 , piece_id=KPY5JVLSAEB5GUKUJJSIF5M6IIHTJISEYYAXCGUSJ7OLI5QCIBNQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.302450837 (377h2m23.228745531s) [noise=true , address=77.90.31.16:4739 , piece_id=H6UCV6RURPXKC2H3YGAT2OXV2VZMW2DWIRG6TFD6CVXKOVMDP2VQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.302958774 (377h2m23.229253667s) [noise=true , address=77.90.31.13:4640 , piece_id=TPPN75DX3UCBM346CSVJN3GSOA47QV4FVPYSSAHWCJYEM42OCVLQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303069086 (377h2m23.229363782s) [noise=true , address=104.36.82.17:4345 , piece_id=YI27O5WLBYO3TPCX4MVUATFORY6U3EQMDKQHGKRHIZMTG7WGFB2A, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303495994 (377h2m23.22979069s) [noise=true , address=104.36.82.36:4644 , piece_id=K2G3MAPD7TWYZPKNHQBMEPJRIQDLE75FYVPAZFX4I3GJ4IUAA7XQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303575840 (377h2m23.229870536s) [noise=true , address=77.90.31.21:4898 , piece_id=75XXVNT4EV4D5UTX7O5XFL44IZOYY5LZSI2VPFQZPRERL6W5M23A, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303949062 (377h2m23.230245641s) [noise=true , address=79.127.201.215:4339 , piece_id=XDCTYE2SVI6RAPCXTRVANFCZYWJDZXLBHHJ47NQY6V7D36DEFPGA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304060410 (377h2m23.230355106s) [noise=true , address=104.36.82.41:4768 , piece_id=BGMF33OKUZSZKGIHJYI3RNXEBDE54TX67DYLJ7JCZMRNK6ZAU22Q, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304407993 (377h2m23.230702685s) [noise=true , address=77.90.31.10:4525 , piece_id=GPVSIZIBZWTUCP5RFYVPFYIYIL2LWU5PU3UASQVRQLY7SSSFMQBA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304548115 (377h2m23.230842812s) [noise=true , address=104.36.82.23:4464 , piece_id=JCJ4MGEZIIZ7YCTM6QXC5ZJ7PI4OEJBOMHZYBU57D2SNGWJWB5CQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304795164 (377h2m23.231089862s) [noise=true , address=77.90.31.18:4838 , piece_id=TGWYXA3AF7LDG4KPVCRNEYFDADAG6TQ5DAX7JJVW5BLVVVNECILA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305012510 (377h2m23.231307211s) [noise=true , address=104.36.82.34:4554 , piece_id=57K5SDV3YEGLVMSJGSLXNKEWOLETXE6CWBM2NPVCNKLFHPVQCJBQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305113737 (377h2m23.231408427s) [noise=true , address=104.36.82.4:4834 , piece_id=YLYIASUPNRTK7KOMRYLN7OQLQBPWC7WFGCPDGUVIOEQJWQQ7A7GA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305383755 (377h2m23.231678434s) [noise=true , address=104.36.82.38:4698 , piece_id=TSOXJNQEWODCWFEITH4IYKZI7A75UZEXRQPIQO7LMP6LT3WLNXUA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305483822 (377h2m23.231778517s) [noise=true , address=104.36.82.42:4830 , piece_id=ENYQTUCIF7OWQPCXTYS24OC5QT63ZJJXOY652GLIG7OQPJEQCGGA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305954019 (377h2m23.232248718s) [noise=true , address=77.90.31.21:4907 , piece_id=2EUWATHUQNZ77BJV65ZFKR5WL4D3AUURLBQ32XR2IVBA7P4TITOQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.306425517 (377h2m23.232720213s) [noise=true , address=104.36.82.22:4541 , piece_id=XN4HDFDNZ4YUJTDPOTMMDFX4FE2YVZDGCIMHEMJGWXRPNHQ6EU7A, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.375470790 (377h2m23.301765481s) [upload=301954]: Next returned nil segment
Jul 9 02:13:56.375489934 (377h2m23.30178463s) [upload=301954]: Waiting for error group managing segments...
Jul 9 02:13:56.381456859 (377h2m23.307751539s) [noise=true , address=77.90.31.10:4525 , piece_id=GPVSIZIBZWTUCP5RFYVPFYIYIL2LWU5PU3UASQVRQLY7SSSFMQBA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.381956907 (377h2m23.308251586s) [noise=true , address=77.90.31.14:4689 , piece_id=LW7OT74AJCCC3TEKY6OZI553YK6YQFQ6C7PORQB53E7FUS24KQKQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.382837088 (377h2m23.309131761s) [noise=true , address=77.90.31.21:4898 , piece_id=75XXVNT4EV4D5UTX7O5XFL44IZOYY5LZSI2VPFQZPRERL6W5M23A, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.393489125 (377h2m23.319783812s) [noise=true , address=77.90.31.21:4907 , piece_id=2EUWATHUQNZ77BJV65ZFKR5WL4D3AUURLBQ32XR2IVBA7P4TITOQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.404692801 (377h2m23.330987481s) [noise=true , address=77.90.31.16:4739 , piece_id=H6UCV6RURPXKC2H3YGAT2OXV2VZMW2DWIRG6TFD6CVXKOVMDP2VQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.404986115 (377h2m23.331280795s) [noise=true , address=77.90.31.18:4838 , piece_id=TGWYXA3AF7LDG4KPVCRNEYFDADAG6TQ5DAX7JJVW5BLVVVNECILA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.433785086 (377h2m23.360079766s) [noise=true , address=77.90.31.13:4640 , piece_id=TPPN75DX3UCBM346CSVJN3GSOA47QV4FVPYSSAHWCJYEM42OCVLQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.598499747 (377h2m23.524794427s) [noise=true , address=79.127.201.215:4339 , piece_id=XDCTYE2SVI6RAPCXTRVANFCZYWJDZXLBHHJ47NQY6V7D36DEFPGA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:11.107785584 (377h2m38.034080273s) [noise=true , address=104.36.82.22:4541 , piece_id=XN4HDFDNZ4YUJTDPOTMMDFX4FE2YVZDGCIMHEMJGWXRPNHQ6EU7A, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:12.515567029 (377h2m39.441861713s) [noise=true , address=104.36.82.17:4345 , piece_id=YI27O5WLBYO3TPCX4MVUATFORY6U3EQMDKQHGKRHIZMTG7WGFB2A, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:12.923090587 (377h2m39.849385267s) [noise=true , address=104.36.82.8:4986 , piece_id=KPY5JVLSAEB5GUKUJJSIF5M6IIHTJISEYYAXCGUSJ7OLI5QCIBNQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:13.453172659 (377h2m40.379467346s) [noise=true , address=104.36.82.38:4698 , piece_id=TSOXJNQEWODCWFEITH4IYKZI7A75UZEXRQPIQO7LMP6LT3WLNXUA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:14.416524856 (377h2m41.342819534s) [noise=true , address=104.36.82.34:4554 , piece_id=57K5SDV3YEGLVMSJGSLXNKEWOLETXE6CWBM2NPVCNKLFHPVQCJBQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:14.846730699 (377h2m41.773025379s) [noise=true , address=104.36.82.23:4464 , piece_id=JCJ4MGEZIIZ7YCTM6QXC5ZJ7PI4OEJBOMHZYBU57D2SNGWJWB5CQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:14.846763777 (377h2m41.773058463s) [seg_pos={0 0}, upload=301954]: Segment reached optimal threshold of 14 pieces.
Jul 9 02:14:14.846894905 (377h2m41.773189582s) [noise=true , address=104.36.82.36:4644 , piece_id=K2G3MAPD7TWYZPKNHQBMEPJRIQDLE75FYVPAZFX4I3GJ4IUAA7XQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1oYHbb73ACitbfkkP5P6cmRyTt6kmaS1y49G4vbsvTvVDwByhT): protocol: expected piece hash; context canceled; EOF
Jul 9 02:14:14.846948435 (377h2m41.773243131s) [noise=true , address=104.36.82.4:4834 , piece_id=YLYIASUPNRTK7KOMRYLN7OQLQBPWC7WFGCPDGUVIOEQJWQQ7A7GA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Z241DCYA5Wig2TdEogxTpY6FzVmFbnKtmHcTwxLxfcZafgooy): protocol: expected piece hash; context canceled; EOF
Jul 9 02:14:14.847094561 (377h2m41.773389249s) [noise=true , address=104.36.82.42:4830 , piece_id=ENYQTUCIF7OWQPCXTYS24OC5QT63ZJJXOY652GLIG7OQPJEQCGGA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:142ocSHHtv8BpSebXFNk5qk1Y23a8pbFAZqE3cq6i3RMX3dPw9): protocol: expected piece hash; context canceled; EOF
Jul 9 02:14:14.847127825 (377h2m41.773422511s) [noise=true , address=104.36.82.41:4768 , piece_id=BGMF33OKUZSZKGIHJYI3RNXEBDE54TX67DYLJ7JCZMRNK6ZAU22Q, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: write tcp 159.223.143.39:43796->104.36.82.41:4768: use of closed network connection; write tcp 159.223.143.39:43796->104.36.82.41:4768: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:43796->104.36.82.41:4768: use of closed network connection
Jul 9 02:14:14.847142999 (377h2m41.773437687s) [seg_pos={0 0}, upload=301954]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jul 9 02:14:14.847204962 (377h2m41.773499652s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:14:14.847209974 (377h2m41.773504671s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:14:14.909581364 (377h2m41.83587606s) [upload=301954]: Done uploading segments...
Jul 9 02:14:14.909605483 (377h2m41.835900172s) [upload=301954]: Upload finished. err: <nil>
Duration: 28.708716386s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
21 out of 60 is failed (threshold=3) [....X....X.........X....X.X..XX.XX..X.X.X.....XXX.XX.X...XXX] duration:
30.001880485s
FAIL (2025-07-09 02:36:22 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 30.001925097s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
4 out of 60 is failed (threshold=3) [.............X.....X...X....................X...............] duration:
419.16217ms
FAIL (2025-07-09 02:36:06 UTC)
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
02:25:27.135824454 (377h13m54.062119141s) [noise=true , address=104.36.82.12:4150 , piece_id=DRG5SBA3EE4KVQPEQOJZ6SS537SFWYFUYQXOAQMWYTT6FPVSJIPQ, seg_pos={0 0}, upload=302099]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: context canceled; context canceled
Jul 9 02:25:27.135860079 (377h13m54.062154769s) [seg_pos={0 0}, upload=302099]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 02:25:27.135957283 (377h13m54.062251971s) [upload=302099]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:25:27.135964211 (377h13m54.062258897s) [upload=302099]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:25:27.199410713 (377h13m54.125705389s) [upload=302099]: Done uploading segments...
Jul 9 02:25:27.199427901 (377h13m54.125722577s) [upload=302099]: Upload finished. err: <nil>
Jul 9 02:25:57.289077655 (377h14m24.215372329s) [upload=302109]: Starting upload
Jul 9 02:25:57.289186750 (377h14m24.215481424s) [upload=302109]: Done starting upload
Jul 9 02:25:57.289444851 (377h14m24.215739548s) [upload=302109]: Uploading segments...
Jul 9 02:25:57.289623846 (377h14m24.215918535s) [upload=302109]: Got next segment. Inline: false
Jul 9 02:25:57.289696741 (377h14m24.215991432s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:25:57.289703375 (377h14m24.215998067s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:25:57.318313152 (377h14m24.244607838s) [seg_pos={0 0}, upload=302109]: Begin upload segment...
Jul 9 02:25:57.318363286 (377h14m24.244657964s) [seg_pos={0 0}, upload=302109]: Done begin upload segment.
Jul 9 02:25:57.318386772 (377h14m24.244681463s) [noise=true , address=77.90.31.11:4553 , piece_id=TYW7NBJWACIT7OQS7A4WGXUPR3MB2FKAN2OMGAZJERB4X2DLAO2Q, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318412127 (377h14m24.244706825s) [noise=true , address=77.90.31.10:4518 , piece_id=VJ4HWM2X4VSRSUCQL5QLMBYPFBYV4JYS5ZXCBCB6MTMB76EHZFGQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318822149 (377h14m24.245116841s) [noise=true , address=45.91.116.9:4263 , piece_id=5YILAJ5BZBGOURSZE343ST4RMOTWM6ZOB3EEQQWL5WUNYWOGSAPA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318965862 (377h14m24.24526056s) [noise=true , address=45.91.116.8:4045 , piece_id=KM6ZYTHR6JGTARKZQZYIESPKBEU5H2HQFA7OARIKD2ZXDHPFSMGA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.319187516 (377h14m24.245482203s) [noise=true , address=104.36.82.7:4909 , piece_id=ZRHLWDQ5KIDRWPJ5FXAANQTWERKXCR2PPEGN442IOSYSRFAG7VZA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.319523815 (377h14m24.245818512s) [noise=true , address=45.91.116.14:4191 , piece_id=GTR6T4OTGZTEZ76FWQZ46OCH5WB3O5X27EN4AV33ZJ7D5HX4535A, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.320556632 (377h14m24.24685133s) [noise=true , address=104.36.82.2:4732 , piece_id=DJIGHJMJRAULH2L4YT2WGRXPRQJMNKZG5LICHPSKOCHUL6QDV6BQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.321019141 (377h14m24.247313835s) [noise=true , address=77.90.31.17:4770 , piece_id=V36UBBIYOFJC6KFVVBXEVT4QBUIWN257D3YSDDRAGZVXH7ZXHTBA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.322591797 (377h14m24.248886493s) [noise=true , address=77.90.31.13:4627 , piece_id=HULEKFVZNDNKS4SWWGTJUHYZWWYPDCER5W4GFCVLLDL65PAHULFQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323120451 (377h14m24.249415138s) [noise=true , address=45.91.116.4:4939 , piece_id=ANA3EZU7OZWELNG4QEMTEAYZY5H3C3H4IJFT4XKPUB7WIF44FEVQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323544591 (377h14m24.249839288s) [noise=true , address=45.91.116.7:4011 , piece_id=VDOJRN6ACK4LO44BQTULO6BB3CLAVQI7YIC6XLR32CZV4UJSPE6Q, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323950189 (377h14m24.250244889s) [noise=true , address=104.36.82.10:4078 , piece_id=ZFVUQAEV6DDFLPKCZA2XJ2BZHMMM745LGB4JRZJZQPG72WECGMDQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.326585810 (377h14m24.252880507s) [noise=true , address=45.91.116.5:4297 , piece_id=MSDWDLY5PJHQVI5565EKANJVSTMZFEUD2OTYZ52C65XQUICNMC3A, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.327871214 (377h14m24.254165904s) [noise=true , address=104.36.82.6:4875 , piece_id=C275F2JPVAF52E34CEHT5QXCHUFGIQZ7XGMOAOUOT4RA7YOHRQLQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.328605288 (377h14m24.254899986s) [noise=true , address=104.36.82.5:4838 , piece_id=WK3ALBOEMWWTVEA4FKCS352SZ6YYB35LELURNWXQZK32K4HYDRNA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.335409087 (377h14m24.261703784s) [noise=true , address=45.91.116.6:4973 , piece_id=Z7HQQ2O5QUSJKT4EMIDMEI5PIAEHNVRYX7MALUR2GQULRFIBUOKA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.475344637 (377h14m24.401639326s) [upload=302109]: Next returned nil segment
Jul 9 02:25:57.475376181 (377h14m24.401670879s) [upload=302109]: Waiting for error group managing segments...
Jul 9 02:25:57.477269278 (377h14m24.403563958s) [noise=true , address=45.91.116.7:4011 , piece_id=VDOJRN6ACK4LO44BQTULO6BB3CLAVQI7YIC6XLR32CZV4UJSPE6Q, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.477736179 (377h14m24.40403086s) [noise=true , address=45.91.116.9:4263 , piece_id=5YILAJ5BZBGOURSZE343ST4RMOTWM6ZOB3EEQQWL5WUNYWOGSAPA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.478396652 (377h14m24.404691332s) [noise=true , address=77.90.31.10:4518 , piece_id=VJ4HWM2X4VSRSUCQL5QLMBYPFBYV4JYS5ZXCBCB6MTMB76EHZFGQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.478919538 (377h14m24.405214224s) [noise=true , address=45.91.116.14:4191 , piece_id=GTR6T4OTGZTEZ76FWQZ46OCH5WB3O5X27EN4AV33ZJ7D5HX4535A, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481426716 (377h14m24.407721396s) [noise=true , address=45.91.116.4:4939 , piece_id=ANA3EZU7OZWELNG4QEMTEAYZY5H3C3H4IJFT4XKPUB7WIF44FEVQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481718373 (377h14m24.408013051s) [noise=true , address=45.91.116.5:4297 , piece_id=MSDWDLY5PJHQVI5565EKANJVSTMZFEUD2OTYZ52C65XQUICNMC3A, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481930356 (377h14m24.408225033s) [noise=true , address=77.90.31.17:4770 , piece_id=V36UBBIYOFJC6KFVVBXEVT4QBUIWN257D3YSDDRAGZVXH7ZXHTBA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.482376147 (377h14m24.408670821s) [noise=true , address=45.91.116.8:4045 , piece_id=KM6ZYTHR6JGTARKZQZYIESPKBEU5H2HQFA7OARIKD2ZXDHPFSMGA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.482439192 (377h14m24.408733868s) [noise=true , address=45.91.116.6:4973 , piece_id=Z7HQQ2O5QUSJKT4EMIDMEI5PIAEHNVRYX7MALUR2GQULRFIBUOKA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.483047970 (377h14m24.409342644s) [noise=true , address=77.90.31.13:4627 , piece_id=HULEKFVZNDNKS4SWWGTJUHYZWWYPDCER5W4GFCVLLDL65PAHULFQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.497788632 (377h14m24.424083312s) [noise=true , address=77.90.31.11:4553 , piece_id=TYW7NBJWACIT7OQS7A4WGXUPR3MB2FKAN2OMGAZJERB4X2DLAO2Q, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:16.688921171 (377h14m43.615215851s) [noise=true , address=104.36.82.6:4875 , piece_id=C275F2JPVAF52E34CEHT5QXCHUFGIQZ7XGMOAOUOT4RA7YOHRQLQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:27.177893955 (377h14m54.104188635s) [noise=true , address=104.36.82.2:4732 , piece_id=DJIGHJMJRAULH2L4YT2WGRXPRQJMNKZG5LICHPSKOCHUL6QDV6BQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:27.177985584 (377h14m54.104280281s) [seg_pos={0 0}, upload=302109]: Segment reached optimal threshold of 13 pieces.
Jul 9 02:26:27.178189789 (377h14m54.104484461s) [noise=true , address=104.36.82.5:4838 , piece_id=WK3ALBOEMWWTVEA4FKCS352SZ6YYB35LELURNWXQZK32K4HYDRNA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178252042 (377h14m54.104546733s) [noise=true , address=104.36.82.7:4909 , piece_id=ZRHLWDQ5KIDRWPJ5FXAANQTWERKXCR2PPEGN442IOSYSRFAG7VZA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178273922 (377h14m54.104568608s) [noise=true , address=104.36.82.10:4078 , piece_id=ZFVUQAEV6DDFLPKCZA2XJ2BZHMMM745LGB4JRZJZQPG72WECGMDQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178305983 (377h14m54.104600682s) [seg_pos={0 0}, upload=302109]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 02:26:27.178428768 (377h14m54.104723465s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:26:27.178455810 (377h14m54.104750501s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:26:27.232166184 (377h14m54.158460883s) [upload=302109]: Done uploading segments...
Jul 9 02:26:27.232188444 (377h14m54.158483123s) [upload=302109]: Upload finished. err: <nil>
Duration: 419.205125ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [..................................X.........................] duration:
29.090219219s
FAIL (2025-07-09 02:35:53 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
3:56.263910643 (377h2m23.19020535s) [upload=301954]: Got next segment. Inline: false
Jul 9 02:13:56.263954358 (377h2m23.190249034s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:13:56.263959891 (377h2m23.190254572s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:13:56.302204249 (377h2m23.228498944s) [seg_pos={0 0}, upload=301954]: Begin upload segment...
Jul 9 02:13:56.302291587 (377h2m23.22858628s) [seg_pos={0 0}, upload=301954]: Done begin upload segment.
Jul 9 02:13:56.302338947 (377h2m23.228633682s) [noise=true , address=77.90.31.14:4689 , piece_id=LW7OT74AJCCC3TEKY6OZI553YK6YQFQ6C7PORQB53E7FUS24KQKQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.302352599 (377h2m23.228647272s) [noise=true , address=104.36.82.8:4986 , piece_id=KPY5JVLSAEB5GUKUJJSIF5M6IIHTJISEYYAXCGUSJ7OLI5QCIBNQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.302450837 (377h2m23.228745531s) [noise=true , address=77.90.31.16:4739 , piece_id=H6UCV6RURPXKC2H3YGAT2OXV2VZMW2DWIRG6TFD6CVXKOVMDP2VQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.302958774 (377h2m23.229253667s) [noise=true , address=77.90.31.13:4640 , piece_id=TPPN75DX3UCBM346CSVJN3GSOA47QV4FVPYSSAHWCJYEM42OCVLQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303069086 (377h2m23.229363782s) [noise=true , address=104.36.82.17:4345 , piece_id=YI27O5WLBYO3TPCX4MVUATFORY6U3EQMDKQHGKRHIZMTG7WGFB2A, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303495994 (377h2m23.22979069s) [noise=true , address=104.36.82.36:4644 , piece_id=K2G3MAPD7TWYZPKNHQBMEPJRIQDLE75FYVPAZFX4I3GJ4IUAA7XQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303575840 (377h2m23.229870536s) [noise=true , address=77.90.31.21:4898 , piece_id=75XXVNT4EV4D5UTX7O5XFL44IZOYY5LZSI2VPFQZPRERL6W5M23A, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303949062 (377h2m23.230245641s) [noise=true , address=79.127.201.215:4339 , piece_id=XDCTYE2SVI6RAPCXTRVANFCZYWJDZXLBHHJ47NQY6V7D36DEFPGA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304060410 (377h2m23.230355106s) [noise=true , address=104.36.82.41:4768 , piece_id=BGMF33OKUZSZKGIHJYI3RNXEBDE54TX67DYLJ7JCZMRNK6ZAU22Q, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304407993 (377h2m23.230702685s) [noise=true , address=77.90.31.10:4525 , piece_id=GPVSIZIBZWTUCP5RFYVPFYIYIL2LWU5PU3UASQVRQLY7SSSFMQBA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304548115 (377h2m23.230842812s) [noise=true , address=104.36.82.23:4464 , piece_id=JCJ4MGEZIIZ7YCTM6QXC5ZJ7PI4OEJBOMHZYBU57D2SNGWJWB5CQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304795164 (377h2m23.231089862s) [noise=true , address=77.90.31.18:4838 , piece_id=TGWYXA3AF7LDG4KPVCRNEYFDADAG6TQ5DAX7JJVW5BLVVVNECILA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305012510 (377h2m23.231307211s) [noise=true , address=104.36.82.34:4554 , piece_id=57K5SDV3YEGLVMSJGSLXNKEWOLETXE6CWBM2NPVCNKLFHPVQCJBQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305113737 (377h2m23.231408427s) [noise=true , address=104.36.82.4:4834 , piece_id=YLYIASUPNRTK7KOMRYLN7OQLQBPWC7WFGCPDGUVIOEQJWQQ7A7GA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305383755 (377h2m23.231678434s) [noise=true , address=104.36.82.38:4698 , piece_id=TSOXJNQEWODCWFEITH4IYKZI7A75UZEXRQPIQO7LMP6LT3WLNXUA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305483822 (377h2m23.231778517s) [noise=true , address=104.36.82.42:4830 , piece_id=ENYQTUCIF7OWQPCXTYS24OC5QT63ZJJXOY652GLIG7OQPJEQCGGA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305954019 (377h2m23.232248718s) [noise=true , address=77.90.31.21:4907 , piece_id=2EUWATHUQNZ77BJV65ZFKR5WL4D3AUURLBQ32XR2IVBA7P4TITOQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.306425517 (377h2m23.232720213s) [noise=true , address=104.36.82.22:4541 , piece_id=XN4HDFDNZ4YUJTDPOTMMDFX4FE2YVZDGCIMHEMJGWXRPNHQ6EU7A, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.375470790 (377h2m23.301765481s) [upload=301954]: Next returned nil segment
Jul 9 02:13:56.375489934 (377h2m23.30178463s) [upload=301954]: Waiting for error group managing segments...
Jul 9 02:13:56.381456859 (377h2m23.307751539s) [noise=true , address=77.90.31.10:4525 , piece_id=GPVSIZIBZWTUCP5RFYVPFYIYIL2LWU5PU3UASQVRQLY7SSSFMQBA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.381956907 (377h2m23.308251586s) [noise=true , address=77.90.31.14:4689 , piece_id=LW7OT74AJCCC3TEKY6OZI553YK6YQFQ6C7PORQB53E7FUS24KQKQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.382837088 (377h2m23.309131761s) [noise=true , address=77.90.31.21:4898 , piece_id=75XXVNT4EV4D5UTX7O5XFL44IZOYY5LZSI2VPFQZPRERL6W5M23A, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.393489125 (377h2m23.319783812s) [noise=true , address=77.90.31.21:4907 , piece_id=2EUWATHUQNZ77BJV65ZFKR5WL4D3AUURLBQ32XR2IVBA7P4TITOQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.404692801 (377h2m23.330987481s) [noise=true , address=77.90.31.16:4739 , piece_id=H6UCV6RURPXKC2H3YGAT2OXV2VZMW2DWIRG6TFD6CVXKOVMDP2VQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.404986115 (377h2m23.331280795s) [noise=true , address=77.90.31.18:4838 , piece_id=TGWYXA3AF7LDG4KPVCRNEYFDADAG6TQ5DAX7JJVW5BLVVVNECILA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.433785086 (377h2m23.360079766s) [noise=true , address=77.90.31.13:4640 , piece_id=TPPN75DX3UCBM346CSVJN3GSOA47QV4FVPYSSAHWCJYEM42OCVLQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.598499747 (377h2m23.524794427s) [noise=true , address=79.127.201.215:4339 , piece_id=XDCTYE2SVI6RAPCXTRVANFCZYWJDZXLBHHJ47NQY6V7D36DEFPGA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:11.107785584 (377h2m38.034080273s) [noise=true , address=104.36.82.22:4541 , piece_id=XN4HDFDNZ4YUJTDPOTMMDFX4FE2YVZDGCIMHEMJGWXRPNHQ6EU7A, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:12.515567029 (377h2m39.441861713s) [noise=true , address=104.36.82.17:4345 , piece_id=YI27O5WLBYO3TPCX4MVUATFORY6U3EQMDKQHGKRHIZMTG7WGFB2A, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:12.923090587 (377h2m39.849385267s) [noise=true , address=104.36.82.8:4986 , piece_id=KPY5JVLSAEB5GUKUJJSIF5M6IIHTJISEYYAXCGUSJ7OLI5QCIBNQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:13.453172659 (377h2m40.379467346s) [noise=true , address=104.36.82.38:4698 , piece_id=TSOXJNQEWODCWFEITH4IYKZI7A75UZEXRQPIQO7LMP6LT3WLNXUA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:14.416524856 (377h2m41.342819534s) [noise=true , address=104.36.82.34:4554 , piece_id=57K5SDV3YEGLVMSJGSLXNKEWOLETXE6CWBM2NPVCNKLFHPVQCJBQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:14.846730699 (377h2m41.773025379s) [noise=true , address=104.36.82.23:4464 , piece_id=JCJ4MGEZIIZ7YCTM6QXC5ZJ7PI4OEJBOMHZYBU57D2SNGWJWB5CQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:14.846763777 (377h2m41.773058463s) [seg_pos={0 0}, upload=301954]: Segment reached optimal threshold of 14 pieces.
Jul 9 02:14:14.846894905 (377h2m41.773189582s) [noise=true , address=104.36.82.36:4644 , piece_id=K2G3MAPD7TWYZPKNHQBMEPJRIQDLE75FYVPAZFX4I3GJ4IUAA7XQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1oYHbb73ACitbfkkP5P6cmRyTt6kmaS1y49G4vbsvTvVDwByhT): protocol: expected piece hash; context canceled; EOF
Jul 9 02:14:14.846948435 (377h2m41.773243131s) [noise=true , address=104.36.82.4:4834 , piece_id=YLYIASUPNRTK7KOMRYLN7OQLQBPWC7WFGCPDGUVIOEQJWQQ7A7GA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Z241DCYA5Wig2TdEogxTpY6FzVmFbnKtmHcTwxLxfcZafgooy): protocol: expected piece hash; context canceled; EOF
Jul 9 02:14:14.847094561 (377h2m41.773389249s) [noise=true , address=104.36.82.42:4830 , piece_id=ENYQTUCIF7OWQPCXTYS24OC5QT63ZJJXOY652GLIG7OQPJEQCGGA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:142ocSHHtv8BpSebXFNk5qk1Y23a8pbFAZqE3cq6i3RMX3dPw9): protocol: expected piece hash; context canceled; EOF
Jul 9 02:14:14.847127825 (377h2m41.773422511s) [noise=true , address=104.36.82.41:4768 , piece_id=BGMF33OKUZSZKGIHJYI3RNXEBDE54TX67DYLJ7JCZMRNK6ZAU22Q, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: write tcp 159.223.143.39:43796->104.36.82.41:4768: use of closed network connection; write tcp 159.223.143.39:43796->104.36.82.41:4768: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:43796->104.36.82.41:4768: use of closed network connection
Jul 9 02:14:14.847142999 (377h2m41.773437687s) [seg_pos={0 0}, upload=301954]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jul 9 02:14:14.847204962 (377h2m41.773499652s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:14:14.847209974 (377h2m41.773504671s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:14:14.909581364 (377h2m41.83587606s) [upload=301954]: Done uploading segments...
Jul 9 02:14:14.909605483 (377h2m41.835900172s) [upload=301954]: Upload finished. err: <nil>
Duration: 29.09025985s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
38 out of 60 is failed (threshold=3) [.XXX.X.X.XXXXXXX..XX....XX.XXX.XXX.XX.XXXXXX.XXXXX.XX.....X.] duration:
51.125751586s
FAIL (2025-07-09 02:35:30 UTC)
operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 51.125809874s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
21 out of 60 is failed (threshold=3) [X....X....X.........X....X.X..XX.XX..X.X.X.....XXX.XX.X...XX] duration:
30.00194004s
FAIL (2025-07-09 02:35:19 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 30.001958623s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [...................................X........................] duration:
28.699310103s
FAIL (2025-07-09 02:35:10 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
3:56.263910643 (377h2m23.19020535s) [upload=301954]: Got next segment. Inline: false
Jul 9 02:13:56.263954358 (377h2m23.190249034s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:13:56.263959891 (377h2m23.190254572s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:13:56.302204249 (377h2m23.228498944s) [seg_pos={0 0}, upload=301954]: Begin upload segment...
Jul 9 02:13:56.302291587 (377h2m23.22858628s) [seg_pos={0 0}, upload=301954]: Done begin upload segment.
Jul 9 02:13:56.302338947 (377h2m23.228633682s) [noise=true , address=77.90.31.14:4689 , piece_id=LW7OT74AJCCC3TEKY6OZI553YK6YQFQ6C7PORQB53E7FUS24KQKQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.302352599 (377h2m23.228647272s) [noise=true , address=104.36.82.8:4986 , piece_id=KPY5JVLSAEB5GUKUJJSIF5M6IIHTJISEYYAXCGUSJ7OLI5QCIBNQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.302450837 (377h2m23.228745531s) [noise=true , address=77.90.31.16:4739 , piece_id=H6UCV6RURPXKC2H3YGAT2OXV2VZMW2DWIRG6TFD6CVXKOVMDP2VQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.302958774 (377h2m23.229253667s) [noise=true , address=77.90.31.13:4640 , piece_id=TPPN75DX3UCBM346CSVJN3GSOA47QV4FVPYSSAHWCJYEM42OCVLQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303069086 (377h2m23.229363782s) [noise=true , address=104.36.82.17:4345 , piece_id=YI27O5WLBYO3TPCX4MVUATFORY6U3EQMDKQHGKRHIZMTG7WGFB2A, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303495994 (377h2m23.22979069s) [noise=true , address=104.36.82.36:4644 , piece_id=K2G3MAPD7TWYZPKNHQBMEPJRIQDLE75FYVPAZFX4I3GJ4IUAA7XQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303575840 (377h2m23.229870536s) [noise=true , address=77.90.31.21:4898 , piece_id=75XXVNT4EV4D5UTX7O5XFL44IZOYY5LZSI2VPFQZPRERL6W5M23A, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303949062 (377h2m23.230245641s) [noise=true , address=79.127.201.215:4339 , piece_id=XDCTYE2SVI6RAPCXTRVANFCZYWJDZXLBHHJ47NQY6V7D36DEFPGA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304060410 (377h2m23.230355106s) [noise=true , address=104.36.82.41:4768 , piece_id=BGMF33OKUZSZKGIHJYI3RNXEBDE54TX67DYLJ7JCZMRNK6ZAU22Q, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304407993 (377h2m23.230702685s) [noise=true , address=77.90.31.10:4525 , piece_id=GPVSIZIBZWTUCP5RFYVPFYIYIL2LWU5PU3UASQVRQLY7SSSFMQBA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304548115 (377h2m23.230842812s) [noise=true , address=104.36.82.23:4464 , piece_id=JCJ4MGEZIIZ7YCTM6QXC5ZJ7PI4OEJBOMHZYBU57D2SNGWJWB5CQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304795164 (377h2m23.231089862s) [noise=true , address=77.90.31.18:4838 , piece_id=TGWYXA3AF7LDG4KPVCRNEYFDADAG6TQ5DAX7JJVW5BLVVVNECILA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305012510 (377h2m23.231307211s) [noise=true , address=104.36.82.34:4554 , piece_id=57K5SDV3YEGLVMSJGSLXNKEWOLETXE6CWBM2NPVCNKLFHPVQCJBQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305113737 (377h2m23.231408427s) [noise=true , address=104.36.82.4:4834 , piece_id=YLYIASUPNRTK7KOMRYLN7OQLQBPWC7WFGCPDGUVIOEQJWQQ7A7GA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305383755 (377h2m23.231678434s) [noise=true , address=104.36.82.38:4698 , piece_id=TSOXJNQEWODCWFEITH4IYKZI7A75UZEXRQPIQO7LMP6LT3WLNXUA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305483822 (377h2m23.231778517s) [noise=true , address=104.36.82.42:4830 , piece_id=ENYQTUCIF7OWQPCXTYS24OC5QT63ZJJXOY652GLIG7OQPJEQCGGA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305954019 (377h2m23.232248718s) [noise=true , address=77.90.31.21:4907 , piece_id=2EUWATHUQNZ77BJV65ZFKR5WL4D3AUURLBQ32XR2IVBA7P4TITOQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.306425517 (377h2m23.232720213s) [noise=true , address=104.36.82.22:4541 , piece_id=XN4HDFDNZ4YUJTDPOTMMDFX4FE2YVZDGCIMHEMJGWXRPNHQ6EU7A, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.375470790 (377h2m23.301765481s) [upload=301954]: Next returned nil segment
Jul 9 02:13:56.375489934 (377h2m23.30178463s) [upload=301954]: Waiting for error group managing segments...
Jul 9 02:13:56.381456859 (377h2m23.307751539s) [noise=true , address=77.90.31.10:4525 , piece_id=GPVSIZIBZWTUCP5RFYVPFYIYIL2LWU5PU3UASQVRQLY7SSSFMQBA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.381956907 (377h2m23.308251586s) [noise=true , address=77.90.31.14:4689 , piece_id=LW7OT74AJCCC3TEKY6OZI553YK6YQFQ6C7PORQB53E7FUS24KQKQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.382837088 (377h2m23.309131761s) [noise=true , address=77.90.31.21:4898 , piece_id=75XXVNT4EV4D5UTX7O5XFL44IZOYY5LZSI2VPFQZPRERL6W5M23A, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.393489125 (377h2m23.319783812s) [noise=true , address=77.90.31.21:4907 , piece_id=2EUWATHUQNZ77BJV65ZFKR5WL4D3AUURLBQ32XR2IVBA7P4TITOQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.404692801 (377h2m23.330987481s) [noise=true , address=77.90.31.16:4739 , piece_id=H6UCV6RURPXKC2H3YGAT2OXV2VZMW2DWIRG6TFD6CVXKOVMDP2VQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.404986115 (377h2m23.331280795s) [noise=true , address=77.90.31.18:4838 , piece_id=TGWYXA3AF7LDG4KPVCRNEYFDADAG6TQ5DAX7JJVW5BLVVVNECILA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.433785086 (377h2m23.360079766s) [noise=true , address=77.90.31.13:4640 , piece_id=TPPN75DX3UCBM346CSVJN3GSOA47QV4FVPYSSAHWCJYEM42OCVLQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.598499747 (377h2m23.524794427s) [noise=true , address=79.127.201.215:4339 , piece_id=XDCTYE2SVI6RAPCXTRVANFCZYWJDZXLBHHJ47NQY6V7D36DEFPGA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:11.107785584 (377h2m38.034080273s) [noise=true , address=104.36.82.22:4541 , piece_id=XN4HDFDNZ4YUJTDPOTMMDFX4FE2YVZDGCIMHEMJGWXRPNHQ6EU7A, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:12.515567029 (377h2m39.441861713s) [noise=true , address=104.36.82.17:4345 , piece_id=YI27O5WLBYO3TPCX4MVUATFORY6U3EQMDKQHGKRHIZMTG7WGFB2A, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:12.923090587 (377h2m39.849385267s) [noise=true , address=104.36.82.8:4986 , piece_id=KPY5JVLSAEB5GUKUJJSIF5M6IIHTJISEYYAXCGUSJ7OLI5QCIBNQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:13.453172659 (377h2m40.379467346s) [noise=true , address=104.36.82.38:4698 , piece_id=TSOXJNQEWODCWFEITH4IYKZI7A75UZEXRQPIQO7LMP6LT3WLNXUA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:14.416524856 (377h2m41.342819534s) [noise=true , address=104.36.82.34:4554 , piece_id=57K5SDV3YEGLVMSJGSLXNKEWOLETXE6CWBM2NPVCNKLFHPVQCJBQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:14.846730699 (377h2m41.773025379s) [noise=true , address=104.36.82.23:4464 , piece_id=JCJ4MGEZIIZ7YCTM6QXC5ZJ7PI4OEJBOMHZYBU57D2SNGWJWB5CQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:14.846763777 (377h2m41.773058463s) [seg_pos={0 0}, upload=301954]: Segment reached optimal threshold of 14 pieces.
Jul 9 02:14:14.846894905 (377h2m41.773189582s) [noise=true , address=104.36.82.36:4644 , piece_id=K2G3MAPD7TWYZPKNHQBMEPJRIQDLE75FYVPAZFX4I3GJ4IUAA7XQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1oYHbb73ACitbfkkP5P6cmRyTt6kmaS1y49G4vbsvTvVDwByhT): protocol: expected piece hash; context canceled; EOF
Jul 9 02:14:14.846948435 (377h2m41.773243131s) [noise=true , address=104.36.82.4:4834 , piece_id=YLYIASUPNRTK7KOMRYLN7OQLQBPWC7WFGCPDGUVIOEQJWQQ7A7GA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Z241DCYA5Wig2TdEogxTpY6FzVmFbnKtmHcTwxLxfcZafgooy): protocol: expected piece hash; context canceled; EOF
Jul 9 02:14:14.847094561 (377h2m41.773389249s) [noise=true , address=104.36.82.42:4830 , piece_id=ENYQTUCIF7OWQPCXTYS24OC5QT63ZJJXOY652GLIG7OQPJEQCGGA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:142ocSHHtv8BpSebXFNk5qk1Y23a8pbFAZqE3cq6i3RMX3dPw9): protocol: expected piece hash; context canceled; EOF
Jul 9 02:14:14.847127825 (377h2m41.773422511s) [noise=true , address=104.36.82.41:4768 , piece_id=BGMF33OKUZSZKGIHJYI3RNXEBDE54TX67DYLJ7JCZMRNK6ZAU22Q, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: write tcp 159.223.143.39:43796->104.36.82.41:4768: use of closed network connection; write tcp 159.223.143.39:43796->104.36.82.41:4768: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:43796->104.36.82.41:4768: use of closed network connection
Jul 9 02:14:14.847142999 (377h2m41.773437687s) [seg_pos={0 0}, upload=301954]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jul 9 02:14:14.847204962 (377h2m41.773499652s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:14:14.847209974 (377h2m41.773504671s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:14:14.909581364 (377h2m41.83587606s) [upload=301954]: Done uploading segments...
Jul 9 02:14:14.909605483 (377h2m41.835900172s) [upload=301954]: Upload finished. err: <nil>
Duration: 28.699342739s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
4 out of 60 is failed (threshold=3) [..............X.....X...X....................X..............] duration:
442.927615ms
FAIL (2025-07-09 02:34:52 UTC)
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
02:25:27.135824454 (377h13m54.062119141s) [noise=true , address=104.36.82.12:4150 , piece_id=DRG5SBA3EE4KVQPEQOJZ6SS537SFWYFUYQXOAQMWYTT6FPVSJIPQ, seg_pos={0 0}, upload=302099]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: context canceled; context canceled
Jul 9 02:25:27.135860079 (377h13m54.062154769s) [seg_pos={0 0}, upload=302099]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 02:25:27.135957283 (377h13m54.062251971s) [upload=302099]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:25:27.135964211 (377h13m54.062258897s) [upload=302099]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:25:27.199410713 (377h13m54.125705389s) [upload=302099]: Done uploading segments...
Jul 9 02:25:27.199427901 (377h13m54.125722577s) [upload=302099]: Upload finished. err: <nil>
Jul 9 02:25:57.289077655 (377h14m24.215372329s) [upload=302109]: Starting upload
Jul 9 02:25:57.289186750 (377h14m24.215481424s) [upload=302109]: Done starting upload
Jul 9 02:25:57.289444851 (377h14m24.215739548s) [upload=302109]: Uploading segments...
Jul 9 02:25:57.289623846 (377h14m24.215918535s) [upload=302109]: Got next segment. Inline: false
Jul 9 02:25:57.289696741 (377h14m24.215991432s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:25:57.289703375 (377h14m24.215998067s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:25:57.318313152 (377h14m24.244607838s) [seg_pos={0 0}, upload=302109]: Begin upload segment...
Jul 9 02:25:57.318363286 (377h14m24.244657964s) [seg_pos={0 0}, upload=302109]: Done begin upload segment.
Jul 9 02:25:57.318386772 (377h14m24.244681463s) [noise=true , address=77.90.31.11:4553 , piece_id=TYW7NBJWACIT7OQS7A4WGXUPR3MB2FKAN2OMGAZJERB4X2DLAO2Q, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318412127 (377h14m24.244706825s) [noise=true , address=77.90.31.10:4518 , piece_id=VJ4HWM2X4VSRSUCQL5QLMBYPFBYV4JYS5ZXCBCB6MTMB76EHZFGQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318822149 (377h14m24.245116841s) [noise=true , address=45.91.116.9:4263 , piece_id=5YILAJ5BZBGOURSZE343ST4RMOTWM6ZOB3EEQQWL5WUNYWOGSAPA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318965862 (377h14m24.24526056s) [noise=true , address=45.91.116.8:4045 , piece_id=KM6ZYTHR6JGTARKZQZYIESPKBEU5H2HQFA7OARIKD2ZXDHPFSMGA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.319187516 (377h14m24.245482203s) [noise=true , address=104.36.82.7:4909 , piece_id=ZRHLWDQ5KIDRWPJ5FXAANQTWERKXCR2PPEGN442IOSYSRFAG7VZA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.319523815 (377h14m24.245818512s) [noise=true , address=45.91.116.14:4191 , piece_id=GTR6T4OTGZTEZ76FWQZ46OCH5WB3O5X27EN4AV33ZJ7D5HX4535A, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.320556632 (377h14m24.24685133s) [noise=true , address=104.36.82.2:4732 , piece_id=DJIGHJMJRAULH2L4YT2WGRXPRQJMNKZG5LICHPSKOCHUL6QDV6BQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.321019141 (377h14m24.247313835s) [noise=true , address=77.90.31.17:4770 , piece_id=V36UBBIYOFJC6KFVVBXEVT4QBUIWN257D3YSDDRAGZVXH7ZXHTBA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.322591797 (377h14m24.248886493s) [noise=true , address=77.90.31.13:4627 , piece_id=HULEKFVZNDNKS4SWWGTJUHYZWWYPDCER5W4GFCVLLDL65PAHULFQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323120451 (377h14m24.249415138s) [noise=true , address=45.91.116.4:4939 , piece_id=ANA3EZU7OZWELNG4QEMTEAYZY5H3C3H4IJFT4XKPUB7WIF44FEVQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323544591 (377h14m24.249839288s) [noise=true , address=45.91.116.7:4011 , piece_id=VDOJRN6ACK4LO44BQTULO6BB3CLAVQI7YIC6XLR32CZV4UJSPE6Q, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323950189 (377h14m24.250244889s) [noise=true , address=104.36.82.10:4078 , piece_id=ZFVUQAEV6DDFLPKCZA2XJ2BZHMMM745LGB4JRZJZQPG72WECGMDQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.326585810 (377h14m24.252880507s) [noise=true , address=45.91.116.5:4297 , piece_id=MSDWDLY5PJHQVI5565EKANJVSTMZFEUD2OTYZ52C65XQUICNMC3A, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.327871214 (377h14m24.254165904s) [noise=true , address=104.36.82.6:4875 , piece_id=C275F2JPVAF52E34CEHT5QXCHUFGIQZ7XGMOAOUOT4RA7YOHRQLQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.328605288 (377h14m24.254899986s) [noise=true , address=104.36.82.5:4838 , piece_id=WK3ALBOEMWWTVEA4FKCS352SZ6YYB35LELURNWXQZK32K4HYDRNA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.335409087 (377h14m24.261703784s) [noise=true , address=45.91.116.6:4973 , piece_id=Z7HQQ2O5QUSJKT4EMIDMEI5PIAEHNVRYX7MALUR2GQULRFIBUOKA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.475344637 (377h14m24.401639326s) [upload=302109]: Next returned nil segment
Jul 9 02:25:57.475376181 (377h14m24.401670879s) [upload=302109]: Waiting for error group managing segments...
Jul 9 02:25:57.477269278 (377h14m24.403563958s) [noise=true , address=45.91.116.7:4011 , piece_id=VDOJRN6ACK4LO44BQTULO6BB3CLAVQI7YIC6XLR32CZV4UJSPE6Q, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.477736179 (377h14m24.40403086s) [noise=true , address=45.91.116.9:4263 , piece_id=5YILAJ5BZBGOURSZE343ST4RMOTWM6ZOB3EEQQWL5WUNYWOGSAPA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.478396652 (377h14m24.404691332s) [noise=true , address=77.90.31.10:4518 , piece_id=VJ4HWM2X4VSRSUCQL5QLMBYPFBYV4JYS5ZXCBCB6MTMB76EHZFGQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.478919538 (377h14m24.405214224s) [noise=true , address=45.91.116.14:4191 , piece_id=GTR6T4OTGZTEZ76FWQZ46OCH5WB3O5X27EN4AV33ZJ7D5HX4535A, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481426716 (377h14m24.407721396s) [noise=true , address=45.91.116.4:4939 , piece_id=ANA3EZU7OZWELNG4QEMTEAYZY5H3C3H4IJFT4XKPUB7WIF44FEVQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481718373 (377h14m24.408013051s) [noise=true , address=45.91.116.5:4297 , piece_id=MSDWDLY5PJHQVI5565EKANJVSTMZFEUD2OTYZ52C65XQUICNMC3A, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481930356 (377h14m24.408225033s) [noise=true , address=77.90.31.17:4770 , piece_id=V36UBBIYOFJC6KFVVBXEVT4QBUIWN257D3YSDDRAGZVXH7ZXHTBA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.482376147 (377h14m24.408670821s) [noise=true , address=45.91.116.8:4045 , piece_id=KM6ZYTHR6JGTARKZQZYIESPKBEU5H2HQFA7OARIKD2ZXDHPFSMGA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.482439192 (377h14m24.408733868s) [noise=true , address=45.91.116.6:4973 , piece_id=Z7HQQ2O5QUSJKT4EMIDMEI5PIAEHNVRYX7MALUR2GQULRFIBUOKA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.483047970 (377h14m24.409342644s) [noise=true , address=77.90.31.13:4627 , piece_id=HULEKFVZNDNKS4SWWGTJUHYZWWYPDCER5W4GFCVLLDL65PAHULFQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.497788632 (377h14m24.424083312s) [noise=true , address=77.90.31.11:4553 , piece_id=TYW7NBJWACIT7OQS7A4WGXUPR3MB2FKAN2OMGAZJERB4X2DLAO2Q, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:16.688921171 (377h14m43.615215851s) [noise=true , address=104.36.82.6:4875 , piece_id=C275F2JPVAF52E34CEHT5QXCHUFGIQZ7XGMOAOUOT4RA7YOHRQLQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:27.177893955 (377h14m54.104188635s) [noise=true , address=104.36.82.2:4732 , piece_id=DJIGHJMJRAULH2L4YT2WGRXPRQJMNKZG5LICHPSKOCHUL6QDV6BQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:27.177985584 (377h14m54.104280281s) [seg_pos={0 0}, upload=302109]: Segment reached optimal threshold of 13 pieces.
Jul 9 02:26:27.178189789 (377h14m54.104484461s) [noise=true , address=104.36.82.5:4838 , piece_id=WK3ALBOEMWWTVEA4FKCS352SZ6YYB35LELURNWXQZK32K4HYDRNA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178252042 (377h14m54.104546733s) [noise=true , address=104.36.82.7:4909 , piece_id=ZRHLWDQ5KIDRWPJ5FXAANQTWERKXCR2PPEGN442IOSYSRFAG7VZA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178273922 (377h14m54.104568608s) [noise=true , address=104.36.82.10:4078 , piece_id=ZFVUQAEV6DDFLPKCZA2XJ2BZHMMM745LGB4JRZJZQPG72WECGMDQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178305983 (377h14m54.104600682s) [seg_pos={0 0}, upload=302109]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 02:26:27.178428768 (377h14m54.104723465s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:26:27.178455810 (377h14m54.104750501s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:26:27.232166184 (377h14m54.158460883s) [upload=302109]: Done uploading segments...
Jul 9 02:26:27.232188444 (377h14m54.158483123s) [upload=302109]: Upload finished. err: <nil>
Duration: 442.957168ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
20 out of 60 is failed (threshold=3) [.X....X....X.........X....X.X..XX.XX..X.X.X.....XXX.XX.X...X] duration:
30.001400911s
FAIL (2025-07-09 02:34:49 UTC)
orbiter warning: context deadline exceeded
Duration: 30.001441993s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
39 out of 60 is failed (threshold=3) [X.XXX.X.X.XXXXXXX..XX....XX.XXX.XXX.XX.XXXXXX.XXXXX.XX.....X] duration:
1m0.000623705s
FAIL (2025-07-09 02:34:39 UTC)
operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1m0.000674189s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [....................................X.......................] duration:
21.105704115s
FAIL (2025-07-09 02:34:14 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
3:56.263910643 (377h2m23.19020535s) [upload=301954]: Got next segment. Inline: false
Jul 9 02:13:56.263954358 (377h2m23.190249034s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:13:56.263959891 (377h2m23.190254572s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:13:56.302204249 (377h2m23.228498944s) [seg_pos={0 0}, upload=301954]: Begin upload segment...
Jul 9 02:13:56.302291587 (377h2m23.22858628s) [seg_pos={0 0}, upload=301954]: Done begin upload segment.
Jul 9 02:13:56.302338947 (377h2m23.228633682s) [noise=true , address=77.90.31.14:4689 , piece_id=LW7OT74AJCCC3TEKY6OZI553YK6YQFQ6C7PORQB53E7FUS24KQKQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.302352599 (377h2m23.228647272s) [noise=true , address=104.36.82.8:4986 , piece_id=KPY5JVLSAEB5GUKUJJSIF5M6IIHTJISEYYAXCGUSJ7OLI5QCIBNQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.302450837 (377h2m23.228745531s) [noise=true , address=77.90.31.16:4739 , piece_id=H6UCV6RURPXKC2H3YGAT2OXV2VZMW2DWIRG6TFD6CVXKOVMDP2VQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.302958774 (377h2m23.229253667s) [noise=true , address=77.90.31.13:4640 , piece_id=TPPN75DX3UCBM346CSVJN3GSOA47QV4FVPYSSAHWCJYEM42OCVLQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303069086 (377h2m23.229363782s) [noise=true , address=104.36.82.17:4345 , piece_id=YI27O5WLBYO3TPCX4MVUATFORY6U3EQMDKQHGKRHIZMTG7WGFB2A, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303495994 (377h2m23.22979069s) [noise=true , address=104.36.82.36:4644 , piece_id=K2G3MAPD7TWYZPKNHQBMEPJRIQDLE75FYVPAZFX4I3GJ4IUAA7XQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303575840 (377h2m23.229870536s) [noise=true , address=77.90.31.21:4898 , piece_id=75XXVNT4EV4D5UTX7O5XFL44IZOYY5LZSI2VPFQZPRERL6W5M23A, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303949062 (377h2m23.230245641s) [noise=true , address=79.127.201.215:4339 , piece_id=XDCTYE2SVI6RAPCXTRVANFCZYWJDZXLBHHJ47NQY6V7D36DEFPGA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304060410 (377h2m23.230355106s) [noise=true , address=104.36.82.41:4768 , piece_id=BGMF33OKUZSZKGIHJYI3RNXEBDE54TX67DYLJ7JCZMRNK6ZAU22Q, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304407993 (377h2m23.230702685s) [noise=true , address=77.90.31.10:4525 , piece_id=GPVSIZIBZWTUCP5RFYVPFYIYIL2LWU5PU3UASQVRQLY7SSSFMQBA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304548115 (377h2m23.230842812s) [noise=true , address=104.36.82.23:4464 , piece_id=JCJ4MGEZIIZ7YCTM6QXC5ZJ7PI4OEJBOMHZYBU57D2SNGWJWB5CQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304795164 (377h2m23.231089862s) [noise=true , address=77.90.31.18:4838 , piece_id=TGWYXA3AF7LDG4KPVCRNEYFDADAG6TQ5DAX7JJVW5BLVVVNECILA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305012510 (377h2m23.231307211s) [noise=true , address=104.36.82.34:4554 , piece_id=57K5SDV3YEGLVMSJGSLXNKEWOLETXE6CWBM2NPVCNKLFHPVQCJBQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305113737 (377h2m23.231408427s) [noise=true , address=104.36.82.4:4834 , piece_id=YLYIASUPNRTK7KOMRYLN7OQLQBPWC7WFGCPDGUVIOEQJWQQ7A7GA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305383755 (377h2m23.231678434s) [noise=true , address=104.36.82.38:4698 , piece_id=TSOXJNQEWODCWFEITH4IYKZI7A75UZEXRQPIQO7LMP6LT3WLNXUA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305483822 (377h2m23.231778517s) [noise=true , address=104.36.82.42:4830 , piece_id=ENYQTUCIF7OWQPCXTYS24OC5QT63ZJJXOY652GLIG7OQPJEQCGGA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305954019 (377h2m23.232248718s) [noise=true , address=77.90.31.21:4907 , piece_id=2EUWATHUQNZ77BJV65ZFKR5WL4D3AUURLBQ32XR2IVBA7P4TITOQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.306425517 (377h2m23.232720213s) [noise=true , address=104.36.82.22:4541 , piece_id=XN4HDFDNZ4YUJTDPOTMMDFX4FE2YVZDGCIMHEMJGWXRPNHQ6EU7A, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.375470790 (377h2m23.301765481s) [upload=301954]: Next returned nil segment
Jul 9 02:13:56.375489934 (377h2m23.30178463s) [upload=301954]: Waiting for error group managing segments...
Jul 9 02:13:56.381456859 (377h2m23.307751539s) [noise=true , address=77.90.31.10:4525 , piece_id=GPVSIZIBZWTUCP5RFYVPFYIYIL2LWU5PU3UASQVRQLY7SSSFMQBA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.381956907 (377h2m23.308251586s) [noise=true , address=77.90.31.14:4689 , piece_id=LW7OT74AJCCC3TEKY6OZI553YK6YQFQ6C7PORQB53E7FUS24KQKQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.382837088 (377h2m23.309131761s) [noise=true , address=77.90.31.21:4898 , piece_id=75XXVNT4EV4D5UTX7O5XFL44IZOYY5LZSI2VPFQZPRERL6W5M23A, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.393489125 (377h2m23.319783812s) [noise=true , address=77.90.31.21:4907 , piece_id=2EUWATHUQNZ77BJV65ZFKR5WL4D3AUURLBQ32XR2IVBA7P4TITOQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.404692801 (377h2m23.330987481s) [noise=true , address=77.90.31.16:4739 , piece_id=H6UCV6RURPXKC2H3YGAT2OXV2VZMW2DWIRG6TFD6CVXKOVMDP2VQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.404986115 (377h2m23.331280795s) [noise=true , address=77.90.31.18:4838 , piece_id=TGWYXA3AF7LDG4KPVCRNEYFDADAG6TQ5DAX7JJVW5BLVVVNECILA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.433785086 (377h2m23.360079766s) [noise=true , address=77.90.31.13:4640 , piece_id=TPPN75DX3UCBM346CSVJN3GSOA47QV4FVPYSSAHWCJYEM42OCVLQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.598499747 (377h2m23.524794427s) [noise=true , address=79.127.201.215:4339 , piece_id=XDCTYE2SVI6RAPCXTRVANFCZYWJDZXLBHHJ47NQY6V7D36DEFPGA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:11.107785584 (377h2m38.034080273s) [noise=true , address=104.36.82.22:4541 , piece_id=XN4HDFDNZ4YUJTDPOTMMDFX4FE2YVZDGCIMHEMJGWXRPNHQ6EU7A, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:12.515567029 (377h2m39.441861713s) [noise=true , address=104.36.82.17:4345 , piece_id=YI27O5WLBYO3TPCX4MVUATFORY6U3EQMDKQHGKRHIZMTG7WGFB2A, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:12.923090587 (377h2m39.849385267s) [noise=true , address=104.36.82.8:4986 , piece_id=KPY5JVLSAEB5GUKUJJSIF5M6IIHTJISEYYAXCGUSJ7OLI5QCIBNQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:13.453172659 (377h2m40.379467346s) [noise=true , address=104.36.82.38:4698 , piece_id=TSOXJNQEWODCWFEITH4IYKZI7A75UZEXRQPIQO7LMP6LT3WLNXUA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:14.416524856 (377h2m41.342819534s) [noise=true , address=104.36.82.34:4554 , piece_id=57K5SDV3YEGLVMSJGSLXNKEWOLETXE6CWBM2NPVCNKLFHPVQCJBQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:14.846730699 (377h2m41.773025379s) [noise=true , address=104.36.82.23:4464 , piece_id=JCJ4MGEZIIZ7YCTM6QXC5ZJ7PI4OEJBOMHZYBU57D2SNGWJWB5CQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:14.846763777 (377h2m41.773058463s) [seg_pos={0 0}, upload=301954]: Segment reached optimal threshold of 14 pieces.
Jul 9 02:14:14.846894905 (377h2m41.773189582s) [noise=true , address=104.36.82.36:4644 , piece_id=K2G3MAPD7TWYZPKNHQBMEPJRIQDLE75FYVPAZFX4I3GJ4IUAA7XQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1oYHbb73ACitbfkkP5P6cmRyTt6kmaS1y49G4vbsvTvVDwByhT): protocol: expected piece hash; context canceled; EOF
Jul 9 02:14:14.846948435 (377h2m41.773243131s) [noise=true , address=104.36.82.4:4834 , piece_id=YLYIASUPNRTK7KOMRYLN7OQLQBPWC7WFGCPDGUVIOEQJWQQ7A7GA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Z241DCYA5Wig2TdEogxTpY6FzVmFbnKtmHcTwxLxfcZafgooy): protocol: expected piece hash; context canceled; EOF
Jul 9 02:14:14.847094561 (377h2m41.773389249s) [noise=true , address=104.36.82.42:4830 , piece_id=ENYQTUCIF7OWQPCXTYS24OC5QT63ZJJXOY652GLIG7OQPJEQCGGA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:142ocSHHtv8BpSebXFNk5qk1Y23a8pbFAZqE3cq6i3RMX3dPw9): protocol: expected piece hash; context canceled; EOF
Jul 9 02:14:14.847127825 (377h2m41.773422511s) [noise=true , address=104.36.82.41:4768 , piece_id=BGMF33OKUZSZKGIHJYI3RNXEBDE54TX67DYLJ7JCZMRNK6ZAU22Q, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: write tcp 159.223.143.39:43796->104.36.82.41:4768: use of closed network connection; write tcp 159.223.143.39:43796->104.36.82.41:4768: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:43796->104.36.82.41:4768: use of closed network connection
Jul 9 02:14:14.847142999 (377h2m41.773437687s) [seg_pos={0 0}, upload=301954]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jul 9 02:14:14.847204962 (377h2m41.773499652s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:14:14.847209974 (377h2m41.773504671s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:14:14.909581364 (377h2m41.83587606s) [upload=301954]: Done uploading segments...
Jul 9 02:14:14.909605483 (377h2m41.835900172s) [upload=301954]: Upload finished. err: <nil>
Duration: 21.105733109s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
4 out of 60 is failed (threshold=3) [...............X.....X...X....................X.............] duration:
469.915431ms
FAIL (2025-07-09 02:34:02 UTC)
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
02:25:27.135824454 (377h13m54.062119141s) [noise=true , address=104.36.82.12:4150 , piece_id=DRG5SBA3EE4KVQPEQOJZ6SS537SFWYFUYQXOAQMWYTT6FPVSJIPQ, seg_pos={0 0}, upload=302099]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: context canceled; context canceled
Jul 9 02:25:27.135860079 (377h13m54.062154769s) [seg_pos={0 0}, upload=302099]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 02:25:27.135957283 (377h13m54.062251971s) [upload=302099]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:25:27.135964211 (377h13m54.062258897s) [upload=302099]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:25:27.199410713 (377h13m54.125705389s) [upload=302099]: Done uploading segments...
Jul 9 02:25:27.199427901 (377h13m54.125722577s) [upload=302099]: Upload finished. err: <nil>
Jul 9 02:25:57.289077655 (377h14m24.215372329s) [upload=302109]: Starting upload
Jul 9 02:25:57.289186750 (377h14m24.215481424s) [upload=302109]: Done starting upload
Jul 9 02:25:57.289444851 (377h14m24.215739548s) [upload=302109]: Uploading segments...
Jul 9 02:25:57.289623846 (377h14m24.215918535s) [upload=302109]: Got next segment. Inline: false
Jul 9 02:25:57.289696741 (377h14m24.215991432s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:25:57.289703375 (377h14m24.215998067s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:25:57.318313152 (377h14m24.244607838s) [seg_pos={0 0}, upload=302109]: Begin upload segment...
Jul 9 02:25:57.318363286 (377h14m24.244657964s) [seg_pos={0 0}, upload=302109]: Done begin upload segment.
Jul 9 02:25:57.318386772 (377h14m24.244681463s) [noise=true , address=77.90.31.11:4553 , piece_id=TYW7NBJWACIT7OQS7A4WGXUPR3MB2FKAN2OMGAZJERB4X2DLAO2Q, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318412127 (377h14m24.244706825s) [noise=true , address=77.90.31.10:4518 , piece_id=VJ4HWM2X4VSRSUCQL5QLMBYPFBYV4JYS5ZXCBCB6MTMB76EHZFGQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318822149 (377h14m24.245116841s) [noise=true , address=45.91.116.9:4263 , piece_id=5YILAJ5BZBGOURSZE343ST4RMOTWM6ZOB3EEQQWL5WUNYWOGSAPA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318965862 (377h14m24.24526056s) [noise=true , address=45.91.116.8:4045 , piece_id=KM6ZYTHR6JGTARKZQZYIESPKBEU5H2HQFA7OARIKD2ZXDHPFSMGA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.319187516 (377h14m24.245482203s) [noise=true , address=104.36.82.7:4909 , piece_id=ZRHLWDQ5KIDRWPJ5FXAANQTWERKXCR2PPEGN442IOSYSRFAG7VZA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.319523815 (377h14m24.245818512s) [noise=true , address=45.91.116.14:4191 , piece_id=GTR6T4OTGZTEZ76FWQZ46OCH5WB3O5X27EN4AV33ZJ7D5HX4535A, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.320556632 (377h14m24.24685133s) [noise=true , address=104.36.82.2:4732 , piece_id=DJIGHJMJRAULH2L4YT2WGRXPRQJMNKZG5LICHPSKOCHUL6QDV6BQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.321019141 (377h14m24.247313835s) [noise=true , address=77.90.31.17:4770 , piece_id=V36UBBIYOFJC6KFVVBXEVT4QBUIWN257D3YSDDRAGZVXH7ZXHTBA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.322591797 (377h14m24.248886493s) [noise=true , address=77.90.31.13:4627 , piece_id=HULEKFVZNDNKS4SWWGTJUHYZWWYPDCER5W4GFCVLLDL65PAHULFQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323120451 (377h14m24.249415138s) [noise=true , address=45.91.116.4:4939 , piece_id=ANA3EZU7OZWELNG4QEMTEAYZY5H3C3H4IJFT4XKPUB7WIF44FEVQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323544591 (377h14m24.249839288s) [noise=true , address=45.91.116.7:4011 , piece_id=VDOJRN6ACK4LO44BQTULO6BB3CLAVQI7YIC6XLR32CZV4UJSPE6Q, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323950189 (377h14m24.250244889s) [noise=true , address=104.36.82.10:4078 , piece_id=ZFVUQAEV6DDFLPKCZA2XJ2BZHMMM745LGB4JRZJZQPG72WECGMDQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.326585810 (377h14m24.252880507s) [noise=true , address=45.91.116.5:4297 , piece_id=MSDWDLY5PJHQVI5565EKANJVSTMZFEUD2OTYZ52C65XQUICNMC3A, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.327871214 (377h14m24.254165904s) [noise=true , address=104.36.82.6:4875 , piece_id=C275F2JPVAF52E34CEHT5QXCHUFGIQZ7XGMOAOUOT4RA7YOHRQLQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.328605288 (377h14m24.254899986s) [noise=true , address=104.36.82.5:4838 , piece_id=WK3ALBOEMWWTVEA4FKCS352SZ6YYB35LELURNWXQZK32K4HYDRNA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.335409087 (377h14m24.261703784s) [noise=true , address=45.91.116.6:4973 , piece_id=Z7HQQ2O5QUSJKT4EMIDMEI5PIAEHNVRYX7MALUR2GQULRFIBUOKA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.475344637 (377h14m24.401639326s) [upload=302109]: Next returned nil segment
Jul 9 02:25:57.475376181 (377h14m24.401670879s) [upload=302109]: Waiting for error group managing segments...
Jul 9 02:25:57.477269278 (377h14m24.403563958s) [noise=true , address=45.91.116.7:4011 , piece_id=VDOJRN6ACK4LO44BQTULO6BB3CLAVQI7YIC6XLR32CZV4UJSPE6Q, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.477736179 (377h14m24.40403086s) [noise=true , address=45.91.116.9:4263 , piece_id=5YILAJ5BZBGOURSZE343ST4RMOTWM6ZOB3EEQQWL5WUNYWOGSAPA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.478396652 (377h14m24.404691332s) [noise=true , address=77.90.31.10:4518 , piece_id=VJ4HWM2X4VSRSUCQL5QLMBYPFBYV4JYS5ZXCBCB6MTMB76EHZFGQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.478919538 (377h14m24.405214224s) [noise=true , address=45.91.116.14:4191 , piece_id=GTR6T4OTGZTEZ76FWQZ46OCH5WB3O5X27EN4AV33ZJ7D5HX4535A, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481426716 (377h14m24.407721396s) [noise=true , address=45.91.116.4:4939 , piece_id=ANA3EZU7OZWELNG4QEMTEAYZY5H3C3H4IJFT4XKPUB7WIF44FEVQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481718373 (377h14m24.408013051s) [noise=true , address=45.91.116.5:4297 , piece_id=MSDWDLY5PJHQVI5565EKANJVSTMZFEUD2OTYZ52C65XQUICNMC3A, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481930356 (377h14m24.408225033s) [noise=true , address=77.90.31.17:4770 , piece_id=V36UBBIYOFJC6KFVVBXEVT4QBUIWN257D3YSDDRAGZVXH7ZXHTBA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.482376147 (377h14m24.408670821s) [noise=true , address=45.91.116.8:4045 , piece_id=KM6ZYTHR6JGTARKZQZYIESPKBEU5H2HQFA7OARIKD2ZXDHPFSMGA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.482439192 (377h14m24.408733868s) [noise=true , address=45.91.116.6:4973 , piece_id=Z7HQQ2O5QUSJKT4EMIDMEI5PIAEHNVRYX7MALUR2GQULRFIBUOKA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.483047970 (377h14m24.409342644s) [noise=true , address=77.90.31.13:4627 , piece_id=HULEKFVZNDNKS4SWWGTJUHYZWWYPDCER5W4GFCVLLDL65PAHULFQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.497788632 (377h14m24.424083312s) [noise=true , address=77.90.31.11:4553 , piece_id=TYW7NBJWACIT7OQS7A4WGXUPR3MB2FKAN2OMGAZJERB4X2DLAO2Q, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:16.688921171 (377h14m43.615215851s) [noise=true , address=104.36.82.6:4875 , piece_id=C275F2JPVAF52E34CEHT5QXCHUFGIQZ7XGMOAOUOT4RA7YOHRQLQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:27.177893955 (377h14m54.104188635s) [noise=true , address=104.36.82.2:4732 , piece_id=DJIGHJMJRAULH2L4YT2WGRXPRQJMNKZG5LICHPSKOCHUL6QDV6BQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:27.177985584 (377h14m54.104280281s) [seg_pos={0 0}, upload=302109]: Segment reached optimal threshold of 13 pieces.
Jul 9 02:26:27.178189789 (377h14m54.104484461s) [noise=true , address=104.36.82.5:4838 , piece_id=WK3ALBOEMWWTVEA4FKCS352SZ6YYB35LELURNWXQZK32K4HYDRNA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178252042 (377h14m54.104546733s) [noise=true , address=104.36.82.7:4909 , piece_id=ZRHLWDQ5KIDRWPJ5FXAANQTWERKXCR2PPEGN442IOSYSRFAG7VZA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178273922 (377h14m54.104568608s) [noise=true , address=104.36.82.10:4078 , piece_id=ZFVUQAEV6DDFLPKCZA2XJ2BZHMMM745LGB4JRZJZQPG72WECGMDQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178305983 (377h14m54.104600682s) [seg_pos={0 0}, upload=302109]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 02:26:27.178428768 (377h14m54.104723465s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:26:27.178455810 (377h14m54.104750501s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:26:27.232166184 (377h14m54.158460883s) [upload=302109]: Done uploading segments...
Jul 9 02:26:27.232188444 (377h14m54.158483123s) [upload=302109]: Upload finished. err: <nil>
Duration: 469.961826ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
39 out of 60 is failed (threshold=3) [XX.XXX.X.X.XXXXXXX..XX....XX.XXX.XXX.XX.XXXXXX.XXXXX.XX.....] duration:
54.767324357s
FAIL (2025-07-09 02:33:39 UTC)
operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 54.767631069s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
4 out of 60 is failed (threshold=3) [................X.....X...X....................X............] duration:
596.242543ms
FAIL (2025-07-09 02:33:36 UTC)
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
02:25:27.135824454 (377h13m54.062119141s) [noise=true , address=104.36.82.12:4150 , piece_id=DRG5SBA3EE4KVQPEQOJZ6SS537SFWYFUYQXOAQMWYTT6FPVSJIPQ, seg_pos={0 0}, upload=302099]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: context canceled; context canceled
Jul 9 02:25:27.135860079 (377h13m54.062154769s) [seg_pos={0 0}, upload=302099]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 02:25:27.135957283 (377h13m54.062251971s) [upload=302099]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:25:27.135964211 (377h13m54.062258897s) [upload=302099]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:25:27.199410713 (377h13m54.125705389s) [upload=302099]: Done uploading segments...
Jul 9 02:25:27.199427901 (377h13m54.125722577s) [upload=302099]: Upload finished. err: <nil>
Jul 9 02:25:57.289077655 (377h14m24.215372329s) [upload=302109]: Starting upload
Jul 9 02:25:57.289186750 (377h14m24.215481424s) [upload=302109]: Done starting upload
Jul 9 02:25:57.289444851 (377h14m24.215739548s) [upload=302109]: Uploading segments...
Jul 9 02:25:57.289623846 (377h14m24.215918535s) [upload=302109]: Got next segment. Inline: false
Jul 9 02:25:57.289696741 (377h14m24.215991432s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:25:57.289703375 (377h14m24.215998067s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:25:57.318313152 (377h14m24.244607838s) [seg_pos={0 0}, upload=302109]: Begin upload segment...
Jul 9 02:25:57.318363286 (377h14m24.244657964s) [seg_pos={0 0}, upload=302109]: Done begin upload segment.
Jul 9 02:25:57.318386772 (377h14m24.244681463s) [noise=true , address=77.90.31.11:4553 , piece_id=TYW7NBJWACIT7OQS7A4WGXUPR3MB2FKAN2OMGAZJERB4X2DLAO2Q, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318412127 (377h14m24.244706825s) [noise=true , address=77.90.31.10:4518 , piece_id=VJ4HWM2X4VSRSUCQL5QLMBYPFBYV4JYS5ZXCBCB6MTMB76EHZFGQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318822149 (377h14m24.245116841s) [noise=true , address=45.91.116.9:4263 , piece_id=5YILAJ5BZBGOURSZE343ST4RMOTWM6ZOB3EEQQWL5WUNYWOGSAPA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318965862 (377h14m24.24526056s) [noise=true , address=45.91.116.8:4045 , piece_id=KM6ZYTHR6JGTARKZQZYIESPKBEU5H2HQFA7OARIKD2ZXDHPFSMGA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.319187516 (377h14m24.245482203s) [noise=true , address=104.36.82.7:4909 , piece_id=ZRHLWDQ5KIDRWPJ5FXAANQTWERKXCR2PPEGN442IOSYSRFAG7VZA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.319523815 (377h14m24.245818512s) [noise=true , address=45.91.116.14:4191 , piece_id=GTR6T4OTGZTEZ76FWQZ46OCH5WB3O5X27EN4AV33ZJ7D5HX4535A, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.320556632 (377h14m24.24685133s) [noise=true , address=104.36.82.2:4732 , piece_id=DJIGHJMJRAULH2L4YT2WGRXPRQJMNKZG5LICHPSKOCHUL6QDV6BQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.321019141 (377h14m24.247313835s) [noise=true , address=77.90.31.17:4770 , piece_id=V36UBBIYOFJC6KFVVBXEVT4QBUIWN257D3YSDDRAGZVXH7ZXHTBA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.322591797 (377h14m24.248886493s) [noise=true , address=77.90.31.13:4627 , piece_id=HULEKFVZNDNKS4SWWGTJUHYZWWYPDCER5W4GFCVLLDL65PAHULFQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323120451 (377h14m24.249415138s) [noise=true , address=45.91.116.4:4939 , piece_id=ANA3EZU7OZWELNG4QEMTEAYZY5H3C3H4IJFT4XKPUB7WIF44FEVQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323544591 (377h14m24.249839288s) [noise=true , address=45.91.116.7:4011 , piece_id=VDOJRN6ACK4LO44BQTULO6BB3CLAVQI7YIC6XLR32CZV4UJSPE6Q, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323950189 (377h14m24.250244889s) [noise=true , address=104.36.82.10:4078 , piece_id=ZFVUQAEV6DDFLPKCZA2XJ2BZHMMM745LGB4JRZJZQPG72WECGMDQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.326585810 (377h14m24.252880507s) [noise=true , address=45.91.116.5:4297 , piece_id=MSDWDLY5PJHQVI5565EKANJVSTMZFEUD2OTYZ52C65XQUICNMC3A, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.327871214 (377h14m24.254165904s) [noise=true , address=104.36.82.6:4875 , piece_id=C275F2JPVAF52E34CEHT5QXCHUFGIQZ7XGMOAOUOT4RA7YOHRQLQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.328605288 (377h14m24.254899986s) [noise=true , address=104.36.82.5:4838 , piece_id=WK3ALBOEMWWTVEA4FKCS352SZ6YYB35LELURNWXQZK32K4HYDRNA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.335409087 (377h14m24.261703784s) [noise=true , address=45.91.116.6:4973 , piece_id=Z7HQQ2O5QUSJKT4EMIDMEI5PIAEHNVRYX7MALUR2GQULRFIBUOKA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.475344637 (377h14m24.401639326s) [upload=302109]: Next returned nil segment
Jul 9 02:25:57.475376181 (377h14m24.401670879s) [upload=302109]: Waiting for error group managing segments...
Jul 9 02:25:57.477269278 (377h14m24.403563958s) [noise=true , address=45.91.116.7:4011 , piece_id=VDOJRN6ACK4LO44BQTULO6BB3CLAVQI7YIC6XLR32CZV4UJSPE6Q, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.477736179 (377h14m24.40403086s) [noise=true , address=45.91.116.9:4263 , piece_id=5YILAJ5BZBGOURSZE343ST4RMOTWM6ZOB3EEQQWL5WUNYWOGSAPA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.478396652 (377h14m24.404691332s) [noise=true , address=77.90.31.10:4518 , piece_id=VJ4HWM2X4VSRSUCQL5QLMBYPFBYV4JYS5ZXCBCB6MTMB76EHZFGQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.478919538 (377h14m24.405214224s) [noise=true , address=45.91.116.14:4191 , piece_id=GTR6T4OTGZTEZ76FWQZ46OCH5WB3O5X27EN4AV33ZJ7D5HX4535A, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481426716 (377h14m24.407721396s) [noise=true , address=45.91.116.4:4939 , piece_id=ANA3EZU7OZWELNG4QEMTEAYZY5H3C3H4IJFT4XKPUB7WIF44FEVQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481718373 (377h14m24.408013051s) [noise=true , address=45.91.116.5:4297 , piece_id=MSDWDLY5PJHQVI5565EKANJVSTMZFEUD2OTYZ52C65XQUICNMC3A, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481930356 (377h14m24.408225033s) [noise=true , address=77.90.31.17:4770 , piece_id=V36UBBIYOFJC6KFVVBXEVT4QBUIWN257D3YSDDRAGZVXH7ZXHTBA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.482376147 (377h14m24.408670821s) [noise=true , address=45.91.116.8:4045 , piece_id=KM6ZYTHR6JGTARKZQZYIESPKBEU5H2HQFA7OARIKD2ZXDHPFSMGA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.482439192 (377h14m24.408733868s) [noise=true , address=45.91.116.6:4973 , piece_id=Z7HQQ2O5QUSJKT4EMIDMEI5PIAEHNVRYX7MALUR2GQULRFIBUOKA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.483047970 (377h14m24.409342644s) [noise=true , address=77.90.31.13:4627 , piece_id=HULEKFVZNDNKS4SWWGTJUHYZWWYPDCER5W4GFCVLLDL65PAHULFQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.497788632 (377h14m24.424083312s) [noise=true , address=77.90.31.11:4553 , piece_id=TYW7NBJWACIT7OQS7A4WGXUPR3MB2FKAN2OMGAZJERB4X2DLAO2Q, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:16.688921171 (377h14m43.615215851s) [noise=true , address=104.36.82.6:4875 , piece_id=C275F2JPVAF52E34CEHT5QXCHUFGIQZ7XGMOAOUOT4RA7YOHRQLQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:27.177893955 (377h14m54.104188635s) [noise=true , address=104.36.82.2:4732 , piece_id=DJIGHJMJRAULH2L4YT2WGRXPRQJMNKZG5LICHPSKOCHUL6QDV6BQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:27.177985584 (377h14m54.104280281s) [seg_pos={0 0}, upload=302109]: Segment reached optimal threshold of 13 pieces.
Jul 9 02:26:27.178189789 (377h14m54.104484461s) [noise=true , address=104.36.82.5:4838 , piece_id=WK3ALBOEMWWTVEA4FKCS352SZ6YYB35LELURNWXQZK32K4HYDRNA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178252042 (377h14m54.104546733s) [noise=true , address=104.36.82.7:4909 , piece_id=ZRHLWDQ5KIDRWPJ5FXAANQTWERKXCR2PPEGN442IOSYSRFAG7VZA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178273922 (377h14m54.104568608s) [noise=true , address=104.36.82.10:4078 , piece_id=ZFVUQAEV6DDFLPKCZA2XJ2BZHMMM745LGB4JRZJZQPG72WECGMDQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178305983 (377h14m54.104600682s) [seg_pos={0 0}, upload=302109]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 02:26:27.178428768 (377h14m54.104723465s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:26:27.178455810 (377h14m54.104750501s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:26:27.232166184 (377h14m54.158460883s) [upload=302109]: Done uploading segments...
Jul 9 02:26:27.232188444 (377h14m54.158483123s) [upload=302109]: Upload finished. err: <nil>
Duration: 596.273906ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
19 out of 60 is failed (threshold=3) [..X....X....X.........X....X.X..XX.XX..X.X.X.....XXX.XX.X...] duration:
493.906282ms
FAIL (2025-07-09 02:33:09 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 493.937043ms
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.....................................X......................] duration:
25.528290566s
FAIL (2025-07-09 02:33:06 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
3:56.263910643 (377h2m23.19020535s) [upload=301954]: Got next segment. Inline: false
Jul 9 02:13:56.263954358 (377h2m23.190249034s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:13:56.263959891 (377h2m23.190254572s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:13:56.302204249 (377h2m23.228498944s) [seg_pos={0 0}, upload=301954]: Begin upload segment...
Jul 9 02:13:56.302291587 (377h2m23.22858628s) [seg_pos={0 0}, upload=301954]: Done begin upload segment.
Jul 9 02:13:56.302338947 (377h2m23.228633682s) [noise=true , address=77.90.31.14:4689 , piece_id=LW7OT74AJCCC3TEKY6OZI553YK6YQFQ6C7PORQB53E7FUS24KQKQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.302352599 (377h2m23.228647272s) [noise=true , address=104.36.82.8:4986 , piece_id=KPY5JVLSAEB5GUKUJJSIF5M6IIHTJISEYYAXCGUSJ7OLI5QCIBNQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.302450837 (377h2m23.228745531s) [noise=true , address=77.90.31.16:4739 , piece_id=H6UCV6RURPXKC2H3YGAT2OXV2VZMW2DWIRG6TFD6CVXKOVMDP2VQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.302958774 (377h2m23.229253667s) [noise=true , address=77.90.31.13:4640 , piece_id=TPPN75DX3UCBM346CSVJN3GSOA47QV4FVPYSSAHWCJYEM42OCVLQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303069086 (377h2m23.229363782s) [noise=true , address=104.36.82.17:4345 , piece_id=YI27O5WLBYO3TPCX4MVUATFORY6U3EQMDKQHGKRHIZMTG7WGFB2A, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303495994 (377h2m23.22979069s) [noise=true , address=104.36.82.36:4644 , piece_id=K2G3MAPD7TWYZPKNHQBMEPJRIQDLE75FYVPAZFX4I3GJ4IUAA7XQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303575840 (377h2m23.229870536s) [noise=true , address=77.90.31.21:4898 , piece_id=75XXVNT4EV4D5UTX7O5XFL44IZOYY5LZSI2VPFQZPRERL6W5M23A, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303949062 (377h2m23.230245641s) [noise=true , address=79.127.201.215:4339 , piece_id=XDCTYE2SVI6RAPCXTRVANFCZYWJDZXLBHHJ47NQY6V7D36DEFPGA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304060410 (377h2m23.230355106s) [noise=true , address=104.36.82.41:4768 , piece_id=BGMF33OKUZSZKGIHJYI3RNXEBDE54TX67DYLJ7JCZMRNK6ZAU22Q, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304407993 (377h2m23.230702685s) [noise=true , address=77.90.31.10:4525 , piece_id=GPVSIZIBZWTUCP5RFYVPFYIYIL2LWU5PU3UASQVRQLY7SSSFMQBA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304548115 (377h2m23.230842812s) [noise=true , address=104.36.82.23:4464 , piece_id=JCJ4MGEZIIZ7YCTM6QXC5ZJ7PI4OEJBOMHZYBU57D2SNGWJWB5CQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304795164 (377h2m23.231089862s) [noise=true , address=77.90.31.18:4838 , piece_id=TGWYXA3AF7LDG4KPVCRNEYFDADAG6TQ5DAX7JJVW5BLVVVNECILA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305012510 (377h2m23.231307211s) [noise=true , address=104.36.82.34:4554 , piece_id=57K5SDV3YEGLVMSJGSLXNKEWOLETXE6CWBM2NPVCNKLFHPVQCJBQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305113737 (377h2m23.231408427s) [noise=true , address=104.36.82.4:4834 , piece_id=YLYIASUPNRTK7KOMRYLN7OQLQBPWC7WFGCPDGUVIOEQJWQQ7A7GA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305383755 (377h2m23.231678434s) [noise=true , address=104.36.82.38:4698 , piece_id=TSOXJNQEWODCWFEITH4IYKZI7A75UZEXRQPIQO7LMP6LT3WLNXUA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305483822 (377h2m23.231778517s) [noise=true , address=104.36.82.42:4830 , piece_id=ENYQTUCIF7OWQPCXTYS24OC5QT63ZJJXOY652GLIG7OQPJEQCGGA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305954019 (377h2m23.232248718s) [noise=true , address=77.90.31.21:4907 , piece_id=2EUWATHUQNZ77BJV65ZFKR5WL4D3AUURLBQ32XR2IVBA7P4TITOQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.306425517 (377h2m23.232720213s) [noise=true , address=104.36.82.22:4541 , piece_id=XN4HDFDNZ4YUJTDPOTMMDFX4FE2YVZDGCIMHEMJGWXRPNHQ6EU7A, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.375470790 (377h2m23.301765481s) [upload=301954]: Next returned nil segment
Jul 9 02:13:56.375489934 (377h2m23.30178463s) [upload=301954]: Waiting for error group managing segments...
Jul 9 02:13:56.381456859 (377h2m23.307751539s) [noise=true , address=77.90.31.10:4525 , piece_id=GPVSIZIBZWTUCP5RFYVPFYIYIL2LWU5PU3UASQVRQLY7SSSFMQBA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.381956907 (377h2m23.308251586s) [noise=true , address=77.90.31.14:4689 , piece_id=LW7OT74AJCCC3TEKY6OZI553YK6YQFQ6C7PORQB53E7FUS24KQKQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.382837088 (377h2m23.309131761s) [noise=true , address=77.90.31.21:4898 , piece_id=75XXVNT4EV4D5UTX7O5XFL44IZOYY5LZSI2VPFQZPRERL6W5M23A, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.393489125 (377h2m23.319783812s) [noise=true , address=77.90.31.21:4907 , piece_id=2EUWATHUQNZ77BJV65ZFKR5WL4D3AUURLBQ32XR2IVBA7P4TITOQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.404692801 (377h2m23.330987481s) [noise=true , address=77.90.31.16:4739 , piece_id=H6UCV6RURPXKC2H3YGAT2OXV2VZMW2DWIRG6TFD6CVXKOVMDP2VQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.404986115 (377h2m23.331280795s) [noise=true , address=77.90.31.18:4838 , piece_id=TGWYXA3AF7LDG4KPVCRNEYFDADAG6TQ5DAX7JJVW5BLVVVNECILA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.433785086 (377h2m23.360079766s) [noise=true , address=77.90.31.13:4640 , piece_id=TPPN75DX3UCBM346CSVJN3GSOA47QV4FVPYSSAHWCJYEM42OCVLQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.598499747 (377h2m23.524794427s) [noise=true , address=79.127.201.215:4339 , piece_id=XDCTYE2SVI6RAPCXTRVANFCZYWJDZXLBHHJ47NQY6V7D36DEFPGA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:11.107785584 (377h2m38.034080273s) [noise=true , address=104.36.82.22:4541 , piece_id=XN4HDFDNZ4YUJTDPOTMMDFX4FE2YVZDGCIMHEMJGWXRPNHQ6EU7A, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:12.515567029 (377h2m39.441861713s) [noise=true , address=104.36.82.17:4345 , piece_id=YI27O5WLBYO3TPCX4MVUATFORY6U3EQMDKQHGKRHIZMTG7WGFB2A, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:12.923090587 (377h2m39.849385267s) [noise=true , address=104.36.82.8:4986 , piece_id=KPY5JVLSAEB5GUKUJJSIF5M6IIHTJISEYYAXCGUSJ7OLI5QCIBNQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:13.453172659 (377h2m40.379467346s) [noise=true , address=104.36.82.38:4698 , piece_id=TSOXJNQEWODCWFEITH4IYKZI7A75UZEXRQPIQO7LMP6LT3WLNXUA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:14.416524856 (377h2m41.342819534s) [noise=true , address=104.36.82.34:4554 , piece_id=57K5SDV3YEGLVMSJGSLXNKEWOLETXE6CWBM2NPVCNKLFHPVQCJBQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:14.846730699 (377h2m41.773025379s) [noise=true , address=104.36.82.23:4464 , piece_id=JCJ4MGEZIIZ7YCTM6QXC5ZJ7PI4OEJBOMHZYBU57D2SNGWJWB5CQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:14.846763777 (377h2m41.773058463s) [seg_pos={0 0}, upload=301954]: Segment reached optimal threshold of 14 pieces.
Jul 9 02:14:14.846894905 (377h2m41.773189582s) [noise=true , address=104.36.82.36:4644 , piece_id=K2G3MAPD7TWYZPKNHQBMEPJRIQDLE75FYVPAZFX4I3GJ4IUAA7XQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1oYHbb73ACitbfkkP5P6cmRyTt6kmaS1y49G4vbsvTvVDwByhT): protocol: expected piece hash; context canceled; EOF
Jul 9 02:14:14.846948435 (377h2m41.773243131s) [noise=true , address=104.36.82.4:4834 , piece_id=YLYIASUPNRTK7KOMRYLN7OQLQBPWC7WFGCPDGUVIOEQJWQQ7A7GA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Z241DCYA5Wig2TdEogxTpY6FzVmFbnKtmHcTwxLxfcZafgooy): protocol: expected piece hash; context canceled; EOF
Jul 9 02:14:14.847094561 (377h2m41.773389249s) [noise=true , address=104.36.82.42:4830 , piece_id=ENYQTUCIF7OWQPCXTYS24OC5QT63ZJJXOY652GLIG7OQPJEQCGGA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:142ocSHHtv8BpSebXFNk5qk1Y23a8pbFAZqE3cq6i3RMX3dPw9): protocol: expected piece hash; context canceled; EOF
Jul 9 02:14:14.847127825 (377h2m41.773422511s) [noise=true , address=104.36.82.41:4768 , piece_id=BGMF33OKUZSZKGIHJYI3RNXEBDE54TX67DYLJ7JCZMRNK6ZAU22Q, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: write tcp 159.223.143.39:43796->104.36.82.41:4768: use of closed network connection; write tcp 159.223.143.39:43796->104.36.82.41:4768: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:43796->104.36.82.41:4768: use of closed network connection
Jul 9 02:14:14.847142999 (377h2m41.773437687s) [seg_pos={0 0}, upload=301954]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jul 9 02:14:14.847204962 (377h2m41.773499652s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:14:14.847209974 (377h2m41.773504671s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:14:14.909581364 (377h2m41.83587606s) [upload=301954]: Done uploading segments...
Jul 9 02:14:14.909605483 (377h2m41.835900172s) [upload=301954]: Upload finished. err: <nil>
Duration: 25.528325479s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
19 out of 60 is failed (threshold=3) [...X....X....X.........X....X.X..XX.XX..X.X.X.....XXX.XX.X..] duration:
22.382712837s
FAIL (2025-07-09 02:32:52 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 22.382751347s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
40 out of 60 is failed (threshold=3) [XXX.XXX.X.X.XXXXXXX..XX....XX.XXX.XXX.XX.XXXXXX.XXXXX.XX....] duration:
43.136218706s
FAIL (2025-07-09 02:32:44 UTC)
operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 43.136235252s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [......................................X.....................] duration:
26.833882699s
FAIL (2025-07-09 02:32:38 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
3:56.263910643 (377h2m23.19020535s) [upload=301954]: Got next segment. Inline: false
Jul 9 02:13:56.263954358 (377h2m23.190249034s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:13:56.263959891 (377h2m23.190254572s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:13:56.302204249 (377h2m23.228498944s) [seg_pos={0 0}, upload=301954]: Begin upload segment...
Jul 9 02:13:56.302291587 (377h2m23.22858628s) [seg_pos={0 0}, upload=301954]: Done begin upload segment.
Jul 9 02:13:56.302338947 (377h2m23.228633682s) [noise=true , address=77.90.31.14:4689 , piece_id=LW7OT74AJCCC3TEKY6OZI553YK6YQFQ6C7PORQB53E7FUS24KQKQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.302352599 (377h2m23.228647272s) [noise=true , address=104.36.82.8:4986 , piece_id=KPY5JVLSAEB5GUKUJJSIF5M6IIHTJISEYYAXCGUSJ7OLI5QCIBNQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.302450837 (377h2m23.228745531s) [noise=true , address=77.90.31.16:4739 , piece_id=H6UCV6RURPXKC2H3YGAT2OXV2VZMW2DWIRG6TFD6CVXKOVMDP2VQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.302958774 (377h2m23.229253667s) [noise=true , address=77.90.31.13:4640 , piece_id=TPPN75DX3UCBM346CSVJN3GSOA47QV4FVPYSSAHWCJYEM42OCVLQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303069086 (377h2m23.229363782s) [noise=true , address=104.36.82.17:4345 , piece_id=YI27O5WLBYO3TPCX4MVUATFORY6U3EQMDKQHGKRHIZMTG7WGFB2A, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303495994 (377h2m23.22979069s) [noise=true , address=104.36.82.36:4644 , piece_id=K2G3MAPD7TWYZPKNHQBMEPJRIQDLE75FYVPAZFX4I3GJ4IUAA7XQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303575840 (377h2m23.229870536s) [noise=true , address=77.90.31.21:4898 , piece_id=75XXVNT4EV4D5UTX7O5XFL44IZOYY5LZSI2VPFQZPRERL6W5M23A, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303949062 (377h2m23.230245641s) [noise=true , address=79.127.201.215:4339 , piece_id=XDCTYE2SVI6RAPCXTRVANFCZYWJDZXLBHHJ47NQY6V7D36DEFPGA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304060410 (377h2m23.230355106s) [noise=true , address=104.36.82.41:4768 , piece_id=BGMF33OKUZSZKGIHJYI3RNXEBDE54TX67DYLJ7JCZMRNK6ZAU22Q, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304407993 (377h2m23.230702685s) [noise=true , address=77.90.31.10:4525 , piece_id=GPVSIZIBZWTUCP5RFYVPFYIYIL2LWU5PU3UASQVRQLY7SSSFMQBA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304548115 (377h2m23.230842812s) [noise=true , address=104.36.82.23:4464 , piece_id=JCJ4MGEZIIZ7YCTM6QXC5ZJ7PI4OEJBOMHZYBU57D2SNGWJWB5CQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304795164 (377h2m23.231089862s) [noise=true , address=77.90.31.18:4838 , piece_id=TGWYXA3AF7LDG4KPVCRNEYFDADAG6TQ5DAX7JJVW5BLVVVNECILA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305012510 (377h2m23.231307211s) [noise=true , address=104.36.82.34:4554 , piece_id=57K5SDV3YEGLVMSJGSLXNKEWOLETXE6CWBM2NPVCNKLFHPVQCJBQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305113737 (377h2m23.231408427s) [noise=true , address=104.36.82.4:4834 , piece_id=YLYIASUPNRTK7KOMRYLN7OQLQBPWC7WFGCPDGUVIOEQJWQQ7A7GA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305383755 (377h2m23.231678434s) [noise=true , address=104.36.82.38:4698 , piece_id=TSOXJNQEWODCWFEITH4IYKZI7A75UZEXRQPIQO7LMP6LT3WLNXUA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305483822 (377h2m23.231778517s) [noise=true , address=104.36.82.42:4830 , piece_id=ENYQTUCIF7OWQPCXTYS24OC5QT63ZJJXOY652GLIG7OQPJEQCGGA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305954019 (377h2m23.232248718s) [noise=true , address=77.90.31.21:4907 , piece_id=2EUWATHUQNZ77BJV65ZFKR5WL4D3AUURLBQ32XR2IVBA7P4TITOQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.306425517 (377h2m23.232720213s) [noise=true , address=104.36.82.22:4541 , piece_id=XN4HDFDNZ4YUJTDPOTMMDFX4FE2YVZDGCIMHEMJGWXRPNHQ6EU7A, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.375470790 (377h2m23.301765481s) [upload=301954]: Next returned nil segment
Jul 9 02:13:56.375489934 (377h2m23.30178463s) [upload=301954]: Waiting for error group managing segments...
Jul 9 02:13:56.381456859 (377h2m23.307751539s) [noise=true , address=77.90.31.10:4525 , piece_id=GPVSIZIBZWTUCP5RFYVPFYIYIL2LWU5PU3UASQVRQLY7SSSFMQBA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.381956907 (377h2m23.308251586s) [noise=true , address=77.90.31.14:4689 , piece_id=LW7OT74AJCCC3TEKY6OZI553YK6YQFQ6C7PORQB53E7FUS24KQKQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.382837088 (377h2m23.309131761s) [noise=true , address=77.90.31.21:4898 , piece_id=75XXVNT4EV4D5UTX7O5XFL44IZOYY5LZSI2VPFQZPRERL6W5M23A, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.393489125 (377h2m23.319783812s) [noise=true , address=77.90.31.21:4907 , piece_id=2EUWATHUQNZ77BJV65ZFKR5WL4D3AUURLBQ32XR2IVBA7P4TITOQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.404692801 (377h2m23.330987481s) [noise=true , address=77.90.31.16:4739 , piece_id=H6UCV6RURPXKC2H3YGAT2OXV2VZMW2DWIRG6TFD6CVXKOVMDP2VQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.404986115 (377h2m23.331280795s) [noise=true , address=77.90.31.18:4838 , piece_id=TGWYXA3AF7LDG4KPVCRNEYFDADAG6TQ5DAX7JJVW5BLVVVNECILA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.433785086 (377h2m23.360079766s) [noise=true , address=77.90.31.13:4640 , piece_id=TPPN75DX3UCBM346CSVJN3GSOA47QV4FVPYSSAHWCJYEM42OCVLQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.598499747 (377h2m23.524794427s) [noise=true , address=79.127.201.215:4339 , piece_id=XDCTYE2SVI6RAPCXTRVANFCZYWJDZXLBHHJ47NQY6V7D36DEFPGA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:11.107785584 (377h2m38.034080273s) [noise=true , address=104.36.82.22:4541 , piece_id=XN4HDFDNZ4YUJTDPOTMMDFX4FE2YVZDGCIMHEMJGWXRPNHQ6EU7A, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:12.515567029 (377h2m39.441861713s) [noise=true , address=104.36.82.17:4345 , piece_id=YI27O5WLBYO3TPCX4MVUATFORY6U3EQMDKQHGKRHIZMTG7WGFB2A, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:12.923090587 (377h2m39.849385267s) [noise=true , address=104.36.82.8:4986 , piece_id=KPY5JVLSAEB5GUKUJJSIF5M6IIHTJISEYYAXCGUSJ7OLI5QCIBNQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:13.453172659 (377h2m40.379467346s) [noise=true , address=104.36.82.38:4698 , piece_id=TSOXJNQEWODCWFEITH4IYKZI7A75UZEXRQPIQO7LMP6LT3WLNXUA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:14.416524856 (377h2m41.342819534s) [noise=true , address=104.36.82.34:4554 , piece_id=57K5SDV3YEGLVMSJGSLXNKEWOLETXE6CWBM2NPVCNKLFHPVQCJBQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:14.846730699 (377h2m41.773025379s) [noise=true , address=104.36.82.23:4464 , piece_id=JCJ4MGEZIIZ7YCTM6QXC5ZJ7PI4OEJBOMHZYBU57D2SNGWJWB5CQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:14.846763777 (377h2m41.773058463s) [seg_pos={0 0}, upload=301954]: Segment reached optimal threshold of 14 pieces.
Jul 9 02:14:14.846894905 (377h2m41.773189582s) [noise=true , address=104.36.82.36:4644 , piece_id=K2G3MAPD7TWYZPKNHQBMEPJRIQDLE75FYVPAZFX4I3GJ4IUAA7XQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1oYHbb73ACitbfkkP5P6cmRyTt6kmaS1y49G4vbsvTvVDwByhT): protocol: expected piece hash; context canceled; EOF
Jul 9 02:14:14.846948435 (377h2m41.773243131s) [noise=true , address=104.36.82.4:4834 , piece_id=YLYIASUPNRTK7KOMRYLN7OQLQBPWC7WFGCPDGUVIOEQJWQQ7A7GA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Z241DCYA5Wig2TdEogxTpY6FzVmFbnKtmHcTwxLxfcZafgooy): protocol: expected piece hash; context canceled; EOF
Jul 9 02:14:14.847094561 (377h2m41.773389249s) [noise=true , address=104.36.82.42:4830 , piece_id=ENYQTUCIF7OWQPCXTYS24OC5QT63ZJJXOY652GLIG7OQPJEQCGGA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:142ocSHHtv8BpSebXFNk5qk1Y23a8pbFAZqE3cq6i3RMX3dPw9): protocol: expected piece hash; context canceled; EOF
Jul 9 02:14:14.847127825 (377h2m41.773422511s) [noise=true , address=104.36.82.41:4768 , piece_id=BGMF33OKUZSZKGIHJYI3RNXEBDE54TX67DYLJ7JCZMRNK6ZAU22Q, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: write tcp 159.223.143.39:43796->104.36.82.41:4768: use of closed network connection; write tcp 159.223.143.39:43796->104.36.82.41:4768: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:43796->104.36.82.41:4768: use of closed network connection
Jul 9 02:14:14.847142999 (377h2m41.773437687s) [seg_pos={0 0}, upload=301954]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jul 9 02:14:14.847204962 (377h2m41.773499652s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:14:14.847209974 (377h2m41.773504671s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:14:14.909581364 (377h2m41.83587606s) [upload=301954]: Done uploading segments...
Jul 9 02:14:14.909605483 (377h2m41.835900172s) [upload=301954]: Upload finished. err: <nil>
Duration: 26.833910168s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
4 out of 60 is failed (threshold=3) [.................X.....X...X....................X...........] duration:
431.523154ms
FAIL (2025-07-09 02:32:25 UTC)
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
02:25:27.135824454 (377h13m54.062119141s) [noise=true , address=104.36.82.12:4150 , piece_id=DRG5SBA3EE4KVQPEQOJZ6SS537SFWYFUYQXOAQMWYTT6FPVSJIPQ, seg_pos={0 0}, upload=302099]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: context canceled; context canceled
Jul 9 02:25:27.135860079 (377h13m54.062154769s) [seg_pos={0 0}, upload=302099]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 02:25:27.135957283 (377h13m54.062251971s) [upload=302099]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:25:27.135964211 (377h13m54.062258897s) [upload=302099]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:25:27.199410713 (377h13m54.125705389s) [upload=302099]: Done uploading segments...
Jul 9 02:25:27.199427901 (377h13m54.125722577s) [upload=302099]: Upload finished. err: <nil>
Jul 9 02:25:57.289077655 (377h14m24.215372329s) [upload=302109]: Starting upload
Jul 9 02:25:57.289186750 (377h14m24.215481424s) [upload=302109]: Done starting upload
Jul 9 02:25:57.289444851 (377h14m24.215739548s) [upload=302109]: Uploading segments...
Jul 9 02:25:57.289623846 (377h14m24.215918535s) [upload=302109]: Got next segment. Inline: false
Jul 9 02:25:57.289696741 (377h14m24.215991432s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:25:57.289703375 (377h14m24.215998067s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:25:57.318313152 (377h14m24.244607838s) [seg_pos={0 0}, upload=302109]: Begin upload segment...
Jul 9 02:25:57.318363286 (377h14m24.244657964s) [seg_pos={0 0}, upload=302109]: Done begin upload segment.
Jul 9 02:25:57.318386772 (377h14m24.244681463s) [noise=true , address=77.90.31.11:4553 , piece_id=TYW7NBJWACIT7OQS7A4WGXUPR3MB2FKAN2OMGAZJERB4X2DLAO2Q, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318412127 (377h14m24.244706825s) [noise=true , address=77.90.31.10:4518 , piece_id=VJ4HWM2X4VSRSUCQL5QLMBYPFBYV4JYS5ZXCBCB6MTMB76EHZFGQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318822149 (377h14m24.245116841s) [noise=true , address=45.91.116.9:4263 , piece_id=5YILAJ5BZBGOURSZE343ST4RMOTWM6ZOB3EEQQWL5WUNYWOGSAPA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318965862 (377h14m24.24526056s) [noise=true , address=45.91.116.8:4045 , piece_id=KM6ZYTHR6JGTARKZQZYIESPKBEU5H2HQFA7OARIKD2ZXDHPFSMGA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.319187516 (377h14m24.245482203s) [noise=true , address=104.36.82.7:4909 , piece_id=ZRHLWDQ5KIDRWPJ5FXAANQTWERKXCR2PPEGN442IOSYSRFAG7VZA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.319523815 (377h14m24.245818512s) [noise=true , address=45.91.116.14:4191 , piece_id=GTR6T4OTGZTEZ76FWQZ46OCH5WB3O5X27EN4AV33ZJ7D5HX4535A, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.320556632 (377h14m24.24685133s) [noise=true , address=104.36.82.2:4732 , piece_id=DJIGHJMJRAULH2L4YT2WGRXPRQJMNKZG5LICHPSKOCHUL6QDV6BQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.321019141 (377h14m24.247313835s) [noise=true , address=77.90.31.17:4770 , piece_id=V36UBBIYOFJC6KFVVBXEVT4QBUIWN257D3YSDDRAGZVXH7ZXHTBA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.322591797 (377h14m24.248886493s) [noise=true , address=77.90.31.13:4627 , piece_id=HULEKFVZNDNKS4SWWGTJUHYZWWYPDCER5W4GFCVLLDL65PAHULFQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323120451 (377h14m24.249415138s) [noise=true , address=45.91.116.4:4939 , piece_id=ANA3EZU7OZWELNG4QEMTEAYZY5H3C3H4IJFT4XKPUB7WIF44FEVQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323544591 (377h14m24.249839288s) [noise=true , address=45.91.116.7:4011 , piece_id=VDOJRN6ACK4LO44BQTULO6BB3CLAVQI7YIC6XLR32CZV4UJSPE6Q, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323950189 (377h14m24.250244889s) [noise=true , address=104.36.82.10:4078 , piece_id=ZFVUQAEV6DDFLPKCZA2XJ2BZHMMM745LGB4JRZJZQPG72WECGMDQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.326585810 (377h14m24.252880507s) [noise=true , address=45.91.116.5:4297 , piece_id=MSDWDLY5PJHQVI5565EKANJVSTMZFEUD2OTYZ52C65XQUICNMC3A, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.327871214 (377h14m24.254165904s) [noise=true , address=104.36.82.6:4875 , piece_id=C275F2JPVAF52E34CEHT5QXCHUFGIQZ7XGMOAOUOT4RA7YOHRQLQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.328605288 (377h14m24.254899986s) [noise=true , address=104.36.82.5:4838 , piece_id=WK3ALBOEMWWTVEA4FKCS352SZ6YYB35LELURNWXQZK32K4HYDRNA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.335409087 (377h14m24.261703784s) [noise=true , address=45.91.116.6:4973 , piece_id=Z7HQQ2O5QUSJKT4EMIDMEI5PIAEHNVRYX7MALUR2GQULRFIBUOKA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.475344637 (377h14m24.401639326s) [upload=302109]: Next returned nil segment
Jul 9 02:25:57.475376181 (377h14m24.401670879s) [upload=302109]: Waiting for error group managing segments...
Jul 9 02:25:57.477269278 (377h14m24.403563958s) [noise=true , address=45.91.116.7:4011 , piece_id=VDOJRN6ACK4LO44BQTULO6BB3CLAVQI7YIC6XLR32CZV4UJSPE6Q, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.477736179 (377h14m24.40403086s) [noise=true , address=45.91.116.9:4263 , piece_id=5YILAJ5BZBGOURSZE343ST4RMOTWM6ZOB3EEQQWL5WUNYWOGSAPA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.478396652 (377h14m24.404691332s) [noise=true , address=77.90.31.10:4518 , piece_id=VJ4HWM2X4VSRSUCQL5QLMBYPFBYV4JYS5ZXCBCB6MTMB76EHZFGQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.478919538 (377h14m24.405214224s) [noise=true , address=45.91.116.14:4191 , piece_id=GTR6T4OTGZTEZ76FWQZ46OCH5WB3O5X27EN4AV33ZJ7D5HX4535A, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481426716 (377h14m24.407721396s) [noise=true , address=45.91.116.4:4939 , piece_id=ANA3EZU7OZWELNG4QEMTEAYZY5H3C3H4IJFT4XKPUB7WIF44FEVQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481718373 (377h14m24.408013051s) [noise=true , address=45.91.116.5:4297 , piece_id=MSDWDLY5PJHQVI5565EKANJVSTMZFEUD2OTYZ52C65XQUICNMC3A, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481930356 (377h14m24.408225033s) [noise=true , address=77.90.31.17:4770 , piece_id=V36UBBIYOFJC6KFVVBXEVT4QBUIWN257D3YSDDRAGZVXH7ZXHTBA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.482376147 (377h14m24.408670821s) [noise=true , address=45.91.116.8:4045 , piece_id=KM6ZYTHR6JGTARKZQZYIESPKBEU5H2HQFA7OARIKD2ZXDHPFSMGA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.482439192 (377h14m24.408733868s) [noise=true , address=45.91.116.6:4973 , piece_id=Z7HQQ2O5QUSJKT4EMIDMEI5PIAEHNVRYX7MALUR2GQULRFIBUOKA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.483047970 (377h14m24.409342644s) [noise=true , address=77.90.31.13:4627 , piece_id=HULEKFVZNDNKS4SWWGTJUHYZWWYPDCER5W4GFCVLLDL65PAHULFQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.497788632 (377h14m24.424083312s) [noise=true , address=77.90.31.11:4553 , piece_id=TYW7NBJWACIT7OQS7A4WGXUPR3MB2FKAN2OMGAZJERB4X2DLAO2Q, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:16.688921171 (377h14m43.615215851s) [noise=true , address=104.36.82.6:4875 , piece_id=C275F2JPVAF52E34CEHT5QXCHUFGIQZ7XGMOAOUOT4RA7YOHRQLQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:27.177893955 (377h14m54.104188635s) [noise=true , address=104.36.82.2:4732 , piece_id=DJIGHJMJRAULH2L4YT2WGRXPRQJMNKZG5LICHPSKOCHUL6QDV6BQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:27.177985584 (377h14m54.104280281s) [seg_pos={0 0}, upload=302109]: Segment reached optimal threshold of 13 pieces.
Jul 9 02:26:27.178189789 (377h14m54.104484461s) [noise=true , address=104.36.82.5:4838 , piece_id=WK3ALBOEMWWTVEA4FKCS352SZ6YYB35LELURNWXQZK32K4HYDRNA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178252042 (377h14m54.104546733s) [noise=true , address=104.36.82.7:4909 , piece_id=ZRHLWDQ5KIDRWPJ5FXAANQTWERKXCR2PPEGN442IOSYSRFAG7VZA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178273922 (377h14m54.104568608s) [noise=true , address=104.36.82.10:4078 , piece_id=ZFVUQAEV6DDFLPKCZA2XJ2BZHMMM745LGB4JRZJZQPG72WECGMDQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178305983 (377h14m54.104600682s) [seg_pos={0 0}, upload=302109]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 02:26:27.178428768 (377h14m54.104723465s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:26:27.178455810 (377h14m54.104750501s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:26:27.232166184 (377h14m54.158460883s) [upload=302109]: Done uploading segments...
Jul 9 02:26:27.232188444 (377h14m54.158483123s) [upload=302109]: Upload finished. err: <nil>
Duration: 431.551285ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
4 out of 60 is failed (threshold=3) [..................X.....X...X....................X..........] duration:
540.241514ms
FAIL (2025-07-09 02:32:08 UTC)
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
02:25:27.135824454 (377h13m54.062119141s) [noise=true , address=104.36.82.12:4150 , piece_id=DRG5SBA3EE4KVQPEQOJZ6SS537SFWYFUYQXOAQMWYTT6FPVSJIPQ, seg_pos={0 0}, upload=302099]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: context canceled; context canceled
Jul 9 02:25:27.135860079 (377h13m54.062154769s) [seg_pos={0 0}, upload=302099]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 02:25:27.135957283 (377h13m54.062251971s) [upload=302099]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:25:27.135964211 (377h13m54.062258897s) [upload=302099]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:25:27.199410713 (377h13m54.125705389s) [upload=302099]: Done uploading segments...
Jul 9 02:25:27.199427901 (377h13m54.125722577s) [upload=302099]: Upload finished. err: <nil>
Jul 9 02:25:57.289077655 (377h14m24.215372329s) [upload=302109]: Starting upload
Jul 9 02:25:57.289186750 (377h14m24.215481424s) [upload=302109]: Done starting upload
Jul 9 02:25:57.289444851 (377h14m24.215739548s) [upload=302109]: Uploading segments...
Jul 9 02:25:57.289623846 (377h14m24.215918535s) [upload=302109]: Got next segment. Inline: false
Jul 9 02:25:57.289696741 (377h14m24.215991432s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:25:57.289703375 (377h14m24.215998067s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:25:57.318313152 (377h14m24.244607838s) [seg_pos={0 0}, upload=302109]: Begin upload segment...
Jul 9 02:25:57.318363286 (377h14m24.244657964s) [seg_pos={0 0}, upload=302109]: Done begin upload segment.
Jul 9 02:25:57.318386772 (377h14m24.244681463s) [noise=true , address=77.90.31.11:4553 , piece_id=TYW7NBJWACIT7OQS7A4WGXUPR3MB2FKAN2OMGAZJERB4X2DLAO2Q, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318412127 (377h14m24.244706825s) [noise=true , address=77.90.31.10:4518 , piece_id=VJ4HWM2X4VSRSUCQL5QLMBYPFBYV4JYS5ZXCBCB6MTMB76EHZFGQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318822149 (377h14m24.245116841s) [noise=true , address=45.91.116.9:4263 , piece_id=5YILAJ5BZBGOURSZE343ST4RMOTWM6ZOB3EEQQWL5WUNYWOGSAPA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318965862 (377h14m24.24526056s) [noise=true , address=45.91.116.8:4045 , piece_id=KM6ZYTHR6JGTARKZQZYIESPKBEU5H2HQFA7OARIKD2ZXDHPFSMGA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.319187516 (377h14m24.245482203s) [noise=true , address=104.36.82.7:4909 , piece_id=ZRHLWDQ5KIDRWPJ5FXAANQTWERKXCR2PPEGN442IOSYSRFAG7VZA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.319523815 (377h14m24.245818512s) [noise=true , address=45.91.116.14:4191 , piece_id=GTR6T4OTGZTEZ76FWQZ46OCH5WB3O5X27EN4AV33ZJ7D5HX4535A, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.320556632 (377h14m24.24685133s) [noise=true , address=104.36.82.2:4732 , piece_id=DJIGHJMJRAULH2L4YT2WGRXPRQJMNKZG5LICHPSKOCHUL6QDV6BQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.321019141 (377h14m24.247313835s) [noise=true , address=77.90.31.17:4770 , piece_id=V36UBBIYOFJC6KFVVBXEVT4QBUIWN257D3YSDDRAGZVXH7ZXHTBA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.322591797 (377h14m24.248886493s) [noise=true , address=77.90.31.13:4627 , piece_id=HULEKFVZNDNKS4SWWGTJUHYZWWYPDCER5W4GFCVLLDL65PAHULFQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323120451 (377h14m24.249415138s) [noise=true , address=45.91.116.4:4939 , piece_id=ANA3EZU7OZWELNG4QEMTEAYZY5H3C3H4IJFT4XKPUB7WIF44FEVQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323544591 (377h14m24.249839288s) [noise=true , address=45.91.116.7:4011 , piece_id=VDOJRN6ACK4LO44BQTULO6BB3CLAVQI7YIC6XLR32CZV4UJSPE6Q, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323950189 (377h14m24.250244889s) [noise=true , address=104.36.82.10:4078 , piece_id=ZFVUQAEV6DDFLPKCZA2XJ2BZHMMM745LGB4JRZJZQPG72WECGMDQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.326585810 (377h14m24.252880507s) [noise=true , address=45.91.116.5:4297 , piece_id=MSDWDLY5PJHQVI5565EKANJVSTMZFEUD2OTYZ52C65XQUICNMC3A, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.327871214 (377h14m24.254165904s) [noise=true , address=104.36.82.6:4875 , piece_id=C275F2JPVAF52E34CEHT5QXCHUFGIQZ7XGMOAOUOT4RA7YOHRQLQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.328605288 (377h14m24.254899986s) [noise=true , address=104.36.82.5:4838 , piece_id=WK3ALBOEMWWTVEA4FKCS352SZ6YYB35LELURNWXQZK32K4HYDRNA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.335409087 (377h14m24.261703784s) [noise=true , address=45.91.116.6:4973 , piece_id=Z7HQQ2O5QUSJKT4EMIDMEI5PIAEHNVRYX7MALUR2GQULRFIBUOKA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.475344637 (377h14m24.401639326s) [upload=302109]: Next returned nil segment
Jul 9 02:25:57.475376181 (377h14m24.401670879s) [upload=302109]: Waiting for error group managing segments...
Jul 9 02:25:57.477269278 (377h14m24.403563958s) [noise=true , address=45.91.116.7:4011 , piece_id=VDOJRN6ACK4LO44BQTULO6BB3CLAVQI7YIC6XLR32CZV4UJSPE6Q, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.477736179 (377h14m24.40403086s) [noise=true , address=45.91.116.9:4263 , piece_id=5YILAJ5BZBGOURSZE343ST4RMOTWM6ZOB3EEQQWL5WUNYWOGSAPA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.478396652 (377h14m24.404691332s) [noise=true , address=77.90.31.10:4518 , piece_id=VJ4HWM2X4VSRSUCQL5QLMBYPFBYV4JYS5ZXCBCB6MTMB76EHZFGQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.478919538 (377h14m24.405214224s) [noise=true , address=45.91.116.14:4191 , piece_id=GTR6T4OTGZTEZ76FWQZ46OCH5WB3O5X27EN4AV33ZJ7D5HX4535A, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481426716 (377h14m24.407721396s) [noise=true , address=45.91.116.4:4939 , piece_id=ANA3EZU7OZWELNG4QEMTEAYZY5H3C3H4IJFT4XKPUB7WIF44FEVQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481718373 (377h14m24.408013051s) [noise=true , address=45.91.116.5:4297 , piece_id=MSDWDLY5PJHQVI5565EKANJVSTMZFEUD2OTYZ52C65XQUICNMC3A, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481930356 (377h14m24.408225033s) [noise=true , address=77.90.31.17:4770 , piece_id=V36UBBIYOFJC6KFVVBXEVT4QBUIWN257D3YSDDRAGZVXH7ZXHTBA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.482376147 (377h14m24.408670821s) [noise=true , address=45.91.116.8:4045 , piece_id=KM6ZYTHR6JGTARKZQZYIESPKBEU5H2HQFA7OARIKD2ZXDHPFSMGA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.482439192 (377h14m24.408733868s) [noise=true , address=45.91.116.6:4973 , piece_id=Z7HQQ2O5QUSJKT4EMIDMEI5PIAEHNVRYX7MALUR2GQULRFIBUOKA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.483047970 (377h14m24.409342644s) [noise=true , address=77.90.31.13:4627 , piece_id=HULEKFVZNDNKS4SWWGTJUHYZWWYPDCER5W4GFCVLLDL65PAHULFQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.497788632 (377h14m24.424083312s) [noise=true , address=77.90.31.11:4553 , piece_id=TYW7NBJWACIT7OQS7A4WGXUPR3MB2FKAN2OMGAZJERB4X2DLAO2Q, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:16.688921171 (377h14m43.615215851s) [noise=true , address=104.36.82.6:4875 , piece_id=C275F2JPVAF52E34CEHT5QXCHUFGIQZ7XGMOAOUOT4RA7YOHRQLQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:27.177893955 (377h14m54.104188635s) [noise=true , address=104.36.82.2:4732 , piece_id=DJIGHJMJRAULH2L4YT2WGRXPRQJMNKZG5LICHPSKOCHUL6QDV6BQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:27.177985584 (377h14m54.104280281s) [seg_pos={0 0}, upload=302109]: Segment reached optimal threshold of 13 pieces.
Jul 9 02:26:27.178189789 (377h14m54.104484461s) [noise=true , address=104.36.82.5:4838 , piece_id=WK3ALBOEMWWTVEA4FKCS352SZ6YYB35LELURNWXQZK32K4HYDRNA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178252042 (377h14m54.104546733s) [noise=true , address=104.36.82.7:4909 , piece_id=ZRHLWDQ5KIDRWPJ5FXAANQTWERKXCR2PPEGN442IOSYSRFAG7VZA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178273922 (377h14m54.104568608s) [noise=true , address=104.36.82.10:4078 , piece_id=ZFVUQAEV6DDFLPKCZA2XJ2BZHMMM745LGB4JRZJZQPG72WECGMDQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178305983 (377h14m54.104600682s) [seg_pos={0 0}, upload=302109]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 02:26:27.178428768 (377h14m54.104723465s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:26:27.178455810 (377h14m54.104750501s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:26:27.232166184 (377h14m54.158460883s) [upload=302109]: Done uploading segments...
Jul 9 02:26:27.232188444 (377h14m54.158483123s) [upload=302109]: Upload finished. err: <nil>
Duration: 540.269018ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
40 out of 60 is failed (threshold=3) [.XXX.XXX.X.X.XXXXXXX..XX....XX.XXX.XXX.XX.XXXXXX.XXXXX.XX...] duration:
52.708791101s
FAIL (2025-07-09 02:32:01 UTC)
operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 52.708811611s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.......................................X....................] duration:
695.974125ms
FAIL (2025-07-09 02:31:42 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
3:56.263910643 (377h2m23.19020535s) [upload=301954]: Got next segment. Inline: false
Jul 9 02:13:56.263954358 (377h2m23.190249034s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:13:56.263959891 (377h2m23.190254572s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:13:56.302204249 (377h2m23.228498944s) [seg_pos={0 0}, upload=301954]: Begin upload segment...
Jul 9 02:13:56.302291587 (377h2m23.22858628s) [seg_pos={0 0}, upload=301954]: Done begin upload segment.
Jul 9 02:13:56.302338947 (377h2m23.228633682s) [noise=true , address=77.90.31.14:4689 , piece_id=LW7OT74AJCCC3TEKY6OZI553YK6YQFQ6C7PORQB53E7FUS24KQKQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.302352599 (377h2m23.228647272s) [noise=true , address=104.36.82.8:4986 , piece_id=KPY5JVLSAEB5GUKUJJSIF5M6IIHTJISEYYAXCGUSJ7OLI5QCIBNQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.302450837 (377h2m23.228745531s) [noise=true , address=77.90.31.16:4739 , piece_id=H6UCV6RURPXKC2H3YGAT2OXV2VZMW2DWIRG6TFD6CVXKOVMDP2VQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.302958774 (377h2m23.229253667s) [noise=true , address=77.90.31.13:4640 , piece_id=TPPN75DX3UCBM346CSVJN3GSOA47QV4FVPYSSAHWCJYEM42OCVLQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303069086 (377h2m23.229363782s) [noise=true , address=104.36.82.17:4345 , piece_id=YI27O5WLBYO3TPCX4MVUATFORY6U3EQMDKQHGKRHIZMTG7WGFB2A, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303495994 (377h2m23.22979069s) [noise=true , address=104.36.82.36:4644 , piece_id=K2G3MAPD7TWYZPKNHQBMEPJRIQDLE75FYVPAZFX4I3GJ4IUAA7XQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303575840 (377h2m23.229870536s) [noise=true , address=77.90.31.21:4898 , piece_id=75XXVNT4EV4D5UTX7O5XFL44IZOYY5LZSI2VPFQZPRERL6W5M23A, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303949062 (377h2m23.230245641s) [noise=true , address=79.127.201.215:4339 , piece_id=XDCTYE2SVI6RAPCXTRVANFCZYWJDZXLBHHJ47NQY6V7D36DEFPGA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304060410 (377h2m23.230355106s) [noise=true , address=104.36.82.41:4768 , piece_id=BGMF33OKUZSZKGIHJYI3RNXEBDE54TX67DYLJ7JCZMRNK6ZAU22Q, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304407993 (377h2m23.230702685s) [noise=true , address=77.90.31.10:4525 , piece_id=GPVSIZIBZWTUCP5RFYVPFYIYIL2LWU5PU3UASQVRQLY7SSSFMQBA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304548115 (377h2m23.230842812s) [noise=true , address=104.36.82.23:4464 , piece_id=JCJ4MGEZIIZ7YCTM6QXC5ZJ7PI4OEJBOMHZYBU57D2SNGWJWB5CQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304795164 (377h2m23.231089862s) [noise=true , address=77.90.31.18:4838 , piece_id=TGWYXA3AF7LDG4KPVCRNEYFDADAG6TQ5DAX7JJVW5BLVVVNECILA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305012510 (377h2m23.231307211s) [noise=true , address=104.36.82.34:4554 , piece_id=57K5SDV3YEGLVMSJGSLXNKEWOLETXE6CWBM2NPVCNKLFHPVQCJBQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305113737 (377h2m23.231408427s) [noise=true , address=104.36.82.4:4834 , piece_id=YLYIASUPNRTK7KOMRYLN7OQLQBPWC7WFGCPDGUVIOEQJWQQ7A7GA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305383755 (377h2m23.231678434s) [noise=true , address=104.36.82.38:4698 , piece_id=TSOXJNQEWODCWFEITH4IYKZI7A75UZEXRQPIQO7LMP6LT3WLNXUA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305483822 (377h2m23.231778517s) [noise=true , address=104.36.82.42:4830 , piece_id=ENYQTUCIF7OWQPCXTYS24OC5QT63ZJJXOY652GLIG7OQPJEQCGGA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305954019 (377h2m23.232248718s) [noise=true , address=77.90.31.21:4907 , piece_id=2EUWATHUQNZ77BJV65ZFKR5WL4D3AUURLBQ32XR2IVBA7P4TITOQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.306425517 (377h2m23.232720213s) [noise=true , address=104.36.82.22:4541 , piece_id=XN4HDFDNZ4YUJTDPOTMMDFX4FE2YVZDGCIMHEMJGWXRPNHQ6EU7A, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.375470790 (377h2m23.301765481s) [upload=301954]: Next returned nil segment
Jul 9 02:13:56.375489934 (377h2m23.30178463s) [upload=301954]: Waiting for error group managing segments...
Jul 9 02:13:56.381456859 (377h2m23.307751539s) [noise=true , address=77.90.31.10:4525 , piece_id=GPVSIZIBZWTUCP5RFYVPFYIYIL2LWU5PU3UASQVRQLY7SSSFMQBA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.381956907 (377h2m23.308251586s) [noise=true , address=77.90.31.14:4689 , piece_id=LW7OT74AJCCC3TEKY6OZI553YK6YQFQ6C7PORQB53E7FUS24KQKQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.382837088 (377h2m23.309131761s) [noise=true , address=77.90.31.21:4898 , piece_id=75XXVNT4EV4D5UTX7O5XFL44IZOYY5LZSI2VPFQZPRERL6W5M23A, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.393489125 (377h2m23.319783812s) [noise=true , address=77.90.31.21:4907 , piece_id=2EUWATHUQNZ77BJV65ZFKR5WL4D3AUURLBQ32XR2IVBA7P4TITOQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.404692801 (377h2m23.330987481s) [noise=true , address=77.90.31.16:4739 , piece_id=H6UCV6RURPXKC2H3YGAT2OXV2VZMW2DWIRG6TFD6CVXKOVMDP2VQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.404986115 (377h2m23.331280795s) [noise=true , address=77.90.31.18:4838 , piece_id=TGWYXA3AF7LDG4KPVCRNEYFDADAG6TQ5DAX7JJVW5BLVVVNECILA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.433785086 (377h2m23.360079766s) [noise=true , address=77.90.31.13:4640 , piece_id=TPPN75DX3UCBM346CSVJN3GSOA47QV4FVPYSSAHWCJYEM42OCVLQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.598499747 (377h2m23.524794427s) [noise=true , address=79.127.201.215:4339 , piece_id=XDCTYE2SVI6RAPCXTRVANFCZYWJDZXLBHHJ47NQY6V7D36DEFPGA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:11.107785584 (377h2m38.034080273s) [noise=true , address=104.36.82.22:4541 , piece_id=XN4HDFDNZ4YUJTDPOTMMDFX4FE2YVZDGCIMHEMJGWXRPNHQ6EU7A, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:12.515567029 (377h2m39.441861713s) [noise=true , address=104.36.82.17:4345 , piece_id=YI27O5WLBYO3TPCX4MVUATFORY6U3EQMDKQHGKRHIZMTG7WGFB2A, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:12.923090587 (377h2m39.849385267s) [noise=true , address=104.36.82.8:4986 , piece_id=KPY5JVLSAEB5GUKUJJSIF5M6IIHTJISEYYAXCGUSJ7OLI5QCIBNQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:13.453172659 (377h2m40.379467346s) [noise=true , address=104.36.82.38:4698 , piece_id=TSOXJNQEWODCWFEITH4IYKZI7A75UZEXRQPIQO7LMP6LT3WLNXUA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:14.416524856 (377h2m41.342819534s) [noise=true , address=104.36.82.34:4554 , piece_id=57K5SDV3YEGLVMSJGSLXNKEWOLETXE6CWBM2NPVCNKLFHPVQCJBQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:14.846730699 (377h2m41.773025379s) [noise=true , address=104.36.82.23:4464 , piece_id=JCJ4MGEZIIZ7YCTM6QXC5ZJ7PI4OEJBOMHZYBU57D2SNGWJWB5CQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:14.846763777 (377h2m41.773058463s) [seg_pos={0 0}, upload=301954]: Segment reached optimal threshold of 14 pieces.
Jul 9 02:14:14.846894905 (377h2m41.773189582s) [noise=true , address=104.36.82.36:4644 , piece_id=K2G3MAPD7TWYZPKNHQBMEPJRIQDLE75FYVPAZFX4I3GJ4IUAA7XQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1oYHbb73ACitbfkkP5P6cmRyTt6kmaS1y49G4vbsvTvVDwByhT): protocol: expected piece hash; context canceled; EOF
Jul 9 02:14:14.846948435 (377h2m41.773243131s) [noise=true , address=104.36.82.4:4834 , piece_id=YLYIASUPNRTK7KOMRYLN7OQLQBPWC7WFGCPDGUVIOEQJWQQ7A7GA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Z241DCYA5Wig2TdEogxTpY6FzVmFbnKtmHcTwxLxfcZafgooy): protocol: expected piece hash; context canceled; EOF
Jul 9 02:14:14.847094561 (377h2m41.773389249s) [noise=true , address=104.36.82.42:4830 , piece_id=ENYQTUCIF7OWQPCXTYS24OC5QT63ZJJXOY652GLIG7OQPJEQCGGA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:142ocSHHtv8BpSebXFNk5qk1Y23a8pbFAZqE3cq6i3RMX3dPw9): protocol: expected piece hash; context canceled; EOF
Jul 9 02:14:14.847127825 (377h2m41.773422511s) [noise=true , address=104.36.82.41:4768 , piece_id=BGMF33OKUZSZKGIHJYI3RNXEBDE54TX67DYLJ7JCZMRNK6ZAU22Q, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: write tcp 159.223.143.39:43796->104.36.82.41:4768: use of closed network connection; write tcp 159.223.143.39:43796->104.36.82.41:4768: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:43796->104.36.82.41:4768: use of closed network connection
Jul 9 02:14:14.847142999 (377h2m41.773437687s) [seg_pos={0 0}, upload=301954]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jul 9 02:14:14.847204962 (377h2m41.773499652s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:14:14.847209974 (377h2m41.773504671s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:14:14.909581364 (377h2m41.83587606s) [upload=301954]: Done uploading segments...
Jul 9 02:14:14.909605483 (377h2m41.835900172s) [upload=301954]: Upload finished. err: <nil>
Duration: 696.006581ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
19 out of 60 is failed (threshold=3) [....X....X....X.........X....X.X..XX.XX..X.X.X.....XXX.XX.X.] duration:
1.090261263s
FAIL (2025-07-09 02:31:35 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.09028297s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
4 out of 60 is failed (threshold=3) [...................X.....X...X....................X.........] duration:
445.330254ms
FAIL (2025-07-09 02:31:27 UTC)
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
02:25:27.135824454 (377h13m54.062119141s) [noise=true , address=104.36.82.12:4150 , piece_id=DRG5SBA3EE4KVQPEQOJZ6SS537SFWYFUYQXOAQMWYTT6FPVSJIPQ, seg_pos={0 0}, upload=302099]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: context canceled; context canceled
Jul 9 02:25:27.135860079 (377h13m54.062154769s) [seg_pos={0 0}, upload=302099]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 02:25:27.135957283 (377h13m54.062251971s) [upload=302099]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:25:27.135964211 (377h13m54.062258897s) [upload=302099]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:25:27.199410713 (377h13m54.125705389s) [upload=302099]: Done uploading segments...
Jul 9 02:25:27.199427901 (377h13m54.125722577s) [upload=302099]: Upload finished. err: <nil>
Jul 9 02:25:57.289077655 (377h14m24.215372329s) [upload=302109]: Starting upload
Jul 9 02:25:57.289186750 (377h14m24.215481424s) [upload=302109]: Done starting upload
Jul 9 02:25:57.289444851 (377h14m24.215739548s) [upload=302109]: Uploading segments...
Jul 9 02:25:57.289623846 (377h14m24.215918535s) [upload=302109]: Got next segment. Inline: false
Jul 9 02:25:57.289696741 (377h14m24.215991432s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:25:57.289703375 (377h14m24.215998067s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:25:57.318313152 (377h14m24.244607838s) [seg_pos={0 0}, upload=302109]: Begin upload segment...
Jul 9 02:25:57.318363286 (377h14m24.244657964s) [seg_pos={0 0}, upload=302109]: Done begin upload segment.
Jul 9 02:25:57.318386772 (377h14m24.244681463s) [noise=true , address=77.90.31.11:4553 , piece_id=TYW7NBJWACIT7OQS7A4WGXUPR3MB2FKAN2OMGAZJERB4X2DLAO2Q, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318412127 (377h14m24.244706825s) [noise=true , address=77.90.31.10:4518 , piece_id=VJ4HWM2X4VSRSUCQL5QLMBYPFBYV4JYS5ZXCBCB6MTMB76EHZFGQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318822149 (377h14m24.245116841s) [noise=true , address=45.91.116.9:4263 , piece_id=5YILAJ5BZBGOURSZE343ST4RMOTWM6ZOB3EEQQWL5WUNYWOGSAPA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318965862 (377h14m24.24526056s) [noise=true , address=45.91.116.8:4045 , piece_id=KM6ZYTHR6JGTARKZQZYIESPKBEU5H2HQFA7OARIKD2ZXDHPFSMGA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.319187516 (377h14m24.245482203s) [noise=true , address=104.36.82.7:4909 , piece_id=ZRHLWDQ5KIDRWPJ5FXAANQTWERKXCR2PPEGN442IOSYSRFAG7VZA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.319523815 (377h14m24.245818512s) [noise=true , address=45.91.116.14:4191 , piece_id=GTR6T4OTGZTEZ76FWQZ46OCH5WB3O5X27EN4AV33ZJ7D5HX4535A, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.320556632 (377h14m24.24685133s) [noise=true , address=104.36.82.2:4732 , piece_id=DJIGHJMJRAULH2L4YT2WGRXPRQJMNKZG5LICHPSKOCHUL6QDV6BQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.321019141 (377h14m24.247313835s) [noise=true , address=77.90.31.17:4770 , piece_id=V36UBBIYOFJC6KFVVBXEVT4QBUIWN257D3YSDDRAGZVXH7ZXHTBA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.322591797 (377h14m24.248886493s) [noise=true , address=77.90.31.13:4627 , piece_id=HULEKFVZNDNKS4SWWGTJUHYZWWYPDCER5W4GFCVLLDL65PAHULFQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323120451 (377h14m24.249415138s) [noise=true , address=45.91.116.4:4939 , piece_id=ANA3EZU7OZWELNG4QEMTEAYZY5H3C3H4IJFT4XKPUB7WIF44FEVQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323544591 (377h14m24.249839288s) [noise=true , address=45.91.116.7:4011 , piece_id=VDOJRN6ACK4LO44BQTULO6BB3CLAVQI7YIC6XLR32CZV4UJSPE6Q, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323950189 (377h14m24.250244889s) [noise=true , address=104.36.82.10:4078 , piece_id=ZFVUQAEV6DDFLPKCZA2XJ2BZHMMM745LGB4JRZJZQPG72WECGMDQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.326585810 (377h14m24.252880507s) [noise=true , address=45.91.116.5:4297 , piece_id=MSDWDLY5PJHQVI5565EKANJVSTMZFEUD2OTYZ52C65XQUICNMC3A, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.327871214 (377h14m24.254165904s) [noise=true , address=104.36.82.6:4875 , piece_id=C275F2JPVAF52E34CEHT5QXCHUFGIQZ7XGMOAOUOT4RA7YOHRQLQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.328605288 (377h14m24.254899986s) [noise=true , address=104.36.82.5:4838 , piece_id=WK3ALBOEMWWTVEA4FKCS352SZ6YYB35LELURNWXQZK32K4HYDRNA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.335409087 (377h14m24.261703784s) [noise=true , address=45.91.116.6:4973 , piece_id=Z7HQQ2O5QUSJKT4EMIDMEI5PIAEHNVRYX7MALUR2GQULRFIBUOKA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.475344637 (377h14m24.401639326s) [upload=302109]: Next returned nil segment
Jul 9 02:25:57.475376181 (377h14m24.401670879s) [upload=302109]: Waiting for error group managing segments...
Jul 9 02:25:57.477269278 (377h14m24.403563958s) [noise=true , address=45.91.116.7:4011 , piece_id=VDOJRN6ACK4LO44BQTULO6BB3CLAVQI7YIC6XLR32CZV4UJSPE6Q, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.477736179 (377h14m24.40403086s) [noise=true , address=45.91.116.9:4263 , piece_id=5YILAJ5BZBGOURSZE343ST4RMOTWM6ZOB3EEQQWL5WUNYWOGSAPA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.478396652 (377h14m24.404691332s) [noise=true , address=77.90.31.10:4518 , piece_id=VJ4HWM2X4VSRSUCQL5QLMBYPFBYV4JYS5ZXCBCB6MTMB76EHZFGQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.478919538 (377h14m24.405214224s) [noise=true , address=45.91.116.14:4191 , piece_id=GTR6T4OTGZTEZ76FWQZ46OCH5WB3O5X27EN4AV33ZJ7D5HX4535A, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481426716 (377h14m24.407721396s) [noise=true , address=45.91.116.4:4939 , piece_id=ANA3EZU7OZWELNG4QEMTEAYZY5H3C3H4IJFT4XKPUB7WIF44FEVQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481718373 (377h14m24.408013051s) [noise=true , address=45.91.116.5:4297 , piece_id=MSDWDLY5PJHQVI5565EKANJVSTMZFEUD2OTYZ52C65XQUICNMC3A, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481930356 (377h14m24.408225033s) [noise=true , address=77.90.31.17:4770 , piece_id=V36UBBIYOFJC6KFVVBXEVT4QBUIWN257D3YSDDRAGZVXH7ZXHTBA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.482376147 (377h14m24.408670821s) [noise=true , address=45.91.116.8:4045 , piece_id=KM6ZYTHR6JGTARKZQZYIESPKBEU5H2HQFA7OARIKD2ZXDHPFSMGA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.482439192 (377h14m24.408733868s) [noise=true , address=45.91.116.6:4973 , piece_id=Z7HQQ2O5QUSJKT4EMIDMEI5PIAEHNVRYX7MALUR2GQULRFIBUOKA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.483047970 (377h14m24.409342644s) [noise=true , address=77.90.31.13:4627 , piece_id=HULEKFVZNDNKS4SWWGTJUHYZWWYPDCER5W4GFCVLLDL65PAHULFQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.497788632 (377h14m24.424083312s) [noise=true , address=77.90.31.11:4553 , piece_id=TYW7NBJWACIT7OQS7A4WGXUPR3MB2FKAN2OMGAZJERB4X2DLAO2Q, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:16.688921171 (377h14m43.615215851s) [noise=true , address=104.36.82.6:4875 , piece_id=C275F2JPVAF52E34CEHT5QXCHUFGIQZ7XGMOAOUOT4RA7YOHRQLQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:27.177893955 (377h14m54.104188635s) [noise=true , address=104.36.82.2:4732 , piece_id=DJIGHJMJRAULH2L4YT2WGRXPRQJMNKZG5LICHPSKOCHUL6QDV6BQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:27.177985584 (377h14m54.104280281s) [seg_pos={0 0}, upload=302109]: Segment reached optimal threshold of 13 pieces.
Jul 9 02:26:27.178189789 (377h14m54.104484461s) [noise=true , address=104.36.82.5:4838 , piece_id=WK3ALBOEMWWTVEA4FKCS352SZ6YYB35LELURNWXQZK32K4HYDRNA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178252042 (377h14m54.104546733s) [noise=true , address=104.36.82.7:4909 , piece_id=ZRHLWDQ5KIDRWPJ5FXAANQTWERKXCR2PPEGN442IOSYSRFAG7VZA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178273922 (377h14m54.104568608s) [noise=true , address=104.36.82.10:4078 , piece_id=ZFVUQAEV6DDFLPKCZA2XJ2BZHMMM745LGB4JRZJZQPG72WECGMDQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178305983 (377h14m54.104600682s) [seg_pos={0 0}, upload=302109]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 02:26:27.178428768 (377h14m54.104723465s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:26:27.178455810 (377h14m54.104750501s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:26:27.232166184 (377h14m54.158460883s) [upload=302109]: Done uploading segments...
Jul 9 02:26:27.232188444 (377h14m54.158483123s) [upload=302109]: Upload finished. err: <nil>
Duration: 445.357077ms
core
uplink
us-select-1
|