Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [............................X...............................] duration:
1.71911373s
FAIL (2025-07-02 17:40:10 UTC)
orbiter warning: uplink: metaclient: manager closed: closed: read tcp 159.223.143.39:49282->34.150.199.48:7777: read: connection reset by peer
Duration: 1.71914849s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............................X..............................] duration:
1.81880205s
FAIL (2025-07-02 17:39:48 UTC)
orbiter warning: uplink: metaclient: manager closed: closed: read tcp 159.223.143.39:49282->34.150.199.48:7777: read: connection reset by peer
Duration: 1.818832566s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............................X.............................] duration:
1.726600315s
FAIL (2025-07-02 17:38:50 UTC)
orbiter warning: uplink: metaclient: manager closed: closed: read tcp 159.223.143.39:49282->34.150.199.48:7777: read: connection reset by peer
Duration: 1.726630336s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............................X............................] duration:
1.753606928s
FAIL (2025-07-02 17:37:39 UTC)
orbiter warning: uplink: metaclient: manager closed: closed: read tcp 159.223.143.39:49282->34.150.199.48:7777: read: connection reset by peer
Duration: 1.753637247s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [................................X...........................] duration:
1.719384299s
FAIL (2025-07-02 17:36:34 UTC)
orbiter warning: uplink: metaclient: manager closed: closed: read tcp 159.223.143.39:49282->34.150.199.48:7777: read: connection reset by peer
Duration: 1.719470036s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
559.808046ms
FAIL (2025-07-02 17:36:00 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
led; ecclient: upload cut due to slow connection (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN): protocol: expected piece hash; context canceled; EOF
Jul 2 16:51:39.852164074 (223h40m6.778458767s) [noise=true , address=104.36.82.12:4150 , piece_id=QHXVON3LSBWYG5AWPVZ45KOTJJPUXIIAXGED7OHG65V5V3IJPULA, seg_pos={0 0}, upload=179153]: Done uploading piece. err: <nil>
Jul 2 16:51:39.852220651 (223h40m6.778515346s) [seg_pos={0 0}, upload=179153]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 2 16:51:39.852324711 (223h40m6.778619405s) [upload=179153]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 16:51:39.852329895 (223h40m6.778624588s) [upload=179153]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 16:51:39.945155283 (223h40m6.871449972s) [upload=179153]: Done uploading segments...
Jul 2 16:51:39.945173208 (223h40m6.871467885s) [upload=179153]: Upload finished. err: <nil>
Jul 2 16:52:25.999553770 (223h40m52.925848461s) [upload=179165]: Starting upload
Jul 2 16:52:25.999676452 (223h40m52.92597114s) [upload=179165]: Done starting upload
Jul 2 16:52:25.999808777 (223h40m52.926103458s) [upload=179165]: Uploading segments...
Jul 2 16:52:26.000049412 (223h40m52.926344119s) [upload=179165]: Got next segment. Inline: false
Jul 2 16:52:26.000208665 (223h40m52.926503358s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 2 16:52:26.000216887 (223h40m52.926511587s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 2 16:52:26.029546629 (223h40m52.95584132s) [seg_pos={0 0}, upload=179165]: Begin upload segment...
Jul 2 16:52:26.029596311 (223h40m52.955890985s) [seg_pos={0 0}, upload=179165]: Done begin upload segment.
Jul 2 16:52:26.029630299 (223h40m52.955924986s) [noise=true , address=45.91.116.12:4226 , piece_id=NFB2MT72D7MLPOY3A6MVO2QQBUFVYJPKN7D5XJQLQCCMYNQ6WJOA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.030073195 (223h40m52.956367894s) [noise=true , address=77.90.31.14:4661 , piece_id=B5E2MZ5JW7Z5AG4GFQ4QGMAFW7ZMKWKRFU276Z6556JFWKYVLXTQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.030398264 (223h40m52.956692955s) [noise=true , address=77.90.31.15:4698 , piece_id=VDVQDBYDQYRMVP7ZUPSEFC2R5UF2LVW45G23JURMYKS5IT6RBQFQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.030716003 (223h40m52.957010692s) [noise=true , address=45.91.116.11:4118 , piece_id=CSQO6I5J4BW4NUAOFH3QDF6VEJGACWOSYNTNGXVELK2HLXIMX4YQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.031068785 (223h40m52.957363472s) [noise=true , address=45.91.116.5:4298 , piece_id=JUYTY35XKYPNJYE7X3R7UMINTVMXDMNM6YZSZORJWATTVDBZFOMQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.031405588 (223h40m52.957700282s) [noise=true , address=104.36.82.8:4970 , piece_id=X7KKQJYHVNI5LJDKGYNV5JHHCEGF4QSHFRKMSBLRE5W3W2ITAAHQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.031730040 (223h40m52.958024715s) [noise=true , address=104.36.82.12:4149 , piece_id=V5265CLKUYZQN5B663WB4CFR3SXR2OA2BT53OAVCLFZJJ2YY6U4A, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.032049179 (223h40m52.958343866s) [noise=true , address=104.36.82.6:4874 , piece_id=XVVQN3VX4I2KS4CIXEADBSFYBVYTVEWS5GMXAHE2LWDJGD2A2KAA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.032432775 (223h40m52.958727467s) [noise=true , address=104.36.82.16:4294 , piece_id=36S2PXQADEAQPWSBMQC4XEAH6H5PDA4AO34JI37VD4KPJOU2WVTA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.032754164 (223h40m52.959048852s) [noise=true , address=77.90.31.19:4843 , piece_id=J74IERT5EP2VDSXY5PNWWBVTHQEXT4CWGLWP72Y3CQLOTSI6XZDA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.033080515 (223h40m52.959375213s) [noise=true , address=104.36.82.11:4115 , piece_id=LOK75B4Y2M3DMRJAHNYCVE46DELAAVN4SFP35AWPFESLUJT2V6KQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.033386908 (223h40m52.959681597s) [noise=true , address=104.36.82.14:4222 , piece_id=3HZK5YRWL2KXKY4E7XASRI2UIPCJA72JSJSTGBLPA4ZBP3MGBEOQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.033709080 (223h40m52.960003768s) [noise=true , address=77.90.31.21:4878 , piece_id=QJARHOYLDCFGYTTCD2EYCH43X3EARFK3Q3C7SCZ33X5T6CTLYDMQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.034030732 (223h40m52.96032542s) [noise=true , address=104.36.82.3:4767 , piece_id=U4L7ULC2R5OKDDL6BGDSEQUZNTVMQSGXJ56CGVKHNCCHTMIIDXDQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.034335911 (223h40m52.960630599s) [noise=true , address=45.91.116.8:4047 , piece_id=IJZDAO2K7KADVBNZGHLACI2ARTDDYOHIC742L2IUJT2WFAMMIGSQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.034644696 (223h40m52.960939386s) [noise=true , address=104.36.82.10:4079 , piece_id=JELVGAGG2NF2OSMKXQAV6DIXDTUFIER5WX4KTCJFQQ5P35EAMOEA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.148244089 (223h40m53.074538765s) [upload=179165]: Next returned nil segment
Jul 2 16:52:26.148268128 (223h40m53.074562811s) [upload=179165]: Waiting for error group managing segments...
Jul 2 16:52:26.155850465 (223h40m53.082145153s) [noise=true , address=45.91.116.5:4298 , piece_id=JUYTY35XKYPNJYE7X3R7UMINTVMXDMNM6YZSZORJWATTVDBZFOMQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.156432201 (223h40m53.082726892s) [noise=true , address=77.90.31.14:4661 , piece_id=B5E2MZ5JW7Z5AG4GFQ4QGMAFW7ZMKWKRFU276Z6556JFWKYVLXTQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.156943265 (223h40m53.083237956s) [noise=true , address=104.36.82.16:4294 , piece_id=36S2PXQADEAQPWSBMQC4XEAH6H5PDA4AO34JI37VD4KPJOU2WVTA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.158591570 (223h40m53.084886251s) [noise=true , address=104.36.82.3:4767 , piece_id=U4L7ULC2R5OKDDL6BGDSEQUZNTVMQSGXJ56CGVKHNCCHTMIIDXDQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.159458953 (223h40m53.085753634s) [noise=true , address=45.91.116.8:4047 , piece_id=IJZDAO2K7KADVBNZGHLACI2ARTDDYOHIC742L2IUJT2WFAMMIGSQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.161102375 (223h40m53.087397055s) [noise=true , address=45.91.116.12:4226 , piece_id=NFB2MT72D7MLPOY3A6MVO2QQBUFVYJPKN7D5XJQLQCCMYNQ6WJOA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.161556163 (223h40m53.087850838s) [noise=true , address=104.36.82.12:4149 , piece_id=V5265CLKUYZQN5B663WB4CFR3SXR2OA2BT53OAVCLFZJJ2YY6U4A, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.162013197 (223h40m53.088307874s) [noise=true , address=104.36.82.14:4222 , piece_id=3HZK5YRWL2KXKY4E7XASRI2UIPCJA72JSJSTGBLPA4ZBP3MGBEOQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.162459436 (223h40m53.088754112s) [noise=true , address=77.90.31.15:4698 , piece_id=VDVQDBYDQYRMVP7ZUPSEFC2R5UF2LVW45G23JURMYKS5IT6RBQFQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.162920549 (223h40m53.089215241s) [noise=true , address=77.90.31.21:4878 , piece_id=QJARHOYLDCFGYTTCD2EYCH43X3EARFK3Q3C7SCZ33X5T6CTLYDMQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.163747035 (223h40m53.090041713s) [noise=true , address=104.36.82.11:4115 , piece_id=LOK75B4Y2M3DMRJAHNYCVE46DELAAVN4SFP35AWPFESLUJT2V6KQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.167885784 (223h40m53.094180467s) [noise=true , address=104.36.82.6:4874 , piece_id=XVVQN3VX4I2KS4CIXEADBSFYBVYTVEWS5GMXAHE2LWDJGD2A2KAA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.168494969 (223h40m53.094789668s) [noise=true , address=104.36.82.10:4079 , piece_id=JELVGAGG2NF2OSMKXQAV6DIXDTUFIER5WX4KTCJFQQ5P35EAMOEA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.168525706 (223h40m53.094820396s) [seg_pos={0 0}, upload=179165]: Segment reached optimal threshold of 13 pieces.
Jul 2 16:52:26.168715378 (223h40m53.09501007s) [noise=true , address=77.90.31.19:4843 , piece_id=J74IERT5EP2VDSXY5PNWWBVTHQEXT4CWGLWP72Y3CQLOTSI6XZDA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: expected piece hash; context canceled; EOF
Jul 2 16:52:26.168766821 (223h40m53.095061509s) [noise=true , address=104.36.82.8:4970 , piece_id=X7KKQJYHVNI5LJDKGYNV5JHHCEGF4QSHFRKMSBLRE5W3W2ITAAHQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe): protocol: expected piece hash; context canceled; EOF
Jul 2 16:52:26.168816209 (223h40m53.095110907s) [noise=true , address=45.91.116.11:4118 , piece_id=CSQO6I5J4BW4NUAOFH3QDF6VEJGACWOSYNTNGXVELK2HLXIMX4YQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF
Jul 2 16:52:26.168851649 (223h40m53.095146335s) [seg_pos={0 0}, upload=179165]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 2 16:52:26.169058251 (223h40m53.095352944s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 16:52:26.169066272 (223h40m53.095360967s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 16:52:26.220481259 (223h40m53.146775936s) [upload=179165]: Done uploading segments...
Jul 2 16:52:26.220500093 (223h40m53.14679477s) [upload=179165]: Upload finished. err: <nil>
Duration: 559.836789ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................................X..........................] duration:
1.647604494s
FAIL (2025-07-02 17:35:34 UTC)
orbiter warning: uplink: metaclient: manager closed: closed: read tcp 159.223.143.39:49282->34.150.199.48:7777: read: connection reset by peer
Duration: 1.647642021s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................................X.........................] duration:
1.781178058s
FAIL (2025-07-02 17:35:12 UTC)
orbiter warning: uplink: metaclient: manager closed: closed: read tcp 159.223.143.39:49282->34.150.199.48:7777: read: connection reset by peer
Duration: 1.781206128s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
450.028551ms
FAIL (2025-07-02 17:35:08 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
led; ecclient: upload cut due to slow connection (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN): protocol: expected piece hash; context canceled; EOF
Jul 2 16:51:39.852164074 (223h40m6.778458767s) [noise=true , address=104.36.82.12:4150 , piece_id=QHXVON3LSBWYG5AWPVZ45KOTJJPUXIIAXGED7OHG65V5V3IJPULA, seg_pos={0 0}, upload=179153]: Done uploading piece. err: <nil>
Jul 2 16:51:39.852220651 (223h40m6.778515346s) [seg_pos={0 0}, upload=179153]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 2 16:51:39.852324711 (223h40m6.778619405s) [upload=179153]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 16:51:39.852329895 (223h40m6.778624588s) [upload=179153]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 16:51:39.945155283 (223h40m6.871449972s) [upload=179153]: Done uploading segments...
Jul 2 16:51:39.945173208 (223h40m6.871467885s) [upload=179153]: Upload finished. err: <nil>
Jul 2 16:52:25.999553770 (223h40m52.925848461s) [upload=179165]: Starting upload
Jul 2 16:52:25.999676452 (223h40m52.92597114s) [upload=179165]: Done starting upload
Jul 2 16:52:25.999808777 (223h40m52.926103458s) [upload=179165]: Uploading segments...
Jul 2 16:52:26.000049412 (223h40m52.926344119s) [upload=179165]: Got next segment. Inline: false
Jul 2 16:52:26.000208665 (223h40m52.926503358s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 2 16:52:26.000216887 (223h40m52.926511587s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 2 16:52:26.029546629 (223h40m52.95584132s) [seg_pos={0 0}, upload=179165]: Begin upload segment...
Jul 2 16:52:26.029596311 (223h40m52.955890985s) [seg_pos={0 0}, upload=179165]: Done begin upload segment.
Jul 2 16:52:26.029630299 (223h40m52.955924986s) [noise=true , address=45.91.116.12:4226 , piece_id=NFB2MT72D7MLPOY3A6MVO2QQBUFVYJPKN7D5XJQLQCCMYNQ6WJOA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.030073195 (223h40m52.956367894s) [noise=true , address=77.90.31.14:4661 , piece_id=B5E2MZ5JW7Z5AG4GFQ4QGMAFW7ZMKWKRFU276Z6556JFWKYVLXTQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.030398264 (223h40m52.956692955s) [noise=true , address=77.90.31.15:4698 , piece_id=VDVQDBYDQYRMVP7ZUPSEFC2R5UF2LVW45G23JURMYKS5IT6RBQFQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.030716003 (223h40m52.957010692s) [noise=true , address=45.91.116.11:4118 , piece_id=CSQO6I5J4BW4NUAOFH3QDF6VEJGACWOSYNTNGXVELK2HLXIMX4YQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.031068785 (223h40m52.957363472s) [noise=true , address=45.91.116.5:4298 , piece_id=JUYTY35XKYPNJYE7X3R7UMINTVMXDMNM6YZSZORJWATTVDBZFOMQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.031405588 (223h40m52.957700282s) [noise=true , address=104.36.82.8:4970 , piece_id=X7KKQJYHVNI5LJDKGYNV5JHHCEGF4QSHFRKMSBLRE5W3W2ITAAHQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.031730040 (223h40m52.958024715s) [noise=true , address=104.36.82.12:4149 , piece_id=V5265CLKUYZQN5B663WB4CFR3SXR2OA2BT53OAVCLFZJJ2YY6U4A, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.032049179 (223h40m52.958343866s) [noise=true , address=104.36.82.6:4874 , piece_id=XVVQN3VX4I2KS4CIXEADBSFYBVYTVEWS5GMXAHE2LWDJGD2A2KAA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.032432775 (223h40m52.958727467s) [noise=true , address=104.36.82.16:4294 , piece_id=36S2PXQADEAQPWSBMQC4XEAH6H5PDA4AO34JI37VD4KPJOU2WVTA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.032754164 (223h40m52.959048852s) [noise=true , address=77.90.31.19:4843 , piece_id=J74IERT5EP2VDSXY5PNWWBVTHQEXT4CWGLWP72Y3CQLOTSI6XZDA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.033080515 (223h40m52.959375213s) [noise=true , address=104.36.82.11:4115 , piece_id=LOK75B4Y2M3DMRJAHNYCVE46DELAAVN4SFP35AWPFESLUJT2V6KQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.033386908 (223h40m52.959681597s) [noise=true , address=104.36.82.14:4222 , piece_id=3HZK5YRWL2KXKY4E7XASRI2UIPCJA72JSJSTGBLPA4ZBP3MGBEOQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.033709080 (223h40m52.960003768s) [noise=true , address=77.90.31.21:4878 , piece_id=QJARHOYLDCFGYTTCD2EYCH43X3EARFK3Q3C7SCZ33X5T6CTLYDMQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.034030732 (223h40m52.96032542s) [noise=true , address=104.36.82.3:4767 , piece_id=U4L7ULC2R5OKDDL6BGDSEQUZNTVMQSGXJ56CGVKHNCCHTMIIDXDQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.034335911 (223h40m52.960630599s) [noise=true , address=45.91.116.8:4047 , piece_id=IJZDAO2K7KADVBNZGHLACI2ARTDDYOHIC742L2IUJT2WFAMMIGSQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.034644696 (223h40m52.960939386s) [noise=true , address=104.36.82.10:4079 , piece_id=JELVGAGG2NF2OSMKXQAV6DIXDTUFIER5WX4KTCJFQQ5P35EAMOEA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.148244089 (223h40m53.074538765s) [upload=179165]: Next returned nil segment
Jul 2 16:52:26.148268128 (223h40m53.074562811s) [upload=179165]: Waiting for error group managing segments...
Jul 2 16:52:26.155850465 (223h40m53.082145153s) [noise=true , address=45.91.116.5:4298 , piece_id=JUYTY35XKYPNJYE7X3R7UMINTVMXDMNM6YZSZORJWATTVDBZFOMQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.156432201 (223h40m53.082726892s) [noise=true , address=77.90.31.14:4661 , piece_id=B5E2MZ5JW7Z5AG4GFQ4QGMAFW7ZMKWKRFU276Z6556JFWKYVLXTQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.156943265 (223h40m53.083237956s) [noise=true , address=104.36.82.16:4294 , piece_id=36S2PXQADEAQPWSBMQC4XEAH6H5PDA4AO34JI37VD4KPJOU2WVTA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.158591570 (223h40m53.084886251s) [noise=true , address=104.36.82.3:4767 , piece_id=U4L7ULC2R5OKDDL6BGDSEQUZNTVMQSGXJ56CGVKHNCCHTMIIDXDQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.159458953 (223h40m53.085753634s) [noise=true , address=45.91.116.8:4047 , piece_id=IJZDAO2K7KADVBNZGHLACI2ARTDDYOHIC742L2IUJT2WFAMMIGSQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.161102375 (223h40m53.087397055s) [noise=true , address=45.91.116.12:4226 , piece_id=NFB2MT72D7MLPOY3A6MVO2QQBUFVYJPKN7D5XJQLQCCMYNQ6WJOA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.161556163 (223h40m53.087850838s) [noise=true , address=104.36.82.12:4149 , piece_id=V5265CLKUYZQN5B663WB4CFR3SXR2OA2BT53OAVCLFZJJ2YY6U4A, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.162013197 (223h40m53.088307874s) [noise=true , address=104.36.82.14:4222 , piece_id=3HZK5YRWL2KXKY4E7XASRI2UIPCJA72JSJSTGBLPA4ZBP3MGBEOQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.162459436 (223h40m53.088754112s) [noise=true , address=77.90.31.15:4698 , piece_id=VDVQDBYDQYRMVP7ZUPSEFC2R5UF2LVW45G23JURMYKS5IT6RBQFQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.162920549 (223h40m53.089215241s) [noise=true , address=77.90.31.21:4878 , piece_id=QJARHOYLDCFGYTTCD2EYCH43X3EARFK3Q3C7SCZ33X5T6CTLYDMQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.163747035 (223h40m53.090041713s) [noise=true , address=104.36.82.11:4115 , piece_id=LOK75B4Y2M3DMRJAHNYCVE46DELAAVN4SFP35AWPFESLUJT2V6KQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.167885784 (223h40m53.094180467s) [noise=true , address=104.36.82.6:4874 , piece_id=XVVQN3VX4I2KS4CIXEADBSFYBVYTVEWS5GMXAHE2LWDJGD2A2KAA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.168494969 (223h40m53.094789668s) [noise=true , address=104.36.82.10:4079 , piece_id=JELVGAGG2NF2OSMKXQAV6DIXDTUFIER5WX4KTCJFQQ5P35EAMOEA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.168525706 (223h40m53.094820396s) [seg_pos={0 0}, upload=179165]: Segment reached optimal threshold of 13 pieces.
Jul 2 16:52:26.168715378 (223h40m53.09501007s) [noise=true , address=77.90.31.19:4843 , piece_id=J74IERT5EP2VDSXY5PNWWBVTHQEXT4CWGLWP72Y3CQLOTSI6XZDA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: expected piece hash; context canceled; EOF
Jul 2 16:52:26.168766821 (223h40m53.095061509s) [noise=true , address=104.36.82.8:4970 , piece_id=X7KKQJYHVNI5LJDKGYNV5JHHCEGF4QSHFRKMSBLRE5W3W2ITAAHQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe): protocol: expected piece hash; context canceled; EOF
Jul 2 16:52:26.168816209 (223h40m53.095110907s) [noise=true , address=45.91.116.11:4118 , piece_id=CSQO6I5J4BW4NUAOFH3QDF6VEJGACWOSYNTNGXVELK2HLXIMX4YQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF
Jul 2 16:52:26.168851649 (223h40m53.095146335s) [seg_pos={0 0}, upload=179165]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 2 16:52:26.169058251 (223h40m53.095352944s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 16:52:26.169066272 (223h40m53.095360967s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 16:52:26.220481259 (223h40m53.146775936s) [upload=179165]: Done uploading segments...
Jul 2 16:52:26.220500093 (223h40m53.14679477s) [upload=179165]: Upload finished. err: <nil>
Duration: 450.056661ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................................X........................] duration:
1.801033633s
FAIL (2025-07-02 17:34:35 UTC)
orbiter warning: uplink: metaclient: manager closed: closed: read tcp 159.223.143.39:49282->34.150.199.48:7777: read: connection reset by peer
Duration: 1.801066643s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
467.45084ms
FAIL (2025-07-02 17:34:24 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
led; ecclient: upload cut due to slow connection (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN): protocol: expected piece hash; context canceled; EOF
Jul 2 16:51:39.852164074 (223h40m6.778458767s) [noise=true , address=104.36.82.12:4150 , piece_id=QHXVON3LSBWYG5AWPVZ45KOTJJPUXIIAXGED7OHG65V5V3IJPULA, seg_pos={0 0}, upload=179153]: Done uploading piece. err: <nil>
Jul 2 16:51:39.852220651 (223h40m6.778515346s) [seg_pos={0 0}, upload=179153]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 2 16:51:39.852324711 (223h40m6.778619405s) [upload=179153]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 16:51:39.852329895 (223h40m6.778624588s) [upload=179153]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 16:51:39.945155283 (223h40m6.871449972s) [upload=179153]: Done uploading segments...
Jul 2 16:51:39.945173208 (223h40m6.871467885s) [upload=179153]: Upload finished. err: <nil>
Jul 2 16:52:25.999553770 (223h40m52.925848461s) [upload=179165]: Starting upload
Jul 2 16:52:25.999676452 (223h40m52.92597114s) [upload=179165]: Done starting upload
Jul 2 16:52:25.999808777 (223h40m52.926103458s) [upload=179165]: Uploading segments...
Jul 2 16:52:26.000049412 (223h40m52.926344119s) [upload=179165]: Got next segment. Inline: false
Jul 2 16:52:26.000208665 (223h40m52.926503358s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 2 16:52:26.000216887 (223h40m52.926511587s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 2 16:52:26.029546629 (223h40m52.95584132s) [seg_pos={0 0}, upload=179165]: Begin upload segment...
Jul 2 16:52:26.029596311 (223h40m52.955890985s) [seg_pos={0 0}, upload=179165]: Done begin upload segment.
Jul 2 16:52:26.029630299 (223h40m52.955924986s) [noise=true , address=45.91.116.12:4226 , piece_id=NFB2MT72D7MLPOY3A6MVO2QQBUFVYJPKN7D5XJQLQCCMYNQ6WJOA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.030073195 (223h40m52.956367894s) [noise=true , address=77.90.31.14:4661 , piece_id=B5E2MZ5JW7Z5AG4GFQ4QGMAFW7ZMKWKRFU276Z6556JFWKYVLXTQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.030398264 (223h40m52.956692955s) [noise=true , address=77.90.31.15:4698 , piece_id=VDVQDBYDQYRMVP7ZUPSEFC2R5UF2LVW45G23JURMYKS5IT6RBQFQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.030716003 (223h40m52.957010692s) [noise=true , address=45.91.116.11:4118 , piece_id=CSQO6I5J4BW4NUAOFH3QDF6VEJGACWOSYNTNGXVELK2HLXIMX4YQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.031068785 (223h40m52.957363472s) [noise=true , address=45.91.116.5:4298 , piece_id=JUYTY35XKYPNJYE7X3R7UMINTVMXDMNM6YZSZORJWATTVDBZFOMQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.031405588 (223h40m52.957700282s) [noise=true , address=104.36.82.8:4970 , piece_id=X7KKQJYHVNI5LJDKGYNV5JHHCEGF4QSHFRKMSBLRE5W3W2ITAAHQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.031730040 (223h40m52.958024715s) [noise=true , address=104.36.82.12:4149 , piece_id=V5265CLKUYZQN5B663WB4CFR3SXR2OA2BT53OAVCLFZJJ2YY6U4A, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.032049179 (223h40m52.958343866s) [noise=true , address=104.36.82.6:4874 , piece_id=XVVQN3VX4I2KS4CIXEADBSFYBVYTVEWS5GMXAHE2LWDJGD2A2KAA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.032432775 (223h40m52.958727467s) [noise=true , address=104.36.82.16:4294 , piece_id=36S2PXQADEAQPWSBMQC4XEAH6H5PDA4AO34JI37VD4KPJOU2WVTA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.032754164 (223h40m52.959048852s) [noise=true , address=77.90.31.19:4843 , piece_id=J74IERT5EP2VDSXY5PNWWBVTHQEXT4CWGLWP72Y3CQLOTSI6XZDA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.033080515 (223h40m52.959375213s) [noise=true , address=104.36.82.11:4115 , piece_id=LOK75B4Y2M3DMRJAHNYCVE46DELAAVN4SFP35AWPFESLUJT2V6KQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.033386908 (223h40m52.959681597s) [noise=true , address=104.36.82.14:4222 , piece_id=3HZK5YRWL2KXKY4E7XASRI2UIPCJA72JSJSTGBLPA4ZBP3MGBEOQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.033709080 (223h40m52.960003768s) [noise=true , address=77.90.31.21:4878 , piece_id=QJARHOYLDCFGYTTCD2EYCH43X3EARFK3Q3C7SCZ33X5T6CTLYDMQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.034030732 (223h40m52.96032542s) [noise=true , address=104.36.82.3:4767 , piece_id=U4L7ULC2R5OKDDL6BGDSEQUZNTVMQSGXJ56CGVKHNCCHTMIIDXDQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.034335911 (223h40m52.960630599s) [noise=true , address=45.91.116.8:4047 , piece_id=IJZDAO2K7KADVBNZGHLACI2ARTDDYOHIC742L2IUJT2WFAMMIGSQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.034644696 (223h40m52.960939386s) [noise=true , address=104.36.82.10:4079 , piece_id=JELVGAGG2NF2OSMKXQAV6DIXDTUFIER5WX4KTCJFQQ5P35EAMOEA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.148244089 (223h40m53.074538765s) [upload=179165]: Next returned nil segment
Jul 2 16:52:26.148268128 (223h40m53.074562811s) [upload=179165]: Waiting for error group managing segments...
Jul 2 16:52:26.155850465 (223h40m53.082145153s) [noise=true , address=45.91.116.5:4298 , piece_id=JUYTY35XKYPNJYE7X3R7UMINTVMXDMNM6YZSZORJWATTVDBZFOMQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.156432201 (223h40m53.082726892s) [noise=true , address=77.90.31.14:4661 , piece_id=B5E2MZ5JW7Z5AG4GFQ4QGMAFW7ZMKWKRFU276Z6556JFWKYVLXTQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.156943265 (223h40m53.083237956s) [noise=true , address=104.36.82.16:4294 , piece_id=36S2PXQADEAQPWSBMQC4XEAH6H5PDA4AO34JI37VD4KPJOU2WVTA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.158591570 (223h40m53.084886251s) [noise=true , address=104.36.82.3:4767 , piece_id=U4L7ULC2R5OKDDL6BGDSEQUZNTVMQSGXJ56CGVKHNCCHTMIIDXDQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.159458953 (223h40m53.085753634s) [noise=true , address=45.91.116.8:4047 , piece_id=IJZDAO2K7KADVBNZGHLACI2ARTDDYOHIC742L2IUJT2WFAMMIGSQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.161102375 (223h40m53.087397055s) [noise=true , address=45.91.116.12:4226 , piece_id=NFB2MT72D7MLPOY3A6MVO2QQBUFVYJPKN7D5XJQLQCCMYNQ6WJOA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.161556163 (223h40m53.087850838s) [noise=true , address=104.36.82.12:4149 , piece_id=V5265CLKUYZQN5B663WB4CFR3SXR2OA2BT53OAVCLFZJJ2YY6U4A, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.162013197 (223h40m53.088307874s) [noise=true , address=104.36.82.14:4222 , piece_id=3HZK5YRWL2KXKY4E7XASRI2UIPCJA72JSJSTGBLPA4ZBP3MGBEOQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.162459436 (223h40m53.088754112s) [noise=true , address=77.90.31.15:4698 , piece_id=VDVQDBYDQYRMVP7ZUPSEFC2R5UF2LVW45G23JURMYKS5IT6RBQFQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.162920549 (223h40m53.089215241s) [noise=true , address=77.90.31.21:4878 , piece_id=QJARHOYLDCFGYTTCD2EYCH43X3EARFK3Q3C7SCZ33X5T6CTLYDMQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.163747035 (223h40m53.090041713s) [noise=true , address=104.36.82.11:4115 , piece_id=LOK75B4Y2M3DMRJAHNYCVE46DELAAVN4SFP35AWPFESLUJT2V6KQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.167885784 (223h40m53.094180467s) [noise=true , address=104.36.82.6:4874 , piece_id=XVVQN3VX4I2KS4CIXEADBSFYBVYTVEWS5GMXAHE2LWDJGD2A2KAA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.168494969 (223h40m53.094789668s) [noise=true , address=104.36.82.10:4079 , piece_id=JELVGAGG2NF2OSMKXQAV6DIXDTUFIER5WX4KTCJFQQ5P35EAMOEA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.168525706 (223h40m53.094820396s) [seg_pos={0 0}, upload=179165]: Segment reached optimal threshold of 13 pieces.
Jul 2 16:52:26.168715378 (223h40m53.09501007s) [noise=true , address=77.90.31.19:4843 , piece_id=J74IERT5EP2VDSXY5PNWWBVTHQEXT4CWGLWP72Y3CQLOTSI6XZDA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: expected piece hash; context canceled; EOF
Jul 2 16:52:26.168766821 (223h40m53.095061509s) [noise=true , address=104.36.82.8:4970 , piece_id=X7KKQJYHVNI5LJDKGYNV5JHHCEGF4QSHFRKMSBLRE5W3W2ITAAHQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe): protocol: expected piece hash; context canceled; EOF
Jul 2 16:52:26.168816209 (223h40m53.095110907s) [noise=true , address=45.91.116.11:4118 , piece_id=CSQO6I5J4BW4NUAOFH3QDF6VEJGACWOSYNTNGXVELK2HLXIMX4YQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF
Jul 2 16:52:26.168851649 (223h40m53.095146335s) [seg_pos={0 0}, upload=179165]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 2 16:52:26.169058251 (223h40m53.095352944s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 16:52:26.169066272 (223h40m53.095360967s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 16:52:26.220481259 (223h40m53.146775936s) [upload=179165]: Done uploading segments...
Jul 2 16:52:26.220500093 (223h40m53.14679477s) [upload=179165]: Upload finished. err: <nil>
Duration: 467.486295ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................................X.......................] duration:
1.70276373s
FAIL (2025-07-02 17:34:01 UTC)
orbiter warning: uplink: metaclient: manager closed: closed: read tcp 159.223.143.39:49282->34.150.199.48:7777: read: connection reset by peer
Duration: 1.702796959s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................] duration:
510.190219ms
FAIL (2025-07-02 17:33:45 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
led; ecclient: upload cut due to slow connection (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN): protocol: expected piece hash; context canceled; EOF
Jul 2 16:51:39.852164074 (223h40m6.778458767s) [noise=true , address=104.36.82.12:4150 , piece_id=QHXVON3LSBWYG5AWPVZ45KOTJJPUXIIAXGED7OHG65V5V3IJPULA, seg_pos={0 0}, upload=179153]: Done uploading piece. err: <nil>
Jul 2 16:51:39.852220651 (223h40m6.778515346s) [seg_pos={0 0}, upload=179153]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 2 16:51:39.852324711 (223h40m6.778619405s) [upload=179153]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 16:51:39.852329895 (223h40m6.778624588s) [upload=179153]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 16:51:39.945155283 (223h40m6.871449972s) [upload=179153]: Done uploading segments...
Jul 2 16:51:39.945173208 (223h40m6.871467885s) [upload=179153]: Upload finished. err: <nil>
Jul 2 16:52:25.999553770 (223h40m52.925848461s) [upload=179165]: Starting upload
Jul 2 16:52:25.999676452 (223h40m52.92597114s) [upload=179165]: Done starting upload
Jul 2 16:52:25.999808777 (223h40m52.926103458s) [upload=179165]: Uploading segments...
Jul 2 16:52:26.000049412 (223h40m52.926344119s) [upload=179165]: Got next segment. Inline: false
Jul 2 16:52:26.000208665 (223h40m52.926503358s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 2 16:52:26.000216887 (223h40m52.926511587s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 2 16:52:26.029546629 (223h40m52.95584132s) [seg_pos={0 0}, upload=179165]: Begin upload segment...
Jul 2 16:52:26.029596311 (223h40m52.955890985s) [seg_pos={0 0}, upload=179165]: Done begin upload segment.
Jul 2 16:52:26.029630299 (223h40m52.955924986s) [noise=true , address=45.91.116.12:4226 , piece_id=NFB2MT72D7MLPOY3A6MVO2QQBUFVYJPKN7D5XJQLQCCMYNQ6WJOA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.030073195 (223h40m52.956367894s) [noise=true , address=77.90.31.14:4661 , piece_id=B5E2MZ5JW7Z5AG4GFQ4QGMAFW7ZMKWKRFU276Z6556JFWKYVLXTQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.030398264 (223h40m52.956692955s) [noise=true , address=77.90.31.15:4698 , piece_id=VDVQDBYDQYRMVP7ZUPSEFC2R5UF2LVW45G23JURMYKS5IT6RBQFQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.030716003 (223h40m52.957010692s) [noise=true , address=45.91.116.11:4118 , piece_id=CSQO6I5J4BW4NUAOFH3QDF6VEJGACWOSYNTNGXVELK2HLXIMX4YQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.031068785 (223h40m52.957363472s) [noise=true , address=45.91.116.5:4298 , piece_id=JUYTY35XKYPNJYE7X3R7UMINTVMXDMNM6YZSZORJWATTVDBZFOMQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.031405588 (223h40m52.957700282s) [noise=true , address=104.36.82.8:4970 , piece_id=X7KKQJYHVNI5LJDKGYNV5JHHCEGF4QSHFRKMSBLRE5W3W2ITAAHQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.031730040 (223h40m52.958024715s) [noise=true , address=104.36.82.12:4149 , piece_id=V5265CLKUYZQN5B663WB4CFR3SXR2OA2BT53OAVCLFZJJ2YY6U4A, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.032049179 (223h40m52.958343866s) [noise=true , address=104.36.82.6:4874 , piece_id=XVVQN3VX4I2KS4CIXEADBSFYBVYTVEWS5GMXAHE2LWDJGD2A2KAA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.032432775 (223h40m52.958727467s) [noise=true , address=104.36.82.16:4294 , piece_id=36S2PXQADEAQPWSBMQC4XEAH6H5PDA4AO34JI37VD4KPJOU2WVTA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.032754164 (223h40m52.959048852s) [noise=true , address=77.90.31.19:4843 , piece_id=J74IERT5EP2VDSXY5PNWWBVTHQEXT4CWGLWP72Y3CQLOTSI6XZDA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.033080515 (223h40m52.959375213s) [noise=true , address=104.36.82.11:4115 , piece_id=LOK75B4Y2M3DMRJAHNYCVE46DELAAVN4SFP35AWPFESLUJT2V6KQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.033386908 (223h40m52.959681597s) [noise=true , address=104.36.82.14:4222 , piece_id=3HZK5YRWL2KXKY4E7XASRI2UIPCJA72JSJSTGBLPA4ZBP3MGBEOQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.033709080 (223h40m52.960003768s) [noise=true , address=77.90.31.21:4878 , piece_id=QJARHOYLDCFGYTTCD2EYCH43X3EARFK3Q3C7SCZ33X5T6CTLYDMQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.034030732 (223h40m52.96032542s) [noise=true , address=104.36.82.3:4767 , piece_id=U4L7ULC2R5OKDDL6BGDSEQUZNTVMQSGXJ56CGVKHNCCHTMIIDXDQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.034335911 (223h40m52.960630599s) [noise=true , address=45.91.116.8:4047 , piece_id=IJZDAO2K7KADVBNZGHLACI2ARTDDYOHIC742L2IUJT2WFAMMIGSQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.034644696 (223h40m52.960939386s) [noise=true , address=104.36.82.10:4079 , piece_id=JELVGAGG2NF2OSMKXQAV6DIXDTUFIER5WX4KTCJFQQ5P35EAMOEA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.148244089 (223h40m53.074538765s) [upload=179165]: Next returned nil segment
Jul 2 16:52:26.148268128 (223h40m53.074562811s) [upload=179165]: Waiting for error group managing segments...
Jul 2 16:52:26.155850465 (223h40m53.082145153s) [noise=true , address=45.91.116.5:4298 , piece_id=JUYTY35XKYPNJYE7X3R7UMINTVMXDMNM6YZSZORJWATTVDBZFOMQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.156432201 (223h40m53.082726892s) [noise=true , address=77.90.31.14:4661 , piece_id=B5E2MZ5JW7Z5AG4GFQ4QGMAFW7ZMKWKRFU276Z6556JFWKYVLXTQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.156943265 (223h40m53.083237956s) [noise=true , address=104.36.82.16:4294 , piece_id=36S2PXQADEAQPWSBMQC4XEAH6H5PDA4AO34JI37VD4KPJOU2WVTA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.158591570 (223h40m53.084886251s) [noise=true , address=104.36.82.3:4767 , piece_id=U4L7ULC2R5OKDDL6BGDSEQUZNTVMQSGXJ56CGVKHNCCHTMIIDXDQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.159458953 (223h40m53.085753634s) [noise=true , address=45.91.116.8:4047 , piece_id=IJZDAO2K7KADVBNZGHLACI2ARTDDYOHIC742L2IUJT2WFAMMIGSQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.161102375 (223h40m53.087397055s) [noise=true , address=45.91.116.12:4226 , piece_id=NFB2MT72D7MLPOY3A6MVO2QQBUFVYJPKN7D5XJQLQCCMYNQ6WJOA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.161556163 (223h40m53.087850838s) [noise=true , address=104.36.82.12:4149 , piece_id=V5265CLKUYZQN5B663WB4CFR3SXR2OA2BT53OAVCLFZJJ2YY6U4A, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.162013197 (223h40m53.088307874s) [noise=true , address=104.36.82.14:4222 , piece_id=3HZK5YRWL2KXKY4E7XASRI2UIPCJA72JSJSTGBLPA4ZBP3MGBEOQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.162459436 (223h40m53.088754112s) [noise=true , address=77.90.31.15:4698 , piece_id=VDVQDBYDQYRMVP7ZUPSEFC2R5UF2LVW45G23JURMYKS5IT6RBQFQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.162920549 (223h40m53.089215241s) [noise=true , address=77.90.31.21:4878 , piece_id=QJARHOYLDCFGYTTCD2EYCH43X3EARFK3Q3C7SCZ33X5T6CTLYDMQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.163747035 (223h40m53.090041713s) [noise=true , address=104.36.82.11:4115 , piece_id=LOK75B4Y2M3DMRJAHNYCVE46DELAAVN4SFP35AWPFESLUJT2V6KQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.167885784 (223h40m53.094180467s) [noise=true , address=104.36.82.6:4874 , piece_id=XVVQN3VX4I2KS4CIXEADBSFYBVYTVEWS5GMXAHE2LWDJGD2A2KAA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.168494969 (223h40m53.094789668s) [noise=true , address=104.36.82.10:4079 , piece_id=JELVGAGG2NF2OSMKXQAV6DIXDTUFIER5WX4KTCJFQQ5P35EAMOEA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.168525706 (223h40m53.094820396s) [seg_pos={0 0}, upload=179165]: Segment reached optimal threshold of 13 pieces.
Jul 2 16:52:26.168715378 (223h40m53.09501007s) [noise=true , address=77.90.31.19:4843 , piece_id=J74IERT5EP2VDSXY5PNWWBVTHQEXT4CWGLWP72Y3CQLOTSI6XZDA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: expected piece hash; context canceled; EOF
Jul 2 16:52:26.168766821 (223h40m53.095061509s) [noise=true , address=104.36.82.8:4970 , piece_id=X7KKQJYHVNI5LJDKGYNV5JHHCEGF4QSHFRKMSBLRE5W3W2ITAAHQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe): protocol: expected piece hash; context canceled; EOF
Jul 2 16:52:26.168816209 (223h40m53.095110907s) [noise=true , address=45.91.116.11:4118 , piece_id=CSQO6I5J4BW4NUAOFH3QDF6VEJGACWOSYNTNGXVELK2HLXIMX4YQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF
Jul 2 16:52:26.168851649 (223h40m53.095146335s) [seg_pos={0 0}, upload=179165]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 2 16:52:26.169058251 (223h40m53.095352944s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 16:52:26.169066272 (223h40m53.095360967s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 16:52:26.220481259 (223h40m53.146775936s) [upload=179165]: Done uploading segments...
Jul 2 16:52:26.220500093 (223h40m53.14679477s) [upload=179165]: Upload finished. err: <nil>
Duration: 510.239911ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................................X......................] duration:
1.803726816s
FAIL (2025-07-02 17:33:16 UTC)
orbiter warning: uplink: metaclient: manager closed: closed: read tcp 159.223.143.39:49282->34.150.199.48:7777: read: connection reset by peer
Duration: 1.803761377s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................................X.....................] duration:
1.795187267s
FAIL (2025-07-02 17:32:54 UTC)
orbiter warning: uplink: metaclient: manager closed: closed: read tcp 159.223.143.39:49282->34.150.199.48:7777: read: connection reset by peer
Duration: 1.795218326s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
454.869268ms
FAIL (2025-07-02 17:32:35 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
led; ecclient: upload cut due to slow connection (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN): protocol: expected piece hash; context canceled; EOF
Jul 2 16:51:39.852164074 (223h40m6.778458767s) [noise=true , address=104.36.82.12:4150 , piece_id=QHXVON3LSBWYG5AWPVZ45KOTJJPUXIIAXGED7OHG65V5V3IJPULA, seg_pos={0 0}, upload=179153]: Done uploading piece. err: <nil>
Jul 2 16:51:39.852220651 (223h40m6.778515346s) [seg_pos={0 0}, upload=179153]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 2 16:51:39.852324711 (223h40m6.778619405s) [upload=179153]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 16:51:39.852329895 (223h40m6.778624588s) [upload=179153]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 16:51:39.945155283 (223h40m6.871449972s) [upload=179153]: Done uploading segments...
Jul 2 16:51:39.945173208 (223h40m6.871467885s) [upload=179153]: Upload finished. err: <nil>
Jul 2 16:52:25.999553770 (223h40m52.925848461s) [upload=179165]: Starting upload
Jul 2 16:52:25.999676452 (223h40m52.92597114s) [upload=179165]: Done starting upload
Jul 2 16:52:25.999808777 (223h40m52.926103458s) [upload=179165]: Uploading segments...
Jul 2 16:52:26.000049412 (223h40m52.926344119s) [upload=179165]: Got next segment. Inline: false
Jul 2 16:52:26.000208665 (223h40m52.926503358s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 2 16:52:26.000216887 (223h40m52.926511587s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 2 16:52:26.029546629 (223h40m52.95584132s) [seg_pos={0 0}, upload=179165]: Begin upload segment...
Jul 2 16:52:26.029596311 (223h40m52.955890985s) [seg_pos={0 0}, upload=179165]: Done begin upload segment.
Jul 2 16:52:26.029630299 (223h40m52.955924986s) [noise=true , address=45.91.116.12:4226 , piece_id=NFB2MT72D7MLPOY3A6MVO2QQBUFVYJPKN7D5XJQLQCCMYNQ6WJOA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.030073195 (223h40m52.956367894s) [noise=true , address=77.90.31.14:4661 , piece_id=B5E2MZ5JW7Z5AG4GFQ4QGMAFW7ZMKWKRFU276Z6556JFWKYVLXTQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.030398264 (223h40m52.956692955s) [noise=true , address=77.90.31.15:4698 , piece_id=VDVQDBYDQYRMVP7ZUPSEFC2R5UF2LVW45G23JURMYKS5IT6RBQFQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.030716003 (223h40m52.957010692s) [noise=true , address=45.91.116.11:4118 , piece_id=CSQO6I5J4BW4NUAOFH3QDF6VEJGACWOSYNTNGXVELK2HLXIMX4YQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.031068785 (223h40m52.957363472s) [noise=true , address=45.91.116.5:4298 , piece_id=JUYTY35XKYPNJYE7X3R7UMINTVMXDMNM6YZSZORJWATTVDBZFOMQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.031405588 (223h40m52.957700282s) [noise=true , address=104.36.82.8:4970 , piece_id=X7KKQJYHVNI5LJDKGYNV5JHHCEGF4QSHFRKMSBLRE5W3W2ITAAHQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.031730040 (223h40m52.958024715s) [noise=true , address=104.36.82.12:4149 , piece_id=V5265CLKUYZQN5B663WB4CFR3SXR2OA2BT53OAVCLFZJJ2YY6U4A, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.032049179 (223h40m52.958343866s) [noise=true , address=104.36.82.6:4874 , piece_id=XVVQN3VX4I2KS4CIXEADBSFYBVYTVEWS5GMXAHE2LWDJGD2A2KAA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.032432775 (223h40m52.958727467s) [noise=true , address=104.36.82.16:4294 , piece_id=36S2PXQADEAQPWSBMQC4XEAH6H5PDA4AO34JI37VD4KPJOU2WVTA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.032754164 (223h40m52.959048852s) [noise=true , address=77.90.31.19:4843 , piece_id=J74IERT5EP2VDSXY5PNWWBVTHQEXT4CWGLWP72Y3CQLOTSI6XZDA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.033080515 (223h40m52.959375213s) [noise=true , address=104.36.82.11:4115 , piece_id=LOK75B4Y2M3DMRJAHNYCVE46DELAAVN4SFP35AWPFESLUJT2V6KQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.033386908 (223h40m52.959681597s) [noise=true , address=104.36.82.14:4222 , piece_id=3HZK5YRWL2KXKY4E7XASRI2UIPCJA72JSJSTGBLPA4ZBP3MGBEOQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.033709080 (223h40m52.960003768s) [noise=true , address=77.90.31.21:4878 , piece_id=QJARHOYLDCFGYTTCD2EYCH43X3EARFK3Q3C7SCZ33X5T6CTLYDMQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.034030732 (223h40m52.96032542s) [noise=true , address=104.36.82.3:4767 , piece_id=U4L7ULC2R5OKDDL6BGDSEQUZNTVMQSGXJ56CGVKHNCCHTMIIDXDQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.034335911 (223h40m52.960630599s) [noise=true , address=45.91.116.8:4047 , piece_id=IJZDAO2K7KADVBNZGHLACI2ARTDDYOHIC742L2IUJT2WFAMMIGSQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.034644696 (223h40m52.960939386s) [noise=true , address=104.36.82.10:4079 , piece_id=JELVGAGG2NF2OSMKXQAV6DIXDTUFIER5WX4KTCJFQQ5P35EAMOEA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.148244089 (223h40m53.074538765s) [upload=179165]: Next returned nil segment
Jul 2 16:52:26.148268128 (223h40m53.074562811s) [upload=179165]: Waiting for error group managing segments...
Jul 2 16:52:26.155850465 (223h40m53.082145153s) [noise=true , address=45.91.116.5:4298 , piece_id=JUYTY35XKYPNJYE7X3R7UMINTVMXDMNM6YZSZORJWATTVDBZFOMQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.156432201 (223h40m53.082726892s) [noise=true , address=77.90.31.14:4661 , piece_id=B5E2MZ5JW7Z5AG4GFQ4QGMAFW7ZMKWKRFU276Z6556JFWKYVLXTQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.156943265 (223h40m53.083237956s) [noise=true , address=104.36.82.16:4294 , piece_id=36S2PXQADEAQPWSBMQC4XEAH6H5PDA4AO34JI37VD4KPJOU2WVTA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.158591570 (223h40m53.084886251s) [noise=true , address=104.36.82.3:4767 , piece_id=U4L7ULC2R5OKDDL6BGDSEQUZNTVMQSGXJ56CGVKHNCCHTMIIDXDQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.159458953 (223h40m53.085753634s) [noise=true , address=45.91.116.8:4047 , piece_id=IJZDAO2K7KADVBNZGHLACI2ARTDDYOHIC742L2IUJT2WFAMMIGSQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.161102375 (223h40m53.087397055s) [noise=true , address=45.91.116.12:4226 , piece_id=NFB2MT72D7MLPOY3A6MVO2QQBUFVYJPKN7D5XJQLQCCMYNQ6WJOA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.161556163 (223h40m53.087850838s) [noise=true , address=104.36.82.12:4149 , piece_id=V5265CLKUYZQN5B663WB4CFR3SXR2OA2BT53OAVCLFZJJ2YY6U4A, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.162013197 (223h40m53.088307874s) [noise=true , address=104.36.82.14:4222 , piece_id=3HZK5YRWL2KXKY4E7XASRI2UIPCJA72JSJSTGBLPA4ZBP3MGBEOQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.162459436 (223h40m53.088754112s) [noise=true , address=77.90.31.15:4698 , piece_id=VDVQDBYDQYRMVP7ZUPSEFC2R5UF2LVW45G23JURMYKS5IT6RBQFQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.162920549 (223h40m53.089215241s) [noise=true , address=77.90.31.21:4878 , piece_id=QJARHOYLDCFGYTTCD2EYCH43X3EARFK3Q3C7SCZ33X5T6CTLYDMQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.163747035 (223h40m53.090041713s) [noise=true , address=104.36.82.11:4115 , piece_id=LOK75B4Y2M3DMRJAHNYCVE46DELAAVN4SFP35AWPFESLUJT2V6KQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.167885784 (223h40m53.094180467s) [noise=true , address=104.36.82.6:4874 , piece_id=XVVQN3VX4I2KS4CIXEADBSFYBVYTVEWS5GMXAHE2LWDJGD2A2KAA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.168494969 (223h40m53.094789668s) [noise=true , address=104.36.82.10:4079 , piece_id=JELVGAGG2NF2OSMKXQAV6DIXDTUFIER5WX4KTCJFQQ5P35EAMOEA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.168525706 (223h40m53.094820396s) [seg_pos={0 0}, upload=179165]: Segment reached optimal threshold of 13 pieces.
Jul 2 16:52:26.168715378 (223h40m53.09501007s) [noise=true , address=77.90.31.19:4843 , piece_id=J74IERT5EP2VDSXY5PNWWBVTHQEXT4CWGLWP72Y3CQLOTSI6XZDA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: expected piece hash; context canceled; EOF
Jul 2 16:52:26.168766821 (223h40m53.095061509s) [noise=true , address=104.36.82.8:4970 , piece_id=X7KKQJYHVNI5LJDKGYNV5JHHCEGF4QSHFRKMSBLRE5W3W2ITAAHQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe): protocol: expected piece hash; context canceled; EOF
Jul 2 16:52:26.168816209 (223h40m53.095110907s) [noise=true , address=45.91.116.11:4118 , piece_id=CSQO6I5J4BW4NUAOFH3QDF6VEJGACWOSYNTNGXVELK2HLXIMX4YQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF
Jul 2 16:52:26.168851649 (223h40m53.095146335s) [seg_pos={0 0}, upload=179165]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 2 16:52:26.169058251 (223h40m53.095352944s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 16:52:26.169066272 (223h40m53.095360967s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 16:52:26.220481259 (223h40m53.146775936s) [upload=179165]: Done uploading segments...
Jul 2 16:52:26.220500093 (223h40m53.14679477s) [upload=179165]: Upload finished. err: <nil>
Duration: 454.896573ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................] duration:
453.654489ms
FAIL (2025-07-02 17:32:11 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
led; ecclient: upload cut due to slow connection (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN): protocol: expected piece hash; context canceled; EOF
Jul 2 16:51:39.852164074 (223h40m6.778458767s) [noise=true , address=104.36.82.12:4150 , piece_id=QHXVON3LSBWYG5AWPVZ45KOTJJPUXIIAXGED7OHG65V5V3IJPULA, seg_pos={0 0}, upload=179153]: Done uploading piece. err: <nil>
Jul 2 16:51:39.852220651 (223h40m6.778515346s) [seg_pos={0 0}, upload=179153]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 2 16:51:39.852324711 (223h40m6.778619405s) [upload=179153]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 16:51:39.852329895 (223h40m6.778624588s) [upload=179153]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 16:51:39.945155283 (223h40m6.871449972s) [upload=179153]: Done uploading segments...
Jul 2 16:51:39.945173208 (223h40m6.871467885s) [upload=179153]: Upload finished. err: <nil>
Jul 2 16:52:25.999553770 (223h40m52.925848461s) [upload=179165]: Starting upload
Jul 2 16:52:25.999676452 (223h40m52.92597114s) [upload=179165]: Done starting upload
Jul 2 16:52:25.999808777 (223h40m52.926103458s) [upload=179165]: Uploading segments...
Jul 2 16:52:26.000049412 (223h40m52.926344119s) [upload=179165]: Got next segment. Inline: false
Jul 2 16:52:26.000208665 (223h40m52.926503358s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 2 16:52:26.000216887 (223h40m52.926511587s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 2 16:52:26.029546629 (223h40m52.95584132s) [seg_pos={0 0}, upload=179165]: Begin upload segment...
Jul 2 16:52:26.029596311 (223h40m52.955890985s) [seg_pos={0 0}, upload=179165]: Done begin upload segment.
Jul 2 16:52:26.029630299 (223h40m52.955924986s) [noise=true , address=45.91.116.12:4226 , piece_id=NFB2MT72D7MLPOY3A6MVO2QQBUFVYJPKN7D5XJQLQCCMYNQ6WJOA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.030073195 (223h40m52.956367894s) [noise=true , address=77.90.31.14:4661 , piece_id=B5E2MZ5JW7Z5AG4GFQ4QGMAFW7ZMKWKRFU276Z6556JFWKYVLXTQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.030398264 (223h40m52.956692955s) [noise=true , address=77.90.31.15:4698 , piece_id=VDVQDBYDQYRMVP7ZUPSEFC2R5UF2LVW45G23JURMYKS5IT6RBQFQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.030716003 (223h40m52.957010692s) [noise=true , address=45.91.116.11:4118 , piece_id=CSQO6I5J4BW4NUAOFH3QDF6VEJGACWOSYNTNGXVELK2HLXIMX4YQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.031068785 (223h40m52.957363472s) [noise=true , address=45.91.116.5:4298 , piece_id=JUYTY35XKYPNJYE7X3R7UMINTVMXDMNM6YZSZORJWATTVDBZFOMQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.031405588 (223h40m52.957700282s) [noise=true , address=104.36.82.8:4970 , piece_id=X7KKQJYHVNI5LJDKGYNV5JHHCEGF4QSHFRKMSBLRE5W3W2ITAAHQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.031730040 (223h40m52.958024715s) [noise=true , address=104.36.82.12:4149 , piece_id=V5265CLKUYZQN5B663WB4CFR3SXR2OA2BT53OAVCLFZJJ2YY6U4A, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.032049179 (223h40m52.958343866s) [noise=true , address=104.36.82.6:4874 , piece_id=XVVQN3VX4I2KS4CIXEADBSFYBVYTVEWS5GMXAHE2LWDJGD2A2KAA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.032432775 (223h40m52.958727467s) [noise=true , address=104.36.82.16:4294 , piece_id=36S2PXQADEAQPWSBMQC4XEAH6H5PDA4AO34JI37VD4KPJOU2WVTA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.032754164 (223h40m52.959048852s) [noise=true , address=77.90.31.19:4843 , piece_id=J74IERT5EP2VDSXY5PNWWBVTHQEXT4CWGLWP72Y3CQLOTSI6XZDA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.033080515 (223h40m52.959375213s) [noise=true , address=104.36.82.11:4115 , piece_id=LOK75B4Y2M3DMRJAHNYCVE46DELAAVN4SFP35AWPFESLUJT2V6KQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.033386908 (223h40m52.959681597s) [noise=true , address=104.36.82.14:4222 , piece_id=3HZK5YRWL2KXKY4E7XASRI2UIPCJA72JSJSTGBLPA4ZBP3MGBEOQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.033709080 (223h40m52.960003768s) [noise=true , address=77.90.31.21:4878 , piece_id=QJARHOYLDCFGYTTCD2EYCH43X3EARFK3Q3C7SCZ33X5T6CTLYDMQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.034030732 (223h40m52.96032542s) [noise=true , address=104.36.82.3:4767 , piece_id=U4L7ULC2R5OKDDL6BGDSEQUZNTVMQSGXJ56CGVKHNCCHTMIIDXDQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.034335911 (223h40m52.960630599s) [noise=true , address=45.91.116.8:4047 , piece_id=IJZDAO2K7KADVBNZGHLACI2ARTDDYOHIC742L2IUJT2WFAMMIGSQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.034644696 (223h40m52.960939386s) [noise=true , address=104.36.82.10:4079 , piece_id=JELVGAGG2NF2OSMKXQAV6DIXDTUFIER5WX4KTCJFQQ5P35EAMOEA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.148244089 (223h40m53.074538765s) [upload=179165]: Next returned nil segment
Jul 2 16:52:26.148268128 (223h40m53.074562811s) [upload=179165]: Waiting for error group managing segments...
Jul 2 16:52:26.155850465 (223h40m53.082145153s) [noise=true , address=45.91.116.5:4298 , piece_id=JUYTY35XKYPNJYE7X3R7UMINTVMXDMNM6YZSZORJWATTVDBZFOMQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.156432201 (223h40m53.082726892s) [noise=true , address=77.90.31.14:4661 , piece_id=B5E2MZ5JW7Z5AG4GFQ4QGMAFW7ZMKWKRFU276Z6556JFWKYVLXTQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.156943265 (223h40m53.083237956s) [noise=true , address=104.36.82.16:4294 , piece_id=36S2PXQADEAQPWSBMQC4XEAH6H5PDA4AO34JI37VD4KPJOU2WVTA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.158591570 (223h40m53.084886251s) [noise=true , address=104.36.82.3:4767 , piece_id=U4L7ULC2R5OKDDL6BGDSEQUZNTVMQSGXJ56CGVKHNCCHTMIIDXDQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.159458953 (223h40m53.085753634s) [noise=true , address=45.91.116.8:4047 , piece_id=IJZDAO2K7KADVBNZGHLACI2ARTDDYOHIC742L2IUJT2WFAMMIGSQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.161102375 (223h40m53.087397055s) [noise=true , address=45.91.116.12:4226 , piece_id=NFB2MT72D7MLPOY3A6MVO2QQBUFVYJPKN7D5XJQLQCCMYNQ6WJOA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.161556163 (223h40m53.087850838s) [noise=true , address=104.36.82.12:4149 , piece_id=V5265CLKUYZQN5B663WB4CFR3SXR2OA2BT53OAVCLFZJJ2YY6U4A, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.162013197 (223h40m53.088307874s) [noise=true , address=104.36.82.14:4222 , piece_id=3HZK5YRWL2KXKY4E7XASRI2UIPCJA72JSJSTGBLPA4ZBP3MGBEOQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.162459436 (223h40m53.088754112s) [noise=true , address=77.90.31.15:4698 , piece_id=VDVQDBYDQYRMVP7ZUPSEFC2R5UF2LVW45G23JURMYKS5IT6RBQFQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.162920549 (223h40m53.089215241s) [noise=true , address=77.90.31.21:4878 , piece_id=QJARHOYLDCFGYTTCD2EYCH43X3EARFK3Q3C7SCZ33X5T6CTLYDMQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.163747035 (223h40m53.090041713s) [noise=true , address=104.36.82.11:4115 , piece_id=LOK75B4Y2M3DMRJAHNYCVE46DELAAVN4SFP35AWPFESLUJT2V6KQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.167885784 (223h40m53.094180467s) [noise=true , address=104.36.82.6:4874 , piece_id=XVVQN3VX4I2KS4CIXEADBSFYBVYTVEWS5GMXAHE2LWDJGD2A2KAA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.168494969 (223h40m53.094789668s) [noise=true , address=104.36.82.10:4079 , piece_id=JELVGAGG2NF2OSMKXQAV6DIXDTUFIER5WX4KTCJFQQ5P35EAMOEA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.168525706 (223h40m53.094820396s) [seg_pos={0 0}, upload=179165]: Segment reached optimal threshold of 13 pieces.
Jul 2 16:52:26.168715378 (223h40m53.09501007s) [noise=true , address=77.90.31.19:4843 , piece_id=J74IERT5EP2VDSXY5PNWWBVTHQEXT4CWGLWP72Y3CQLOTSI6XZDA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: expected piece hash; context canceled; EOF
Jul 2 16:52:26.168766821 (223h40m53.095061509s) [noise=true , address=104.36.82.8:4970 , piece_id=X7KKQJYHVNI5LJDKGYNV5JHHCEGF4QSHFRKMSBLRE5W3W2ITAAHQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe): protocol: expected piece hash; context canceled; EOF
Jul 2 16:52:26.168816209 (223h40m53.095110907s) [noise=true , address=45.91.116.11:4118 , piece_id=CSQO6I5J4BW4NUAOFH3QDF6VEJGACWOSYNTNGXVELK2HLXIMX4YQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF
Jul 2 16:52:26.168851649 (223h40m53.095146335s) [seg_pos={0 0}, upload=179165]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 2 16:52:26.169058251 (223h40m53.095352944s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 16:52:26.169066272 (223h40m53.095360967s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 16:52:26.220481259 (223h40m53.146775936s) [upload=179165]: Done uploading segments...
Jul 2 16:52:26.220500093 (223h40m53.14679477s) [upload=179165]: Upload finished. err: <nil>
Duration: 453.690073ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......................................X....................] duration:
1.730321234s
FAIL (2025-07-02 17:31:52 UTC)
orbiter warning: uplink: metaclient: manager closed: closed: read tcp 159.223.143.39:49282->34.150.199.48:7777: read: connection reset by peer
Duration: 1.730351837s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......X.....................................................] duration:
1.285122502s
FAIL (2025-07-02 17:31:17 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
led; ecclient: upload cut due to slow connection (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN): protocol: expected piece hash; context canceled; EOF
Jul 2 16:51:39.852164074 (223h40m6.778458767s) [noise=true , address=104.36.82.12:4150 , piece_id=QHXVON3LSBWYG5AWPVZ45KOTJJPUXIIAXGED7OHG65V5V3IJPULA, seg_pos={0 0}, upload=179153]: Done uploading piece. err: <nil>
Jul 2 16:51:39.852220651 (223h40m6.778515346s) [seg_pos={0 0}, upload=179153]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 2 16:51:39.852324711 (223h40m6.778619405s) [upload=179153]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 16:51:39.852329895 (223h40m6.778624588s) [upload=179153]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 16:51:39.945155283 (223h40m6.871449972s) [upload=179153]: Done uploading segments...
Jul 2 16:51:39.945173208 (223h40m6.871467885s) [upload=179153]: Upload finished. err: <nil>
Jul 2 16:52:25.999553770 (223h40m52.925848461s) [upload=179165]: Starting upload
Jul 2 16:52:25.999676452 (223h40m52.92597114s) [upload=179165]: Done starting upload
Jul 2 16:52:25.999808777 (223h40m52.926103458s) [upload=179165]: Uploading segments...
Jul 2 16:52:26.000049412 (223h40m52.926344119s) [upload=179165]: Got next segment. Inline: false
Jul 2 16:52:26.000208665 (223h40m52.926503358s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 2 16:52:26.000216887 (223h40m52.926511587s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 2 16:52:26.029546629 (223h40m52.95584132s) [seg_pos={0 0}, upload=179165]: Begin upload segment...
Jul 2 16:52:26.029596311 (223h40m52.955890985s) [seg_pos={0 0}, upload=179165]: Done begin upload segment.
Jul 2 16:52:26.029630299 (223h40m52.955924986s) [noise=true , address=45.91.116.12:4226 , piece_id=NFB2MT72D7MLPOY3A6MVO2QQBUFVYJPKN7D5XJQLQCCMYNQ6WJOA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.030073195 (223h40m52.956367894s) [noise=true , address=77.90.31.14:4661 , piece_id=B5E2MZ5JW7Z5AG4GFQ4QGMAFW7ZMKWKRFU276Z6556JFWKYVLXTQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.030398264 (223h40m52.956692955s) [noise=true , address=77.90.31.15:4698 , piece_id=VDVQDBYDQYRMVP7ZUPSEFC2R5UF2LVW45G23JURMYKS5IT6RBQFQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.030716003 (223h40m52.957010692s) [noise=true , address=45.91.116.11:4118 , piece_id=CSQO6I5J4BW4NUAOFH3QDF6VEJGACWOSYNTNGXVELK2HLXIMX4YQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.031068785 (223h40m52.957363472s) [noise=true , address=45.91.116.5:4298 , piece_id=JUYTY35XKYPNJYE7X3R7UMINTVMXDMNM6YZSZORJWATTVDBZFOMQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.031405588 (223h40m52.957700282s) [noise=true , address=104.36.82.8:4970 , piece_id=X7KKQJYHVNI5LJDKGYNV5JHHCEGF4QSHFRKMSBLRE5W3W2ITAAHQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.031730040 (223h40m52.958024715s) [noise=true , address=104.36.82.12:4149 , piece_id=V5265CLKUYZQN5B663WB4CFR3SXR2OA2BT53OAVCLFZJJ2YY6U4A, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.032049179 (223h40m52.958343866s) [noise=true , address=104.36.82.6:4874 , piece_id=XVVQN3VX4I2KS4CIXEADBSFYBVYTVEWS5GMXAHE2LWDJGD2A2KAA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.032432775 (223h40m52.958727467s) [noise=true , address=104.36.82.16:4294 , piece_id=36S2PXQADEAQPWSBMQC4XEAH6H5PDA4AO34JI37VD4KPJOU2WVTA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.032754164 (223h40m52.959048852s) [noise=true , address=77.90.31.19:4843 , piece_id=J74IERT5EP2VDSXY5PNWWBVTHQEXT4CWGLWP72Y3CQLOTSI6XZDA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.033080515 (223h40m52.959375213s) [noise=true , address=104.36.82.11:4115 , piece_id=LOK75B4Y2M3DMRJAHNYCVE46DELAAVN4SFP35AWPFESLUJT2V6KQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.033386908 (223h40m52.959681597s) [noise=true , address=104.36.82.14:4222 , piece_id=3HZK5YRWL2KXKY4E7XASRI2UIPCJA72JSJSTGBLPA4ZBP3MGBEOQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.033709080 (223h40m52.960003768s) [noise=true , address=77.90.31.21:4878 , piece_id=QJARHOYLDCFGYTTCD2EYCH43X3EARFK3Q3C7SCZ33X5T6CTLYDMQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.034030732 (223h40m52.96032542s) [noise=true , address=104.36.82.3:4767 , piece_id=U4L7ULC2R5OKDDL6BGDSEQUZNTVMQSGXJ56CGVKHNCCHTMIIDXDQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.034335911 (223h40m52.960630599s) [noise=true , address=45.91.116.8:4047 , piece_id=IJZDAO2K7KADVBNZGHLACI2ARTDDYOHIC742L2IUJT2WFAMMIGSQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.034644696 (223h40m52.960939386s) [noise=true , address=104.36.82.10:4079 , piece_id=JELVGAGG2NF2OSMKXQAV6DIXDTUFIER5WX4KTCJFQQ5P35EAMOEA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.148244089 (223h40m53.074538765s) [upload=179165]: Next returned nil segment
Jul 2 16:52:26.148268128 (223h40m53.074562811s) [upload=179165]: Waiting for error group managing segments...
Jul 2 16:52:26.155850465 (223h40m53.082145153s) [noise=true , address=45.91.116.5:4298 , piece_id=JUYTY35XKYPNJYE7X3R7UMINTVMXDMNM6YZSZORJWATTVDBZFOMQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.156432201 (223h40m53.082726892s) [noise=true , address=77.90.31.14:4661 , piece_id=B5E2MZ5JW7Z5AG4GFQ4QGMAFW7ZMKWKRFU276Z6556JFWKYVLXTQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.156943265 (223h40m53.083237956s) [noise=true , address=104.36.82.16:4294 , piece_id=36S2PXQADEAQPWSBMQC4XEAH6H5PDA4AO34JI37VD4KPJOU2WVTA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.158591570 (223h40m53.084886251s) [noise=true , address=104.36.82.3:4767 , piece_id=U4L7ULC2R5OKDDL6BGDSEQUZNTVMQSGXJ56CGVKHNCCHTMIIDXDQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.159458953 (223h40m53.085753634s) [noise=true , address=45.91.116.8:4047 , piece_id=IJZDAO2K7KADVBNZGHLACI2ARTDDYOHIC742L2IUJT2WFAMMIGSQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.161102375 (223h40m53.087397055s) [noise=true , address=45.91.116.12:4226 , piece_id=NFB2MT72D7MLPOY3A6MVO2QQBUFVYJPKN7D5XJQLQCCMYNQ6WJOA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.161556163 (223h40m53.087850838s) [noise=true , address=104.36.82.12:4149 , piece_id=V5265CLKUYZQN5B663WB4CFR3SXR2OA2BT53OAVCLFZJJ2YY6U4A, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.162013197 (223h40m53.088307874s) [noise=true , address=104.36.82.14:4222 , piece_id=3HZK5YRWL2KXKY4E7XASRI2UIPCJA72JSJSTGBLPA4ZBP3MGBEOQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.162459436 (223h40m53.088754112s) [noise=true , address=77.90.31.15:4698 , piece_id=VDVQDBYDQYRMVP7ZUPSEFC2R5UF2LVW45G23JURMYKS5IT6RBQFQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.162920549 (223h40m53.089215241s) [noise=true , address=77.90.31.21:4878 , piece_id=QJARHOYLDCFGYTTCD2EYCH43X3EARFK3Q3C7SCZ33X5T6CTLYDMQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.163747035 (223h40m53.090041713s) [noise=true , address=104.36.82.11:4115 , piece_id=LOK75B4Y2M3DMRJAHNYCVE46DELAAVN4SFP35AWPFESLUJT2V6KQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.167885784 (223h40m53.094180467s) [noise=true , address=104.36.82.6:4874 , piece_id=XVVQN3VX4I2KS4CIXEADBSFYBVYTVEWS5GMXAHE2LWDJGD2A2KAA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.168494969 (223h40m53.094789668s) [noise=true , address=104.36.82.10:4079 , piece_id=JELVGAGG2NF2OSMKXQAV6DIXDTUFIER5WX4KTCJFQQ5P35EAMOEA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.168525706 (223h40m53.094820396s) [seg_pos={0 0}, upload=179165]: Segment reached optimal threshold of 13 pieces.
Jul 2 16:52:26.168715378 (223h40m53.09501007s) [noise=true , address=77.90.31.19:4843 , piece_id=J74IERT5EP2VDSXY5PNWWBVTHQEXT4CWGLWP72Y3CQLOTSI6XZDA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: expected piece hash; context canceled; EOF
Jul 2 16:52:26.168766821 (223h40m53.095061509s) [noise=true , address=104.36.82.8:4970 , piece_id=X7KKQJYHVNI5LJDKGYNV5JHHCEGF4QSHFRKMSBLRE5W3W2ITAAHQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe): protocol: expected piece hash; context canceled; EOF
Jul 2 16:52:26.168816209 (223h40m53.095110907s) [noise=true , address=45.91.116.11:4118 , piece_id=CSQO6I5J4BW4NUAOFH3QDF6VEJGACWOSYNTNGXVELK2HLXIMX4YQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF
Jul 2 16:52:26.168851649 (223h40m53.095146335s) [seg_pos={0 0}, upload=179165]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 2 16:52:26.169058251 (223h40m53.095352944s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 16:52:26.169066272 (223h40m53.095360967s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 16:52:26.220481259 (223h40m53.146775936s) [upload=179165]: Done uploading segments...
Jul 2 16:52:26.220500093 (223h40m53.14679477s) [upload=179165]: Upload finished. err: <nil>
Duration: 1.285167431s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................................X...................] duration:
1.739419151s
FAIL (2025-07-02 17:30:52 UTC)
orbiter warning: uplink: metaclient: manager closed: closed: read tcp 159.223.143.39:49282->34.150.199.48:7777: read: connection reset by peer
Duration: 1.739450449s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......X....................................................] duration:
733.092506ms
FAIL (2025-07-02 17:30:33 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
led; ecclient: upload cut due to slow connection (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN): protocol: expected piece hash; context canceled; EOF
Jul 2 16:51:39.852164074 (223h40m6.778458767s) [noise=true , address=104.36.82.12:4150 , piece_id=QHXVON3LSBWYG5AWPVZ45KOTJJPUXIIAXGED7OHG65V5V3IJPULA, seg_pos={0 0}, upload=179153]: Done uploading piece. err: <nil>
Jul 2 16:51:39.852220651 (223h40m6.778515346s) [seg_pos={0 0}, upload=179153]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 2 16:51:39.852324711 (223h40m6.778619405s) [upload=179153]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 16:51:39.852329895 (223h40m6.778624588s) [upload=179153]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 16:51:39.945155283 (223h40m6.871449972s) [upload=179153]: Done uploading segments...
Jul 2 16:51:39.945173208 (223h40m6.871467885s) [upload=179153]: Upload finished. err: <nil>
Jul 2 16:52:25.999553770 (223h40m52.925848461s) [upload=179165]: Starting upload
Jul 2 16:52:25.999676452 (223h40m52.92597114s) [upload=179165]: Done starting upload
Jul 2 16:52:25.999808777 (223h40m52.926103458s) [upload=179165]: Uploading segments...
Jul 2 16:52:26.000049412 (223h40m52.926344119s) [upload=179165]: Got next segment. Inline: false
Jul 2 16:52:26.000208665 (223h40m52.926503358s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 2 16:52:26.000216887 (223h40m52.926511587s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 2 16:52:26.029546629 (223h40m52.95584132s) [seg_pos={0 0}, upload=179165]: Begin upload segment...
Jul 2 16:52:26.029596311 (223h40m52.955890985s) [seg_pos={0 0}, upload=179165]: Done begin upload segment.
Jul 2 16:52:26.029630299 (223h40m52.955924986s) [noise=true , address=45.91.116.12:4226 , piece_id=NFB2MT72D7MLPOY3A6MVO2QQBUFVYJPKN7D5XJQLQCCMYNQ6WJOA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.030073195 (223h40m52.956367894s) [noise=true , address=77.90.31.14:4661 , piece_id=B5E2MZ5JW7Z5AG4GFQ4QGMAFW7ZMKWKRFU276Z6556JFWKYVLXTQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.030398264 (223h40m52.956692955s) [noise=true , address=77.90.31.15:4698 , piece_id=VDVQDBYDQYRMVP7ZUPSEFC2R5UF2LVW45G23JURMYKS5IT6RBQFQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.030716003 (223h40m52.957010692s) [noise=true , address=45.91.116.11:4118 , piece_id=CSQO6I5J4BW4NUAOFH3QDF6VEJGACWOSYNTNGXVELK2HLXIMX4YQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.031068785 (223h40m52.957363472s) [noise=true , address=45.91.116.5:4298 , piece_id=JUYTY35XKYPNJYE7X3R7UMINTVMXDMNM6YZSZORJWATTVDBZFOMQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.031405588 (223h40m52.957700282s) [noise=true , address=104.36.82.8:4970 , piece_id=X7KKQJYHVNI5LJDKGYNV5JHHCEGF4QSHFRKMSBLRE5W3W2ITAAHQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.031730040 (223h40m52.958024715s) [noise=true , address=104.36.82.12:4149 , piece_id=V5265CLKUYZQN5B663WB4CFR3SXR2OA2BT53OAVCLFZJJ2YY6U4A, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.032049179 (223h40m52.958343866s) [noise=true , address=104.36.82.6:4874 , piece_id=XVVQN3VX4I2KS4CIXEADBSFYBVYTVEWS5GMXAHE2LWDJGD2A2KAA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.032432775 (223h40m52.958727467s) [noise=true , address=104.36.82.16:4294 , piece_id=36S2PXQADEAQPWSBMQC4XEAH6H5PDA4AO34JI37VD4KPJOU2WVTA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.032754164 (223h40m52.959048852s) [noise=true , address=77.90.31.19:4843 , piece_id=J74IERT5EP2VDSXY5PNWWBVTHQEXT4CWGLWP72Y3CQLOTSI6XZDA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.033080515 (223h40m52.959375213s) [noise=true , address=104.36.82.11:4115 , piece_id=LOK75B4Y2M3DMRJAHNYCVE46DELAAVN4SFP35AWPFESLUJT2V6KQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.033386908 (223h40m52.959681597s) [noise=true , address=104.36.82.14:4222 , piece_id=3HZK5YRWL2KXKY4E7XASRI2UIPCJA72JSJSTGBLPA4ZBP3MGBEOQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.033709080 (223h40m52.960003768s) [noise=true , address=77.90.31.21:4878 , piece_id=QJARHOYLDCFGYTTCD2EYCH43X3EARFK3Q3C7SCZ33X5T6CTLYDMQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.034030732 (223h40m52.96032542s) [noise=true , address=104.36.82.3:4767 , piece_id=U4L7ULC2R5OKDDL6BGDSEQUZNTVMQSGXJ56CGVKHNCCHTMIIDXDQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.034335911 (223h40m52.960630599s) [noise=true , address=45.91.116.8:4047 , piece_id=IJZDAO2K7KADVBNZGHLACI2ARTDDYOHIC742L2IUJT2WFAMMIGSQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.034644696 (223h40m52.960939386s) [noise=true , address=104.36.82.10:4079 , piece_id=JELVGAGG2NF2OSMKXQAV6DIXDTUFIER5WX4KTCJFQQ5P35EAMOEA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.148244089 (223h40m53.074538765s) [upload=179165]: Next returned nil segment
Jul 2 16:52:26.148268128 (223h40m53.074562811s) [upload=179165]: Waiting for error group managing segments...
Jul 2 16:52:26.155850465 (223h40m53.082145153s) [noise=true , address=45.91.116.5:4298 , piece_id=JUYTY35XKYPNJYE7X3R7UMINTVMXDMNM6YZSZORJWATTVDBZFOMQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.156432201 (223h40m53.082726892s) [noise=true , address=77.90.31.14:4661 , piece_id=B5E2MZ5JW7Z5AG4GFQ4QGMAFW7ZMKWKRFU276Z6556JFWKYVLXTQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.156943265 (223h40m53.083237956s) [noise=true , address=104.36.82.16:4294 , piece_id=36S2PXQADEAQPWSBMQC4XEAH6H5PDA4AO34JI37VD4KPJOU2WVTA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.158591570 (223h40m53.084886251s) [noise=true , address=104.36.82.3:4767 , piece_id=U4L7ULC2R5OKDDL6BGDSEQUZNTVMQSGXJ56CGVKHNCCHTMIIDXDQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.159458953 (223h40m53.085753634s) [noise=true , address=45.91.116.8:4047 , piece_id=IJZDAO2K7KADVBNZGHLACI2ARTDDYOHIC742L2IUJT2WFAMMIGSQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.161102375 (223h40m53.087397055s) [noise=true , address=45.91.116.12:4226 , piece_id=NFB2MT72D7MLPOY3A6MVO2QQBUFVYJPKN7D5XJQLQCCMYNQ6WJOA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.161556163 (223h40m53.087850838s) [noise=true , address=104.36.82.12:4149 , piece_id=V5265CLKUYZQN5B663WB4CFR3SXR2OA2BT53OAVCLFZJJ2YY6U4A, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.162013197 (223h40m53.088307874s) [noise=true , address=104.36.82.14:4222 , piece_id=3HZK5YRWL2KXKY4E7XASRI2UIPCJA72JSJSTGBLPA4ZBP3MGBEOQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.162459436 (223h40m53.088754112s) [noise=true , address=77.90.31.15:4698 , piece_id=VDVQDBYDQYRMVP7ZUPSEFC2R5UF2LVW45G23JURMYKS5IT6RBQFQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.162920549 (223h40m53.089215241s) [noise=true , address=77.90.31.21:4878 , piece_id=QJARHOYLDCFGYTTCD2EYCH43X3EARFK3Q3C7SCZ33X5T6CTLYDMQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.163747035 (223h40m53.090041713s) [noise=true , address=104.36.82.11:4115 , piece_id=LOK75B4Y2M3DMRJAHNYCVE46DELAAVN4SFP35AWPFESLUJT2V6KQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.167885784 (223h40m53.094180467s) [noise=true , address=104.36.82.6:4874 , piece_id=XVVQN3VX4I2KS4CIXEADBSFYBVYTVEWS5GMXAHE2LWDJGD2A2KAA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.168494969 (223h40m53.094789668s) [noise=true , address=104.36.82.10:4079 , piece_id=JELVGAGG2NF2OSMKXQAV6DIXDTUFIER5WX4KTCJFQQ5P35EAMOEA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.168525706 (223h40m53.094820396s) [seg_pos={0 0}, upload=179165]: Segment reached optimal threshold of 13 pieces.
Jul 2 16:52:26.168715378 (223h40m53.09501007s) [noise=true , address=77.90.31.19:4843 , piece_id=J74IERT5EP2VDSXY5PNWWBVTHQEXT4CWGLWP72Y3CQLOTSI6XZDA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: expected piece hash; context canceled; EOF
Jul 2 16:52:26.168766821 (223h40m53.095061509s) [noise=true , address=104.36.82.8:4970 , piece_id=X7KKQJYHVNI5LJDKGYNV5JHHCEGF4QSHFRKMSBLRE5W3W2ITAAHQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe): protocol: expected piece hash; context canceled; EOF
Jul 2 16:52:26.168816209 (223h40m53.095110907s) [noise=true , address=45.91.116.11:4118 , piece_id=CSQO6I5J4BW4NUAOFH3QDF6VEJGACWOSYNTNGXVELK2HLXIMX4YQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF
Jul 2 16:52:26.168851649 (223h40m53.095146335s) [seg_pos={0 0}, upload=179165]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 2 16:52:26.169058251 (223h40m53.095352944s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 16:52:26.169066272 (223h40m53.095360967s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 16:52:26.220481259 (223h40m53.146775936s) [upload=179165]: Done uploading segments...
Jul 2 16:52:26.220500093 (223h40m53.14679477s) [upload=179165]: Upload finished. err: <nil>
Duration: 733.122738ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................................X..................] duration:
1.766498843s
FAIL (2025-07-02 17:30:27 UTC)
orbiter warning: uplink: metaclient: manager closed: closed: read tcp 159.223.143.39:49282->34.150.199.48:7777: read: connection reset by peer
Duration: 1.766530972s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........X...................................................] duration:
507.347701ms
FAIL (2025-07-02 17:29:53 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
led; ecclient: upload cut due to slow connection (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN): protocol: expected piece hash; context canceled; EOF
Jul 2 16:51:39.852164074 (223h40m6.778458767s) [noise=true , address=104.36.82.12:4150 , piece_id=QHXVON3LSBWYG5AWPVZ45KOTJJPUXIIAXGED7OHG65V5V3IJPULA, seg_pos={0 0}, upload=179153]: Done uploading piece. err: <nil>
Jul 2 16:51:39.852220651 (223h40m6.778515346s) [seg_pos={0 0}, upload=179153]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 2 16:51:39.852324711 (223h40m6.778619405s) [upload=179153]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 16:51:39.852329895 (223h40m6.778624588s) [upload=179153]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 16:51:39.945155283 (223h40m6.871449972s) [upload=179153]: Done uploading segments...
Jul 2 16:51:39.945173208 (223h40m6.871467885s) [upload=179153]: Upload finished. err: <nil>
Jul 2 16:52:25.999553770 (223h40m52.925848461s) [upload=179165]: Starting upload
Jul 2 16:52:25.999676452 (223h40m52.92597114s) [upload=179165]: Done starting upload
Jul 2 16:52:25.999808777 (223h40m52.926103458s) [upload=179165]: Uploading segments...
Jul 2 16:52:26.000049412 (223h40m52.926344119s) [upload=179165]: Got next segment. Inline: false
Jul 2 16:52:26.000208665 (223h40m52.926503358s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 2 16:52:26.000216887 (223h40m52.926511587s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 2 16:52:26.029546629 (223h40m52.95584132s) [seg_pos={0 0}, upload=179165]: Begin upload segment...
Jul 2 16:52:26.029596311 (223h40m52.955890985s) [seg_pos={0 0}, upload=179165]: Done begin upload segment.
Jul 2 16:52:26.029630299 (223h40m52.955924986s) [noise=true , address=45.91.116.12:4226 , piece_id=NFB2MT72D7MLPOY3A6MVO2QQBUFVYJPKN7D5XJQLQCCMYNQ6WJOA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.030073195 (223h40m52.956367894s) [noise=true , address=77.90.31.14:4661 , piece_id=B5E2MZ5JW7Z5AG4GFQ4QGMAFW7ZMKWKRFU276Z6556JFWKYVLXTQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.030398264 (223h40m52.956692955s) [noise=true , address=77.90.31.15:4698 , piece_id=VDVQDBYDQYRMVP7ZUPSEFC2R5UF2LVW45G23JURMYKS5IT6RBQFQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.030716003 (223h40m52.957010692s) [noise=true , address=45.91.116.11:4118 , piece_id=CSQO6I5J4BW4NUAOFH3QDF6VEJGACWOSYNTNGXVELK2HLXIMX4YQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.031068785 (223h40m52.957363472s) [noise=true , address=45.91.116.5:4298 , piece_id=JUYTY35XKYPNJYE7X3R7UMINTVMXDMNM6YZSZORJWATTVDBZFOMQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.031405588 (223h40m52.957700282s) [noise=true , address=104.36.82.8:4970 , piece_id=X7KKQJYHVNI5LJDKGYNV5JHHCEGF4QSHFRKMSBLRE5W3W2ITAAHQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.031730040 (223h40m52.958024715s) [noise=true , address=104.36.82.12:4149 , piece_id=V5265CLKUYZQN5B663WB4CFR3SXR2OA2BT53OAVCLFZJJ2YY6U4A, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.032049179 (223h40m52.958343866s) [noise=true , address=104.36.82.6:4874 , piece_id=XVVQN3VX4I2KS4CIXEADBSFYBVYTVEWS5GMXAHE2LWDJGD2A2KAA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.032432775 (223h40m52.958727467s) [noise=true , address=104.36.82.16:4294 , piece_id=36S2PXQADEAQPWSBMQC4XEAH6H5PDA4AO34JI37VD4KPJOU2WVTA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.032754164 (223h40m52.959048852s) [noise=true , address=77.90.31.19:4843 , piece_id=J74IERT5EP2VDSXY5PNWWBVTHQEXT4CWGLWP72Y3CQLOTSI6XZDA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.033080515 (223h40m52.959375213s) [noise=true , address=104.36.82.11:4115 , piece_id=LOK75B4Y2M3DMRJAHNYCVE46DELAAVN4SFP35AWPFESLUJT2V6KQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.033386908 (223h40m52.959681597s) [noise=true , address=104.36.82.14:4222 , piece_id=3HZK5YRWL2KXKY4E7XASRI2UIPCJA72JSJSTGBLPA4ZBP3MGBEOQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.033709080 (223h40m52.960003768s) [noise=true , address=77.90.31.21:4878 , piece_id=QJARHOYLDCFGYTTCD2EYCH43X3EARFK3Q3C7SCZ33X5T6CTLYDMQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.034030732 (223h40m52.96032542s) [noise=true , address=104.36.82.3:4767 , piece_id=U4L7ULC2R5OKDDL6BGDSEQUZNTVMQSGXJ56CGVKHNCCHTMIIDXDQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.034335911 (223h40m52.960630599s) [noise=true , address=45.91.116.8:4047 , piece_id=IJZDAO2K7KADVBNZGHLACI2ARTDDYOHIC742L2IUJT2WFAMMIGSQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.034644696 (223h40m52.960939386s) [noise=true , address=104.36.82.10:4079 , piece_id=JELVGAGG2NF2OSMKXQAV6DIXDTUFIER5WX4KTCJFQQ5P35EAMOEA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.148244089 (223h40m53.074538765s) [upload=179165]: Next returned nil segment
Jul 2 16:52:26.148268128 (223h40m53.074562811s) [upload=179165]: Waiting for error group managing segments...
Jul 2 16:52:26.155850465 (223h40m53.082145153s) [noise=true , address=45.91.116.5:4298 , piece_id=JUYTY35XKYPNJYE7X3R7UMINTVMXDMNM6YZSZORJWATTVDBZFOMQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.156432201 (223h40m53.082726892s) [noise=true , address=77.90.31.14:4661 , piece_id=B5E2MZ5JW7Z5AG4GFQ4QGMAFW7ZMKWKRFU276Z6556JFWKYVLXTQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.156943265 (223h40m53.083237956s) [noise=true , address=104.36.82.16:4294 , piece_id=36S2PXQADEAQPWSBMQC4XEAH6H5PDA4AO34JI37VD4KPJOU2WVTA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.158591570 (223h40m53.084886251s) [noise=true , address=104.36.82.3:4767 , piece_id=U4L7ULC2R5OKDDL6BGDSEQUZNTVMQSGXJ56CGVKHNCCHTMIIDXDQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.159458953 (223h40m53.085753634s) [noise=true , address=45.91.116.8:4047 , piece_id=IJZDAO2K7KADVBNZGHLACI2ARTDDYOHIC742L2IUJT2WFAMMIGSQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.161102375 (223h40m53.087397055s) [noise=true , address=45.91.116.12:4226 , piece_id=NFB2MT72D7MLPOY3A6MVO2QQBUFVYJPKN7D5XJQLQCCMYNQ6WJOA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.161556163 (223h40m53.087850838s) [noise=true , address=104.36.82.12:4149 , piece_id=V5265CLKUYZQN5B663WB4CFR3SXR2OA2BT53OAVCLFZJJ2YY6U4A, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.162013197 (223h40m53.088307874s) [noise=true , address=104.36.82.14:4222 , piece_id=3HZK5YRWL2KXKY4E7XASRI2UIPCJA72JSJSTGBLPA4ZBP3MGBEOQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.162459436 (223h40m53.088754112s) [noise=true , address=77.90.31.15:4698 , piece_id=VDVQDBYDQYRMVP7ZUPSEFC2R5UF2LVW45G23JURMYKS5IT6RBQFQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.162920549 (223h40m53.089215241s) [noise=true , address=77.90.31.21:4878 , piece_id=QJARHOYLDCFGYTTCD2EYCH43X3EARFK3Q3C7SCZ33X5T6CTLYDMQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.163747035 (223h40m53.090041713s) [noise=true , address=104.36.82.11:4115 , piece_id=LOK75B4Y2M3DMRJAHNYCVE46DELAAVN4SFP35AWPFESLUJT2V6KQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.167885784 (223h40m53.094180467s) [noise=true , address=104.36.82.6:4874 , piece_id=XVVQN3VX4I2KS4CIXEADBSFYBVYTVEWS5GMXAHE2LWDJGD2A2KAA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.168494969 (223h40m53.094789668s) [noise=true , address=104.36.82.10:4079 , piece_id=JELVGAGG2NF2OSMKXQAV6DIXDTUFIER5WX4KTCJFQQ5P35EAMOEA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.168525706 (223h40m53.094820396s) [seg_pos={0 0}, upload=179165]: Segment reached optimal threshold of 13 pieces.
Jul 2 16:52:26.168715378 (223h40m53.09501007s) [noise=true , address=77.90.31.19:4843 , piece_id=J74IERT5EP2VDSXY5PNWWBVTHQEXT4CWGLWP72Y3CQLOTSI6XZDA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: expected piece hash; context canceled; EOF
Jul 2 16:52:26.168766821 (223h40m53.095061509s) [noise=true , address=104.36.82.8:4970 , piece_id=X7KKQJYHVNI5LJDKGYNV5JHHCEGF4QSHFRKMSBLRE5W3W2ITAAHQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe): protocol: expected piece hash; context canceled; EOF
Jul 2 16:52:26.168816209 (223h40m53.095110907s) [noise=true , address=45.91.116.11:4118 , piece_id=CSQO6I5J4BW4NUAOFH3QDF6VEJGACWOSYNTNGXVELK2HLXIMX4YQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF
Jul 2 16:52:26.168851649 (223h40m53.095146335s) [seg_pos={0 0}, upload=179165]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 2 16:52:26.169058251 (223h40m53.095352944s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 16:52:26.169066272 (223h40m53.095360967s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 16:52:26.220481259 (223h40m53.146775936s) [upload=179165]: Done uploading segments...
Jul 2 16:52:26.220500093 (223h40m53.14679477s) [upload=179165]: Upload finished. err: <nil>
Duration: 507.376439ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................................X.................] duration:
1.721060218s
FAIL (2025-07-02 17:29:33 UTC)
orbiter warning: uplink: metaclient: manager closed: closed: read tcp 159.223.143.39:49282->34.150.199.48:7777: read: connection reset by peer
Duration: 1.72109511s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................] duration:
1.68179971s
FAIL (2025-07-02 17:28:43 UTC)
orbiter warning: uplink: metaclient: manager closed: closed: read tcp 159.223.143.39:49282->34.150.199.48:7777: read: connection reset by peer
Duration: 1.681828954s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........X..................................................] duration:
489.92454ms
FAIL (2025-07-02 17:28:38 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
led; ecclient: upload cut due to slow connection (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN): protocol: expected piece hash; context canceled; EOF
Jul 2 16:51:39.852164074 (223h40m6.778458767s) [noise=true , address=104.36.82.12:4150 , piece_id=QHXVON3LSBWYG5AWPVZ45KOTJJPUXIIAXGED7OHG65V5V3IJPULA, seg_pos={0 0}, upload=179153]: Done uploading piece. err: <nil>
Jul 2 16:51:39.852220651 (223h40m6.778515346s) [seg_pos={0 0}, upload=179153]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 2 16:51:39.852324711 (223h40m6.778619405s) [upload=179153]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 16:51:39.852329895 (223h40m6.778624588s) [upload=179153]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 16:51:39.945155283 (223h40m6.871449972s) [upload=179153]: Done uploading segments...
Jul 2 16:51:39.945173208 (223h40m6.871467885s) [upload=179153]: Upload finished. err: <nil>
Jul 2 16:52:25.999553770 (223h40m52.925848461s) [upload=179165]: Starting upload
Jul 2 16:52:25.999676452 (223h40m52.92597114s) [upload=179165]: Done starting upload
Jul 2 16:52:25.999808777 (223h40m52.926103458s) [upload=179165]: Uploading segments...
Jul 2 16:52:26.000049412 (223h40m52.926344119s) [upload=179165]: Got next segment. Inline: false
Jul 2 16:52:26.000208665 (223h40m52.926503358s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 2 16:52:26.000216887 (223h40m52.926511587s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 2 16:52:26.029546629 (223h40m52.95584132s) [seg_pos={0 0}, upload=179165]: Begin upload segment...
Jul 2 16:52:26.029596311 (223h40m52.955890985s) [seg_pos={0 0}, upload=179165]: Done begin upload segment.
Jul 2 16:52:26.029630299 (223h40m52.955924986s) [noise=true , address=45.91.116.12:4226 , piece_id=NFB2MT72D7MLPOY3A6MVO2QQBUFVYJPKN7D5XJQLQCCMYNQ6WJOA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.030073195 (223h40m52.956367894s) [noise=true , address=77.90.31.14:4661 , piece_id=B5E2MZ5JW7Z5AG4GFQ4QGMAFW7ZMKWKRFU276Z6556JFWKYVLXTQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.030398264 (223h40m52.956692955s) [noise=true , address=77.90.31.15:4698 , piece_id=VDVQDBYDQYRMVP7ZUPSEFC2R5UF2LVW45G23JURMYKS5IT6RBQFQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.030716003 (223h40m52.957010692s) [noise=true , address=45.91.116.11:4118 , piece_id=CSQO6I5J4BW4NUAOFH3QDF6VEJGACWOSYNTNGXVELK2HLXIMX4YQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.031068785 (223h40m52.957363472s) [noise=true , address=45.91.116.5:4298 , piece_id=JUYTY35XKYPNJYE7X3R7UMINTVMXDMNM6YZSZORJWATTVDBZFOMQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.031405588 (223h40m52.957700282s) [noise=true , address=104.36.82.8:4970 , piece_id=X7KKQJYHVNI5LJDKGYNV5JHHCEGF4QSHFRKMSBLRE5W3W2ITAAHQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.031730040 (223h40m52.958024715s) [noise=true , address=104.36.82.12:4149 , piece_id=V5265CLKUYZQN5B663WB4CFR3SXR2OA2BT53OAVCLFZJJ2YY6U4A, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.032049179 (223h40m52.958343866s) [noise=true , address=104.36.82.6:4874 , piece_id=XVVQN3VX4I2KS4CIXEADBSFYBVYTVEWS5GMXAHE2LWDJGD2A2KAA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.032432775 (223h40m52.958727467s) [noise=true , address=104.36.82.16:4294 , piece_id=36S2PXQADEAQPWSBMQC4XEAH6H5PDA4AO34JI37VD4KPJOU2WVTA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.032754164 (223h40m52.959048852s) [noise=true , address=77.90.31.19:4843 , piece_id=J74IERT5EP2VDSXY5PNWWBVTHQEXT4CWGLWP72Y3CQLOTSI6XZDA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.033080515 (223h40m52.959375213s) [noise=true , address=104.36.82.11:4115 , piece_id=LOK75B4Y2M3DMRJAHNYCVE46DELAAVN4SFP35AWPFESLUJT2V6KQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.033386908 (223h40m52.959681597s) [noise=true , address=104.36.82.14:4222 , piece_id=3HZK5YRWL2KXKY4E7XASRI2UIPCJA72JSJSTGBLPA4ZBP3MGBEOQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.033709080 (223h40m52.960003768s) [noise=true , address=77.90.31.21:4878 , piece_id=QJARHOYLDCFGYTTCD2EYCH43X3EARFK3Q3C7SCZ33X5T6CTLYDMQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.034030732 (223h40m52.96032542s) [noise=true , address=104.36.82.3:4767 , piece_id=U4L7ULC2R5OKDDL6BGDSEQUZNTVMQSGXJ56CGVKHNCCHTMIIDXDQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.034335911 (223h40m52.960630599s) [noise=true , address=45.91.116.8:4047 , piece_id=IJZDAO2K7KADVBNZGHLACI2ARTDDYOHIC742L2IUJT2WFAMMIGSQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.034644696 (223h40m52.960939386s) [noise=true , address=104.36.82.10:4079 , piece_id=JELVGAGG2NF2OSMKXQAV6DIXDTUFIER5WX4KTCJFQQ5P35EAMOEA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.148244089 (223h40m53.074538765s) [upload=179165]: Next returned nil segment
Jul 2 16:52:26.148268128 (223h40m53.074562811s) [upload=179165]: Waiting for error group managing segments...
Jul 2 16:52:26.155850465 (223h40m53.082145153s) [noise=true , address=45.91.116.5:4298 , piece_id=JUYTY35XKYPNJYE7X3R7UMINTVMXDMNM6YZSZORJWATTVDBZFOMQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.156432201 (223h40m53.082726892s) [noise=true , address=77.90.31.14:4661 , piece_id=B5E2MZ5JW7Z5AG4GFQ4QGMAFW7ZMKWKRFU276Z6556JFWKYVLXTQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.156943265 (223h40m53.083237956s) [noise=true , address=104.36.82.16:4294 , piece_id=36S2PXQADEAQPWSBMQC4XEAH6H5PDA4AO34JI37VD4KPJOU2WVTA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.158591570 (223h40m53.084886251s) [noise=true , address=104.36.82.3:4767 , piece_id=U4L7ULC2R5OKDDL6BGDSEQUZNTVMQSGXJ56CGVKHNCCHTMIIDXDQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.159458953 (223h40m53.085753634s) [noise=true , address=45.91.116.8:4047 , piece_id=IJZDAO2K7KADVBNZGHLACI2ARTDDYOHIC742L2IUJT2WFAMMIGSQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.161102375 (223h40m53.087397055s) [noise=true , address=45.91.116.12:4226 , piece_id=NFB2MT72D7MLPOY3A6MVO2QQBUFVYJPKN7D5XJQLQCCMYNQ6WJOA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.161556163 (223h40m53.087850838s) [noise=true , address=104.36.82.12:4149 , piece_id=V5265CLKUYZQN5B663WB4CFR3SXR2OA2BT53OAVCLFZJJ2YY6U4A, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.162013197 (223h40m53.088307874s) [noise=true , address=104.36.82.14:4222 , piece_id=3HZK5YRWL2KXKY4E7XASRI2UIPCJA72JSJSTGBLPA4ZBP3MGBEOQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.162459436 (223h40m53.088754112s) [noise=true , address=77.90.31.15:4698 , piece_id=VDVQDBYDQYRMVP7ZUPSEFC2R5UF2LVW45G23JURMYKS5IT6RBQFQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.162920549 (223h40m53.089215241s) [noise=true , address=77.90.31.21:4878 , piece_id=QJARHOYLDCFGYTTCD2EYCH43X3EARFK3Q3C7SCZ33X5T6CTLYDMQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.163747035 (223h40m53.090041713s) [noise=true , address=104.36.82.11:4115 , piece_id=LOK75B4Y2M3DMRJAHNYCVE46DELAAVN4SFP35AWPFESLUJT2V6KQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.167885784 (223h40m53.094180467s) [noise=true , address=104.36.82.6:4874 , piece_id=XVVQN3VX4I2KS4CIXEADBSFYBVYTVEWS5GMXAHE2LWDJGD2A2KAA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.168494969 (223h40m53.094789668s) [noise=true , address=104.36.82.10:4079 , piece_id=JELVGAGG2NF2OSMKXQAV6DIXDTUFIER5WX4KTCJFQQ5P35EAMOEA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.168525706 (223h40m53.094820396s) [seg_pos={0 0}, upload=179165]: Segment reached optimal threshold of 13 pieces.
Jul 2 16:52:26.168715378 (223h40m53.09501007s) [noise=true , address=77.90.31.19:4843 , piece_id=J74IERT5EP2VDSXY5PNWWBVTHQEXT4CWGLWP72Y3CQLOTSI6XZDA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: expected piece hash; context canceled; EOF
Jul 2 16:52:26.168766821 (223h40m53.095061509s) [noise=true , address=104.36.82.8:4970 , piece_id=X7KKQJYHVNI5LJDKGYNV5JHHCEGF4QSHFRKMSBLRE5W3W2ITAAHQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe): protocol: expected piece hash; context canceled; EOF
Jul 2 16:52:26.168816209 (223h40m53.095110907s) [noise=true , address=45.91.116.11:4118 , piece_id=CSQO6I5J4BW4NUAOFH3QDF6VEJGACWOSYNTNGXVELK2HLXIMX4YQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF
Jul 2 16:52:26.168851649 (223h40m53.095146335s) [seg_pos={0 0}, upload=179165]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 2 16:52:26.169058251 (223h40m53.095352944s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 16:52:26.169066272 (223h40m53.095360967s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 16:52:26.220481259 (223h40m53.146775936s) [upload=179165]: Done uploading segments...
Jul 2 16:52:26.220500093 (223h40m53.14679477s) [upload=179165]: Upload finished. err: <nil>
Duration: 490.111794ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........X.................................................] duration:
461.37688ms
FAIL (2025-07-02 17:28:04 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
led; ecclient: upload cut due to slow connection (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN): protocol: expected piece hash; context canceled; EOF
Jul 2 16:51:39.852164074 (223h40m6.778458767s) [noise=true , address=104.36.82.12:4150 , piece_id=QHXVON3LSBWYG5AWPVZ45KOTJJPUXIIAXGED7OHG65V5V3IJPULA, seg_pos={0 0}, upload=179153]: Done uploading piece. err: <nil>
Jul 2 16:51:39.852220651 (223h40m6.778515346s) [seg_pos={0 0}, upload=179153]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 2 16:51:39.852324711 (223h40m6.778619405s) [upload=179153]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 16:51:39.852329895 (223h40m6.778624588s) [upload=179153]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 16:51:39.945155283 (223h40m6.871449972s) [upload=179153]: Done uploading segments...
Jul 2 16:51:39.945173208 (223h40m6.871467885s) [upload=179153]: Upload finished. err: <nil>
Jul 2 16:52:25.999553770 (223h40m52.925848461s) [upload=179165]: Starting upload
Jul 2 16:52:25.999676452 (223h40m52.92597114s) [upload=179165]: Done starting upload
Jul 2 16:52:25.999808777 (223h40m52.926103458s) [upload=179165]: Uploading segments...
Jul 2 16:52:26.000049412 (223h40m52.926344119s) [upload=179165]: Got next segment. Inline: false
Jul 2 16:52:26.000208665 (223h40m52.926503358s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 2 16:52:26.000216887 (223h40m52.926511587s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 2 16:52:26.029546629 (223h40m52.95584132s) [seg_pos={0 0}, upload=179165]: Begin upload segment...
Jul 2 16:52:26.029596311 (223h40m52.955890985s) [seg_pos={0 0}, upload=179165]: Done begin upload segment.
Jul 2 16:52:26.029630299 (223h40m52.955924986s) [noise=true , address=45.91.116.12:4226 , piece_id=NFB2MT72D7MLPOY3A6MVO2QQBUFVYJPKN7D5XJQLQCCMYNQ6WJOA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.030073195 (223h40m52.956367894s) [noise=true , address=77.90.31.14:4661 , piece_id=B5E2MZ5JW7Z5AG4GFQ4QGMAFW7ZMKWKRFU276Z6556JFWKYVLXTQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.030398264 (223h40m52.956692955s) [noise=true , address=77.90.31.15:4698 , piece_id=VDVQDBYDQYRMVP7ZUPSEFC2R5UF2LVW45G23JURMYKS5IT6RBQFQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.030716003 (223h40m52.957010692s) [noise=true , address=45.91.116.11:4118 , piece_id=CSQO6I5J4BW4NUAOFH3QDF6VEJGACWOSYNTNGXVELK2HLXIMX4YQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.031068785 (223h40m52.957363472s) [noise=true , address=45.91.116.5:4298 , piece_id=JUYTY35XKYPNJYE7X3R7UMINTVMXDMNM6YZSZORJWATTVDBZFOMQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.031405588 (223h40m52.957700282s) [noise=true , address=104.36.82.8:4970 , piece_id=X7KKQJYHVNI5LJDKGYNV5JHHCEGF4QSHFRKMSBLRE5W3W2ITAAHQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.031730040 (223h40m52.958024715s) [noise=true , address=104.36.82.12:4149 , piece_id=V5265CLKUYZQN5B663WB4CFR3SXR2OA2BT53OAVCLFZJJ2YY6U4A, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.032049179 (223h40m52.958343866s) [noise=true , address=104.36.82.6:4874 , piece_id=XVVQN3VX4I2KS4CIXEADBSFYBVYTVEWS5GMXAHE2LWDJGD2A2KAA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.032432775 (223h40m52.958727467s) [noise=true , address=104.36.82.16:4294 , piece_id=36S2PXQADEAQPWSBMQC4XEAH6H5PDA4AO34JI37VD4KPJOU2WVTA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.032754164 (223h40m52.959048852s) [noise=true , address=77.90.31.19:4843 , piece_id=J74IERT5EP2VDSXY5PNWWBVTHQEXT4CWGLWP72Y3CQLOTSI6XZDA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.033080515 (223h40m52.959375213s) [noise=true , address=104.36.82.11:4115 , piece_id=LOK75B4Y2M3DMRJAHNYCVE46DELAAVN4SFP35AWPFESLUJT2V6KQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.033386908 (223h40m52.959681597s) [noise=true , address=104.36.82.14:4222 , piece_id=3HZK5YRWL2KXKY4E7XASRI2UIPCJA72JSJSTGBLPA4ZBP3MGBEOQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.033709080 (223h40m52.960003768s) [noise=true , address=77.90.31.21:4878 , piece_id=QJARHOYLDCFGYTTCD2EYCH43X3EARFK3Q3C7SCZ33X5T6CTLYDMQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.034030732 (223h40m52.96032542s) [noise=true , address=104.36.82.3:4767 , piece_id=U4L7ULC2R5OKDDL6BGDSEQUZNTVMQSGXJ56CGVKHNCCHTMIIDXDQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.034335911 (223h40m52.960630599s) [noise=true , address=45.91.116.8:4047 , piece_id=IJZDAO2K7KADVBNZGHLACI2ARTDDYOHIC742L2IUJT2WFAMMIGSQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.034644696 (223h40m52.960939386s) [noise=true , address=104.36.82.10:4079 , piece_id=JELVGAGG2NF2OSMKXQAV6DIXDTUFIER5WX4KTCJFQQ5P35EAMOEA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.148244089 (223h40m53.074538765s) [upload=179165]: Next returned nil segment
Jul 2 16:52:26.148268128 (223h40m53.074562811s) [upload=179165]: Waiting for error group managing segments...
Jul 2 16:52:26.155850465 (223h40m53.082145153s) [noise=true , address=45.91.116.5:4298 , piece_id=JUYTY35XKYPNJYE7X3R7UMINTVMXDMNM6YZSZORJWATTVDBZFOMQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.156432201 (223h40m53.082726892s) [noise=true , address=77.90.31.14:4661 , piece_id=B5E2MZ5JW7Z5AG4GFQ4QGMAFW7ZMKWKRFU276Z6556JFWKYVLXTQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.156943265 (223h40m53.083237956s) [noise=true , address=104.36.82.16:4294 , piece_id=36S2PXQADEAQPWSBMQC4XEAH6H5PDA4AO34JI37VD4KPJOU2WVTA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.158591570 (223h40m53.084886251s) [noise=true , address=104.36.82.3:4767 , piece_id=U4L7ULC2R5OKDDL6BGDSEQUZNTVMQSGXJ56CGVKHNCCHTMIIDXDQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.159458953 (223h40m53.085753634s) [noise=true , address=45.91.116.8:4047 , piece_id=IJZDAO2K7KADVBNZGHLACI2ARTDDYOHIC742L2IUJT2WFAMMIGSQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.161102375 (223h40m53.087397055s) [noise=true , address=45.91.116.12:4226 , piece_id=NFB2MT72D7MLPOY3A6MVO2QQBUFVYJPKN7D5XJQLQCCMYNQ6WJOA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.161556163 (223h40m53.087850838s) [noise=true , address=104.36.82.12:4149 , piece_id=V5265CLKUYZQN5B663WB4CFR3SXR2OA2BT53OAVCLFZJJ2YY6U4A, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.162013197 (223h40m53.088307874s) [noise=true , address=104.36.82.14:4222 , piece_id=3HZK5YRWL2KXKY4E7XASRI2UIPCJA72JSJSTGBLPA4ZBP3MGBEOQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.162459436 (223h40m53.088754112s) [noise=true , address=77.90.31.15:4698 , piece_id=VDVQDBYDQYRMVP7ZUPSEFC2R5UF2LVW45G23JURMYKS5IT6RBQFQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.162920549 (223h40m53.089215241s) [noise=true , address=77.90.31.21:4878 , piece_id=QJARHOYLDCFGYTTCD2EYCH43X3EARFK3Q3C7SCZ33X5T6CTLYDMQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.163747035 (223h40m53.090041713s) [noise=true , address=104.36.82.11:4115 , piece_id=LOK75B4Y2M3DMRJAHNYCVE46DELAAVN4SFP35AWPFESLUJT2V6KQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.167885784 (223h40m53.094180467s) [noise=true , address=104.36.82.6:4874 , piece_id=XVVQN3VX4I2KS4CIXEADBSFYBVYTVEWS5GMXAHE2LWDJGD2A2KAA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.168494969 (223h40m53.094789668s) [noise=true , address=104.36.82.10:4079 , piece_id=JELVGAGG2NF2OSMKXQAV6DIXDTUFIER5WX4KTCJFQQ5P35EAMOEA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.168525706 (223h40m53.094820396s) [seg_pos={0 0}, upload=179165]: Segment reached optimal threshold of 13 pieces.
Jul 2 16:52:26.168715378 (223h40m53.09501007s) [noise=true , address=77.90.31.19:4843 , piece_id=J74IERT5EP2VDSXY5PNWWBVTHQEXT4CWGLWP72Y3CQLOTSI6XZDA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: expected piece hash; context canceled; EOF
Jul 2 16:52:26.168766821 (223h40m53.095061509s) [noise=true , address=104.36.82.8:4970 , piece_id=X7KKQJYHVNI5LJDKGYNV5JHHCEGF4QSHFRKMSBLRE5W3W2ITAAHQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe): protocol: expected piece hash; context canceled; EOF
Jul 2 16:52:26.168816209 (223h40m53.095110907s) [noise=true , address=45.91.116.11:4118 , piece_id=CSQO6I5J4BW4NUAOFH3QDF6VEJGACWOSYNTNGXVELK2HLXIMX4YQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF
Jul 2 16:52:26.168851649 (223h40m53.095146335s) [seg_pos={0 0}, upload=179165]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 2 16:52:26.169058251 (223h40m53.095352944s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 16:52:26.169066272 (223h40m53.095360967s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 16:52:26.220481259 (223h40m53.146775936s) [upload=179165]: Done uploading segments...
Jul 2 16:52:26.220500093 (223h40m53.14679477s) [upload=179165]: Upload finished. err: <nil>
Duration: 461.430073ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............] duration:
1.706436965s
FAIL (2025-07-02 17:27:59 UTC)
orbiter warning: uplink: metaclient: manager closed: closed: read tcp 159.223.143.39:49282->34.150.199.48:7777: read: connection reset by peer
Duration: 1.706469123s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............] duration:
1.75104158s
FAIL (2025-07-02 17:27:29 UTC)
orbiter warning: uplink: metaclient: manager closed: closed: read tcp 159.223.143.39:49282->34.150.199.48:7777: read: connection reset by peer
Duration: 1.751072099s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........X................................................] duration:
465.318272ms
FAIL (2025-07-02 17:27:26 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
led; ecclient: upload cut due to slow connection (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN): protocol: expected piece hash; context canceled; EOF
Jul 2 16:51:39.852164074 (223h40m6.778458767s) [noise=true , address=104.36.82.12:4150 , piece_id=QHXVON3LSBWYG5AWPVZ45KOTJJPUXIIAXGED7OHG65V5V3IJPULA, seg_pos={0 0}, upload=179153]: Done uploading piece. err: <nil>
Jul 2 16:51:39.852220651 (223h40m6.778515346s) [seg_pos={0 0}, upload=179153]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 2 16:51:39.852324711 (223h40m6.778619405s) [upload=179153]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 16:51:39.852329895 (223h40m6.778624588s) [upload=179153]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 16:51:39.945155283 (223h40m6.871449972s) [upload=179153]: Done uploading segments...
Jul 2 16:51:39.945173208 (223h40m6.871467885s) [upload=179153]: Upload finished. err: <nil>
Jul 2 16:52:25.999553770 (223h40m52.925848461s) [upload=179165]: Starting upload
Jul 2 16:52:25.999676452 (223h40m52.92597114s) [upload=179165]: Done starting upload
Jul 2 16:52:25.999808777 (223h40m52.926103458s) [upload=179165]: Uploading segments...
Jul 2 16:52:26.000049412 (223h40m52.926344119s) [upload=179165]: Got next segment. Inline: false
Jul 2 16:52:26.000208665 (223h40m52.926503358s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 2 16:52:26.000216887 (223h40m52.926511587s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 2 16:52:26.029546629 (223h40m52.95584132s) [seg_pos={0 0}, upload=179165]: Begin upload segment...
Jul 2 16:52:26.029596311 (223h40m52.955890985s) [seg_pos={0 0}, upload=179165]: Done begin upload segment.
Jul 2 16:52:26.029630299 (223h40m52.955924986s) [noise=true , address=45.91.116.12:4226 , piece_id=NFB2MT72D7MLPOY3A6MVO2QQBUFVYJPKN7D5XJQLQCCMYNQ6WJOA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.030073195 (223h40m52.956367894s) [noise=true , address=77.90.31.14:4661 , piece_id=B5E2MZ5JW7Z5AG4GFQ4QGMAFW7ZMKWKRFU276Z6556JFWKYVLXTQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.030398264 (223h40m52.956692955s) [noise=true , address=77.90.31.15:4698 , piece_id=VDVQDBYDQYRMVP7ZUPSEFC2R5UF2LVW45G23JURMYKS5IT6RBQFQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.030716003 (223h40m52.957010692s) [noise=true , address=45.91.116.11:4118 , piece_id=CSQO6I5J4BW4NUAOFH3QDF6VEJGACWOSYNTNGXVELK2HLXIMX4YQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.031068785 (223h40m52.957363472s) [noise=true , address=45.91.116.5:4298 , piece_id=JUYTY35XKYPNJYE7X3R7UMINTVMXDMNM6YZSZORJWATTVDBZFOMQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.031405588 (223h40m52.957700282s) [noise=true , address=104.36.82.8:4970 , piece_id=X7KKQJYHVNI5LJDKGYNV5JHHCEGF4QSHFRKMSBLRE5W3W2ITAAHQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.031730040 (223h40m52.958024715s) [noise=true , address=104.36.82.12:4149 , piece_id=V5265CLKUYZQN5B663WB4CFR3SXR2OA2BT53OAVCLFZJJ2YY6U4A, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.032049179 (223h40m52.958343866s) [noise=true , address=104.36.82.6:4874 , piece_id=XVVQN3VX4I2KS4CIXEADBSFYBVYTVEWS5GMXAHE2LWDJGD2A2KAA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.032432775 (223h40m52.958727467s) [noise=true , address=104.36.82.16:4294 , piece_id=36S2PXQADEAQPWSBMQC4XEAH6H5PDA4AO34JI37VD4KPJOU2WVTA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.032754164 (223h40m52.959048852s) [noise=true , address=77.90.31.19:4843 , piece_id=J74IERT5EP2VDSXY5PNWWBVTHQEXT4CWGLWP72Y3CQLOTSI6XZDA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.033080515 (223h40m52.959375213s) [noise=true , address=104.36.82.11:4115 , piece_id=LOK75B4Y2M3DMRJAHNYCVE46DELAAVN4SFP35AWPFESLUJT2V6KQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.033386908 (223h40m52.959681597s) [noise=true , address=104.36.82.14:4222 , piece_id=3HZK5YRWL2KXKY4E7XASRI2UIPCJA72JSJSTGBLPA4ZBP3MGBEOQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.033709080 (223h40m52.960003768s) [noise=true , address=77.90.31.21:4878 , piece_id=QJARHOYLDCFGYTTCD2EYCH43X3EARFK3Q3C7SCZ33X5T6CTLYDMQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.034030732 (223h40m52.96032542s) [noise=true , address=104.36.82.3:4767 , piece_id=U4L7ULC2R5OKDDL6BGDSEQUZNTVMQSGXJ56CGVKHNCCHTMIIDXDQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.034335911 (223h40m52.960630599s) [noise=true , address=45.91.116.8:4047 , piece_id=IJZDAO2K7KADVBNZGHLACI2ARTDDYOHIC742L2IUJT2WFAMMIGSQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.034644696 (223h40m52.960939386s) [noise=true , address=104.36.82.10:4079 , piece_id=JELVGAGG2NF2OSMKXQAV6DIXDTUFIER5WX4KTCJFQQ5P35EAMOEA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.148244089 (223h40m53.074538765s) [upload=179165]: Next returned nil segment
Jul 2 16:52:26.148268128 (223h40m53.074562811s) [upload=179165]: Waiting for error group managing segments...
Jul 2 16:52:26.155850465 (223h40m53.082145153s) [noise=true , address=45.91.116.5:4298 , piece_id=JUYTY35XKYPNJYE7X3R7UMINTVMXDMNM6YZSZORJWATTVDBZFOMQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.156432201 (223h40m53.082726892s) [noise=true , address=77.90.31.14:4661 , piece_id=B5E2MZ5JW7Z5AG4GFQ4QGMAFW7ZMKWKRFU276Z6556JFWKYVLXTQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.156943265 (223h40m53.083237956s) [noise=true , address=104.36.82.16:4294 , piece_id=36S2PXQADEAQPWSBMQC4XEAH6H5PDA4AO34JI37VD4KPJOU2WVTA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.158591570 (223h40m53.084886251s) [noise=true , address=104.36.82.3:4767 , piece_id=U4L7ULC2R5OKDDL6BGDSEQUZNTVMQSGXJ56CGVKHNCCHTMIIDXDQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.159458953 (223h40m53.085753634s) [noise=true , address=45.91.116.8:4047 , piece_id=IJZDAO2K7KADVBNZGHLACI2ARTDDYOHIC742L2IUJT2WFAMMIGSQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.161102375 (223h40m53.087397055s) [noise=true , address=45.91.116.12:4226 , piece_id=NFB2MT72D7MLPOY3A6MVO2QQBUFVYJPKN7D5XJQLQCCMYNQ6WJOA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.161556163 (223h40m53.087850838s) [noise=true , address=104.36.82.12:4149 , piece_id=V5265CLKUYZQN5B663WB4CFR3SXR2OA2BT53OAVCLFZJJ2YY6U4A, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.162013197 (223h40m53.088307874s) [noise=true , address=104.36.82.14:4222 , piece_id=3HZK5YRWL2KXKY4E7XASRI2UIPCJA72JSJSTGBLPA4ZBP3MGBEOQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.162459436 (223h40m53.088754112s) [noise=true , address=77.90.31.15:4698 , piece_id=VDVQDBYDQYRMVP7ZUPSEFC2R5UF2LVW45G23JURMYKS5IT6RBQFQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.162920549 (223h40m53.089215241s) [noise=true , address=77.90.31.21:4878 , piece_id=QJARHOYLDCFGYTTCD2EYCH43X3EARFK3Q3C7SCZ33X5T6CTLYDMQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.163747035 (223h40m53.090041713s) [noise=true , address=104.36.82.11:4115 , piece_id=LOK75B4Y2M3DMRJAHNYCVE46DELAAVN4SFP35AWPFESLUJT2V6KQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.167885784 (223h40m53.094180467s) [noise=true , address=104.36.82.6:4874 , piece_id=XVVQN3VX4I2KS4CIXEADBSFYBVYTVEWS5GMXAHE2LWDJGD2A2KAA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.168494969 (223h40m53.094789668s) [noise=true , address=104.36.82.10:4079 , piece_id=JELVGAGG2NF2OSMKXQAV6DIXDTUFIER5WX4KTCJFQQ5P35EAMOEA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.168525706 (223h40m53.094820396s) [seg_pos={0 0}, upload=179165]: Segment reached optimal threshold of 13 pieces.
Jul 2 16:52:26.168715378 (223h40m53.09501007s) [noise=true , address=77.90.31.19:4843 , piece_id=J74IERT5EP2VDSXY5PNWWBVTHQEXT4CWGLWP72Y3CQLOTSI6XZDA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: expected piece hash; context canceled; EOF
Jul 2 16:52:26.168766821 (223h40m53.095061509s) [noise=true , address=104.36.82.8:4970 , piece_id=X7KKQJYHVNI5LJDKGYNV5JHHCEGF4QSHFRKMSBLRE5W3W2ITAAHQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe): protocol: expected piece hash; context canceled; EOF
Jul 2 16:52:26.168816209 (223h40m53.095110907s) [noise=true , address=45.91.116.11:4118 , piece_id=CSQO6I5J4BW4NUAOFH3QDF6VEJGACWOSYNTNGXVELK2HLXIMX4YQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF
Jul 2 16:52:26.168851649 (223h40m53.095146335s) [seg_pos={0 0}, upload=179165]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 2 16:52:26.169058251 (223h40m53.095352944s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 16:52:26.169066272 (223h40m53.095360967s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 16:52:26.220481259 (223h40m53.146775936s) [upload=179165]: Done uploading segments...
Jul 2 16:52:26.220500093 (223h40m53.14679477s) [upload=179165]: Upload finished. err: <nil>
Duration: 465.366605ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [............X...............................................] duration:
451.454953ms
FAIL (2025-07-02 17:26:22 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
led; ecclient: upload cut due to slow connection (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN): protocol: expected piece hash; context canceled; EOF
Jul 2 16:51:39.852164074 (223h40m6.778458767s) [noise=true , address=104.36.82.12:4150 , piece_id=QHXVON3LSBWYG5AWPVZ45KOTJJPUXIIAXGED7OHG65V5V3IJPULA, seg_pos={0 0}, upload=179153]: Done uploading piece. err: <nil>
Jul 2 16:51:39.852220651 (223h40m6.778515346s) [seg_pos={0 0}, upload=179153]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 2 16:51:39.852324711 (223h40m6.778619405s) [upload=179153]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 16:51:39.852329895 (223h40m6.778624588s) [upload=179153]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 16:51:39.945155283 (223h40m6.871449972s) [upload=179153]: Done uploading segments...
Jul 2 16:51:39.945173208 (223h40m6.871467885s) [upload=179153]: Upload finished. err: <nil>
Jul 2 16:52:25.999553770 (223h40m52.925848461s) [upload=179165]: Starting upload
Jul 2 16:52:25.999676452 (223h40m52.92597114s) [upload=179165]: Done starting upload
Jul 2 16:52:25.999808777 (223h40m52.926103458s) [upload=179165]: Uploading segments...
Jul 2 16:52:26.000049412 (223h40m52.926344119s) [upload=179165]: Got next segment. Inline: false
Jul 2 16:52:26.000208665 (223h40m52.926503358s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 2 16:52:26.000216887 (223h40m52.926511587s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 2 16:52:26.029546629 (223h40m52.95584132s) [seg_pos={0 0}, upload=179165]: Begin upload segment...
Jul 2 16:52:26.029596311 (223h40m52.955890985s) [seg_pos={0 0}, upload=179165]: Done begin upload segment.
Jul 2 16:52:26.029630299 (223h40m52.955924986s) [noise=true , address=45.91.116.12:4226 , piece_id=NFB2MT72D7MLPOY3A6MVO2QQBUFVYJPKN7D5XJQLQCCMYNQ6WJOA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.030073195 (223h40m52.956367894s) [noise=true , address=77.90.31.14:4661 , piece_id=B5E2MZ5JW7Z5AG4GFQ4QGMAFW7ZMKWKRFU276Z6556JFWKYVLXTQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.030398264 (223h40m52.956692955s) [noise=true , address=77.90.31.15:4698 , piece_id=VDVQDBYDQYRMVP7ZUPSEFC2R5UF2LVW45G23JURMYKS5IT6RBQFQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.030716003 (223h40m52.957010692s) [noise=true , address=45.91.116.11:4118 , piece_id=CSQO6I5J4BW4NUAOFH3QDF6VEJGACWOSYNTNGXVELK2HLXIMX4YQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.031068785 (223h40m52.957363472s) [noise=true , address=45.91.116.5:4298 , piece_id=JUYTY35XKYPNJYE7X3R7UMINTVMXDMNM6YZSZORJWATTVDBZFOMQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.031405588 (223h40m52.957700282s) [noise=true , address=104.36.82.8:4970 , piece_id=X7KKQJYHVNI5LJDKGYNV5JHHCEGF4QSHFRKMSBLRE5W3W2ITAAHQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.031730040 (223h40m52.958024715s) [noise=true , address=104.36.82.12:4149 , piece_id=V5265CLKUYZQN5B663WB4CFR3SXR2OA2BT53OAVCLFZJJ2YY6U4A, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.032049179 (223h40m52.958343866s) [noise=true , address=104.36.82.6:4874 , piece_id=XVVQN3VX4I2KS4CIXEADBSFYBVYTVEWS5GMXAHE2LWDJGD2A2KAA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.032432775 (223h40m52.958727467s) [noise=true , address=104.36.82.16:4294 , piece_id=36S2PXQADEAQPWSBMQC4XEAH6H5PDA4AO34JI37VD4KPJOU2WVTA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.032754164 (223h40m52.959048852s) [noise=true , address=77.90.31.19:4843 , piece_id=J74IERT5EP2VDSXY5PNWWBVTHQEXT4CWGLWP72Y3CQLOTSI6XZDA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.033080515 (223h40m52.959375213s) [noise=true , address=104.36.82.11:4115 , piece_id=LOK75B4Y2M3DMRJAHNYCVE46DELAAVN4SFP35AWPFESLUJT2V6KQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.033386908 (223h40m52.959681597s) [noise=true , address=104.36.82.14:4222 , piece_id=3HZK5YRWL2KXKY4E7XASRI2UIPCJA72JSJSTGBLPA4ZBP3MGBEOQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.033709080 (223h40m52.960003768s) [noise=true , address=77.90.31.21:4878 , piece_id=QJARHOYLDCFGYTTCD2EYCH43X3EARFK3Q3C7SCZ33X5T6CTLYDMQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.034030732 (223h40m52.96032542s) [noise=true , address=104.36.82.3:4767 , piece_id=U4L7ULC2R5OKDDL6BGDSEQUZNTVMQSGXJ56CGVKHNCCHTMIIDXDQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.034335911 (223h40m52.960630599s) [noise=true , address=45.91.116.8:4047 , piece_id=IJZDAO2K7KADVBNZGHLACI2ARTDDYOHIC742L2IUJT2WFAMMIGSQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.034644696 (223h40m52.960939386s) [noise=true , address=104.36.82.10:4079 , piece_id=JELVGAGG2NF2OSMKXQAV6DIXDTUFIER5WX4KTCJFQQ5P35EAMOEA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.148244089 (223h40m53.074538765s) [upload=179165]: Next returned nil segment
Jul 2 16:52:26.148268128 (223h40m53.074562811s) [upload=179165]: Waiting for error group managing segments...
Jul 2 16:52:26.155850465 (223h40m53.082145153s) [noise=true , address=45.91.116.5:4298 , piece_id=JUYTY35XKYPNJYE7X3R7UMINTVMXDMNM6YZSZORJWATTVDBZFOMQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.156432201 (223h40m53.082726892s) [noise=true , address=77.90.31.14:4661 , piece_id=B5E2MZ5JW7Z5AG4GFQ4QGMAFW7ZMKWKRFU276Z6556JFWKYVLXTQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.156943265 (223h40m53.083237956s) [noise=true , address=104.36.82.16:4294 , piece_id=36S2PXQADEAQPWSBMQC4XEAH6H5PDA4AO34JI37VD4KPJOU2WVTA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.158591570 (223h40m53.084886251s) [noise=true , address=104.36.82.3:4767 , piece_id=U4L7ULC2R5OKDDL6BGDSEQUZNTVMQSGXJ56CGVKHNCCHTMIIDXDQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.159458953 (223h40m53.085753634s) [noise=true , address=45.91.116.8:4047 , piece_id=IJZDAO2K7KADVBNZGHLACI2ARTDDYOHIC742L2IUJT2WFAMMIGSQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.161102375 (223h40m53.087397055s) [noise=true , address=45.91.116.12:4226 , piece_id=NFB2MT72D7MLPOY3A6MVO2QQBUFVYJPKN7D5XJQLQCCMYNQ6WJOA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.161556163 (223h40m53.087850838s) [noise=true , address=104.36.82.12:4149 , piece_id=V5265CLKUYZQN5B663WB4CFR3SXR2OA2BT53OAVCLFZJJ2YY6U4A, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.162013197 (223h40m53.088307874s) [noise=true , address=104.36.82.14:4222 , piece_id=3HZK5YRWL2KXKY4E7XASRI2UIPCJA72JSJSTGBLPA4ZBP3MGBEOQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.162459436 (223h40m53.088754112s) [noise=true , address=77.90.31.15:4698 , piece_id=VDVQDBYDQYRMVP7ZUPSEFC2R5UF2LVW45G23JURMYKS5IT6RBQFQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.162920549 (223h40m53.089215241s) [noise=true , address=77.90.31.21:4878 , piece_id=QJARHOYLDCFGYTTCD2EYCH43X3EARFK3Q3C7SCZ33X5T6CTLYDMQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.163747035 (223h40m53.090041713s) [noise=true , address=104.36.82.11:4115 , piece_id=LOK75B4Y2M3DMRJAHNYCVE46DELAAVN4SFP35AWPFESLUJT2V6KQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.167885784 (223h40m53.094180467s) [noise=true , address=104.36.82.6:4874 , piece_id=XVVQN3VX4I2KS4CIXEADBSFYBVYTVEWS5GMXAHE2LWDJGD2A2KAA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.168494969 (223h40m53.094789668s) [noise=true , address=104.36.82.10:4079 , piece_id=JELVGAGG2NF2OSMKXQAV6DIXDTUFIER5WX4KTCJFQQ5P35EAMOEA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.168525706 (223h40m53.094820396s) [seg_pos={0 0}, upload=179165]: Segment reached optimal threshold of 13 pieces.
Jul 2 16:52:26.168715378 (223h40m53.09501007s) [noise=true , address=77.90.31.19:4843 , piece_id=J74IERT5EP2VDSXY5PNWWBVTHQEXT4CWGLWP72Y3CQLOTSI6XZDA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: expected piece hash; context canceled; EOF
Jul 2 16:52:26.168766821 (223h40m53.095061509s) [noise=true , address=104.36.82.8:4970 , piece_id=X7KKQJYHVNI5LJDKGYNV5JHHCEGF4QSHFRKMSBLRE5W3W2ITAAHQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe): protocol: expected piece hash; context canceled; EOF
Jul 2 16:52:26.168816209 (223h40m53.095110907s) [noise=true , address=45.91.116.11:4118 , piece_id=CSQO6I5J4BW4NUAOFH3QDF6VEJGACWOSYNTNGXVELK2HLXIMX4YQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF
Jul 2 16:52:26.168851649 (223h40m53.095146335s) [seg_pos={0 0}, upload=179165]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 2 16:52:26.169058251 (223h40m53.095352944s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 16:52:26.169066272 (223h40m53.095360967s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 16:52:26.220481259 (223h40m53.146775936s) [upload=179165]: Done uploading segments...
Jul 2 16:52:26.220500093 (223h40m53.14679477s) [upload=179165]: Upload finished. err: <nil>
Duration: 451.534324ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............................................X.............] duration:
1.793301041s
FAIL (2025-07-02 17:26:17 UTC)
orbiter warning: uplink: metaclient: manager closed: closed: read tcp 159.223.143.39:49282->34.150.199.48:7777: read: connection reset by peer
Duration: 1.793334304s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............X..............................................] duration:
467.258066ms
FAIL (2025-07-02 17:25:40 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
led; ecclient: upload cut due to slow connection (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN): protocol: expected piece hash; context canceled; EOF
Jul 2 16:51:39.852164074 (223h40m6.778458767s) [noise=true , address=104.36.82.12:4150 , piece_id=QHXVON3LSBWYG5AWPVZ45KOTJJPUXIIAXGED7OHG65V5V3IJPULA, seg_pos={0 0}, upload=179153]: Done uploading piece. err: <nil>
Jul 2 16:51:39.852220651 (223h40m6.778515346s) [seg_pos={0 0}, upload=179153]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 2 16:51:39.852324711 (223h40m6.778619405s) [upload=179153]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 16:51:39.852329895 (223h40m6.778624588s) [upload=179153]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 16:51:39.945155283 (223h40m6.871449972s) [upload=179153]: Done uploading segments...
Jul 2 16:51:39.945173208 (223h40m6.871467885s) [upload=179153]: Upload finished. err: <nil>
Jul 2 16:52:25.999553770 (223h40m52.925848461s) [upload=179165]: Starting upload
Jul 2 16:52:25.999676452 (223h40m52.92597114s) [upload=179165]: Done starting upload
Jul 2 16:52:25.999808777 (223h40m52.926103458s) [upload=179165]: Uploading segments...
Jul 2 16:52:26.000049412 (223h40m52.926344119s) [upload=179165]: Got next segment. Inline: false
Jul 2 16:52:26.000208665 (223h40m52.926503358s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 2 16:52:26.000216887 (223h40m52.926511587s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 2 16:52:26.029546629 (223h40m52.95584132s) [seg_pos={0 0}, upload=179165]: Begin upload segment...
Jul 2 16:52:26.029596311 (223h40m52.955890985s) [seg_pos={0 0}, upload=179165]: Done begin upload segment.
Jul 2 16:52:26.029630299 (223h40m52.955924986s) [noise=true , address=45.91.116.12:4226 , piece_id=NFB2MT72D7MLPOY3A6MVO2QQBUFVYJPKN7D5XJQLQCCMYNQ6WJOA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.030073195 (223h40m52.956367894s) [noise=true , address=77.90.31.14:4661 , piece_id=B5E2MZ5JW7Z5AG4GFQ4QGMAFW7ZMKWKRFU276Z6556JFWKYVLXTQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.030398264 (223h40m52.956692955s) [noise=true , address=77.90.31.15:4698 , piece_id=VDVQDBYDQYRMVP7ZUPSEFC2R5UF2LVW45G23JURMYKS5IT6RBQFQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.030716003 (223h40m52.957010692s) [noise=true , address=45.91.116.11:4118 , piece_id=CSQO6I5J4BW4NUAOFH3QDF6VEJGACWOSYNTNGXVELK2HLXIMX4YQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.031068785 (223h40m52.957363472s) [noise=true , address=45.91.116.5:4298 , piece_id=JUYTY35XKYPNJYE7X3R7UMINTVMXDMNM6YZSZORJWATTVDBZFOMQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.031405588 (223h40m52.957700282s) [noise=true , address=104.36.82.8:4970 , piece_id=X7KKQJYHVNI5LJDKGYNV5JHHCEGF4QSHFRKMSBLRE5W3W2ITAAHQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.031730040 (223h40m52.958024715s) [noise=true , address=104.36.82.12:4149 , piece_id=V5265CLKUYZQN5B663WB4CFR3SXR2OA2BT53OAVCLFZJJ2YY6U4A, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.032049179 (223h40m52.958343866s) [noise=true , address=104.36.82.6:4874 , piece_id=XVVQN3VX4I2KS4CIXEADBSFYBVYTVEWS5GMXAHE2LWDJGD2A2KAA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.032432775 (223h40m52.958727467s) [noise=true , address=104.36.82.16:4294 , piece_id=36S2PXQADEAQPWSBMQC4XEAH6H5PDA4AO34JI37VD4KPJOU2WVTA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.032754164 (223h40m52.959048852s) [noise=true , address=77.90.31.19:4843 , piece_id=J74IERT5EP2VDSXY5PNWWBVTHQEXT4CWGLWP72Y3CQLOTSI6XZDA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.033080515 (223h40m52.959375213s) [noise=true , address=104.36.82.11:4115 , piece_id=LOK75B4Y2M3DMRJAHNYCVE46DELAAVN4SFP35AWPFESLUJT2V6KQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.033386908 (223h40m52.959681597s) [noise=true , address=104.36.82.14:4222 , piece_id=3HZK5YRWL2KXKY4E7XASRI2UIPCJA72JSJSTGBLPA4ZBP3MGBEOQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.033709080 (223h40m52.960003768s) [noise=true , address=77.90.31.21:4878 , piece_id=QJARHOYLDCFGYTTCD2EYCH43X3EARFK3Q3C7SCZ33X5T6CTLYDMQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.034030732 (223h40m52.96032542s) [noise=true , address=104.36.82.3:4767 , piece_id=U4L7ULC2R5OKDDL6BGDSEQUZNTVMQSGXJ56CGVKHNCCHTMIIDXDQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.034335911 (223h40m52.960630599s) [noise=true , address=45.91.116.8:4047 , piece_id=IJZDAO2K7KADVBNZGHLACI2ARTDDYOHIC742L2IUJT2WFAMMIGSQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.034644696 (223h40m52.960939386s) [noise=true , address=104.36.82.10:4079 , piece_id=JELVGAGG2NF2OSMKXQAV6DIXDTUFIER5WX4KTCJFQQ5P35EAMOEA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.148244089 (223h40m53.074538765s) [upload=179165]: Next returned nil segment
Jul 2 16:52:26.148268128 (223h40m53.074562811s) [upload=179165]: Waiting for error group managing segments...
Jul 2 16:52:26.155850465 (223h40m53.082145153s) [noise=true , address=45.91.116.5:4298 , piece_id=JUYTY35XKYPNJYE7X3R7UMINTVMXDMNM6YZSZORJWATTVDBZFOMQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.156432201 (223h40m53.082726892s) [noise=true , address=77.90.31.14:4661 , piece_id=B5E2MZ5JW7Z5AG4GFQ4QGMAFW7ZMKWKRFU276Z6556JFWKYVLXTQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.156943265 (223h40m53.083237956s) [noise=true , address=104.36.82.16:4294 , piece_id=36S2PXQADEAQPWSBMQC4XEAH6H5PDA4AO34JI37VD4KPJOU2WVTA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.158591570 (223h40m53.084886251s) [noise=true , address=104.36.82.3:4767 , piece_id=U4L7ULC2R5OKDDL6BGDSEQUZNTVMQSGXJ56CGVKHNCCHTMIIDXDQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.159458953 (223h40m53.085753634s) [noise=true , address=45.91.116.8:4047 , piece_id=IJZDAO2K7KADVBNZGHLACI2ARTDDYOHIC742L2IUJT2WFAMMIGSQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.161102375 (223h40m53.087397055s) [noise=true , address=45.91.116.12:4226 , piece_id=NFB2MT72D7MLPOY3A6MVO2QQBUFVYJPKN7D5XJQLQCCMYNQ6WJOA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.161556163 (223h40m53.087850838s) [noise=true , address=104.36.82.12:4149 , piece_id=V5265CLKUYZQN5B663WB4CFR3SXR2OA2BT53OAVCLFZJJ2YY6U4A, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.162013197 (223h40m53.088307874s) [noise=true , address=104.36.82.14:4222 , piece_id=3HZK5YRWL2KXKY4E7XASRI2UIPCJA72JSJSTGBLPA4ZBP3MGBEOQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.162459436 (223h40m53.088754112s) [noise=true , address=77.90.31.15:4698 , piece_id=VDVQDBYDQYRMVP7ZUPSEFC2R5UF2LVW45G23JURMYKS5IT6RBQFQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.162920549 (223h40m53.089215241s) [noise=true , address=77.90.31.21:4878 , piece_id=QJARHOYLDCFGYTTCD2EYCH43X3EARFK3Q3C7SCZ33X5T6CTLYDMQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.163747035 (223h40m53.090041713s) [noise=true , address=104.36.82.11:4115 , piece_id=LOK75B4Y2M3DMRJAHNYCVE46DELAAVN4SFP35AWPFESLUJT2V6KQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.167885784 (223h40m53.094180467s) [noise=true , address=104.36.82.6:4874 , piece_id=XVVQN3VX4I2KS4CIXEADBSFYBVYTVEWS5GMXAHE2LWDJGD2A2KAA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.168494969 (223h40m53.094789668s) [noise=true , address=104.36.82.10:4079 , piece_id=JELVGAGG2NF2OSMKXQAV6DIXDTUFIER5WX4KTCJFQQ5P35EAMOEA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.168525706 (223h40m53.094820396s) [seg_pos={0 0}, upload=179165]: Segment reached optimal threshold of 13 pieces.
Jul 2 16:52:26.168715378 (223h40m53.09501007s) [noise=true , address=77.90.31.19:4843 , piece_id=J74IERT5EP2VDSXY5PNWWBVTHQEXT4CWGLWP72Y3CQLOTSI6XZDA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: expected piece hash; context canceled; EOF
Jul 2 16:52:26.168766821 (223h40m53.095061509s) [noise=true , address=104.36.82.8:4970 , piece_id=X7KKQJYHVNI5LJDKGYNV5JHHCEGF4QSHFRKMSBLRE5W3W2ITAAHQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe): protocol: expected piece hash; context canceled; EOF
Jul 2 16:52:26.168816209 (223h40m53.095110907s) [noise=true , address=45.91.116.11:4118 , piece_id=CSQO6I5J4BW4NUAOFH3QDF6VEJGACWOSYNTNGXVELK2HLXIMX4YQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF
Jul 2 16:52:26.168851649 (223h40m53.095146335s) [seg_pos={0 0}, upload=179165]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 2 16:52:26.169058251 (223h40m53.095352944s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 16:52:26.169066272 (223h40m53.095360967s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 16:52:26.220481259 (223h40m53.146775936s) [upload=179165]: Done uploading segments...
Jul 2 16:52:26.220500093 (223h40m53.14679477s) [upload=179165]: Upload finished. err: <nil>
Duration: 467.298845ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............] duration:
1.720945387s
FAIL (2025-07-02 17:25:19 UTC)
orbiter warning: uplink: metaclient: manager closed: closed: read tcp 159.223.143.39:49282->34.150.199.48:7777: read: connection reset by peer
Duration: 1.720978055s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........] duration:
1.733382881s
FAIL (2025-07-02 17:24:46 UTC)
orbiter warning: uplink: metaclient: manager closed: closed: read tcp 159.223.143.39:49282->34.150.199.48:7777: read: connection reset by peer
Duration: 1.733430832s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............X.............................................] duration:
477.833094ms
FAIL (2025-07-02 17:24:43 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
led; ecclient: upload cut due to slow connection (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN): protocol: expected piece hash; context canceled; EOF
Jul 2 16:51:39.852164074 (223h40m6.778458767s) [noise=true , address=104.36.82.12:4150 , piece_id=QHXVON3LSBWYG5AWPVZ45KOTJJPUXIIAXGED7OHG65V5V3IJPULA, seg_pos={0 0}, upload=179153]: Done uploading piece. err: <nil>
Jul 2 16:51:39.852220651 (223h40m6.778515346s) [seg_pos={0 0}, upload=179153]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 2 16:51:39.852324711 (223h40m6.778619405s) [upload=179153]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 16:51:39.852329895 (223h40m6.778624588s) [upload=179153]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 16:51:39.945155283 (223h40m6.871449972s) [upload=179153]: Done uploading segments...
Jul 2 16:51:39.945173208 (223h40m6.871467885s) [upload=179153]: Upload finished. err: <nil>
Jul 2 16:52:25.999553770 (223h40m52.925848461s) [upload=179165]: Starting upload
Jul 2 16:52:25.999676452 (223h40m52.92597114s) [upload=179165]: Done starting upload
Jul 2 16:52:25.999808777 (223h40m52.926103458s) [upload=179165]: Uploading segments...
Jul 2 16:52:26.000049412 (223h40m52.926344119s) [upload=179165]: Got next segment. Inline: false
Jul 2 16:52:26.000208665 (223h40m52.926503358s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 2 16:52:26.000216887 (223h40m52.926511587s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 2 16:52:26.029546629 (223h40m52.95584132s) [seg_pos={0 0}, upload=179165]: Begin upload segment...
Jul 2 16:52:26.029596311 (223h40m52.955890985s) [seg_pos={0 0}, upload=179165]: Done begin upload segment.
Jul 2 16:52:26.029630299 (223h40m52.955924986s) [noise=true , address=45.91.116.12:4226 , piece_id=NFB2MT72D7MLPOY3A6MVO2QQBUFVYJPKN7D5XJQLQCCMYNQ6WJOA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.030073195 (223h40m52.956367894s) [noise=true , address=77.90.31.14:4661 , piece_id=B5E2MZ5JW7Z5AG4GFQ4QGMAFW7ZMKWKRFU276Z6556JFWKYVLXTQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.030398264 (223h40m52.956692955s) [noise=true , address=77.90.31.15:4698 , piece_id=VDVQDBYDQYRMVP7ZUPSEFC2R5UF2LVW45G23JURMYKS5IT6RBQFQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.030716003 (223h40m52.957010692s) [noise=true , address=45.91.116.11:4118 , piece_id=CSQO6I5J4BW4NUAOFH3QDF6VEJGACWOSYNTNGXVELK2HLXIMX4YQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.031068785 (223h40m52.957363472s) [noise=true , address=45.91.116.5:4298 , piece_id=JUYTY35XKYPNJYE7X3R7UMINTVMXDMNM6YZSZORJWATTVDBZFOMQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.031405588 (223h40m52.957700282s) [noise=true , address=104.36.82.8:4970 , piece_id=X7KKQJYHVNI5LJDKGYNV5JHHCEGF4QSHFRKMSBLRE5W3W2ITAAHQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.031730040 (223h40m52.958024715s) [noise=true , address=104.36.82.12:4149 , piece_id=V5265CLKUYZQN5B663WB4CFR3SXR2OA2BT53OAVCLFZJJ2YY6U4A, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.032049179 (223h40m52.958343866s) [noise=true , address=104.36.82.6:4874 , piece_id=XVVQN3VX4I2KS4CIXEADBSFYBVYTVEWS5GMXAHE2LWDJGD2A2KAA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.032432775 (223h40m52.958727467s) [noise=true , address=104.36.82.16:4294 , piece_id=36S2PXQADEAQPWSBMQC4XEAH6H5PDA4AO34JI37VD4KPJOU2WVTA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.032754164 (223h40m52.959048852s) [noise=true , address=77.90.31.19:4843 , piece_id=J74IERT5EP2VDSXY5PNWWBVTHQEXT4CWGLWP72Y3CQLOTSI6XZDA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.033080515 (223h40m52.959375213s) [noise=true , address=104.36.82.11:4115 , piece_id=LOK75B4Y2M3DMRJAHNYCVE46DELAAVN4SFP35AWPFESLUJT2V6KQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.033386908 (223h40m52.959681597s) [noise=true , address=104.36.82.14:4222 , piece_id=3HZK5YRWL2KXKY4E7XASRI2UIPCJA72JSJSTGBLPA4ZBP3MGBEOQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.033709080 (223h40m52.960003768s) [noise=true , address=77.90.31.21:4878 , piece_id=QJARHOYLDCFGYTTCD2EYCH43X3EARFK3Q3C7SCZ33X5T6CTLYDMQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.034030732 (223h40m52.96032542s) [noise=true , address=104.36.82.3:4767 , piece_id=U4L7ULC2R5OKDDL6BGDSEQUZNTVMQSGXJ56CGVKHNCCHTMIIDXDQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.034335911 (223h40m52.960630599s) [noise=true , address=45.91.116.8:4047 , piece_id=IJZDAO2K7KADVBNZGHLACI2ARTDDYOHIC742L2IUJT2WFAMMIGSQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.034644696 (223h40m52.960939386s) [noise=true , address=104.36.82.10:4079 , piece_id=JELVGAGG2NF2OSMKXQAV6DIXDTUFIER5WX4KTCJFQQ5P35EAMOEA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.148244089 (223h40m53.074538765s) [upload=179165]: Next returned nil segment
Jul 2 16:52:26.148268128 (223h40m53.074562811s) [upload=179165]: Waiting for error group managing segments...
Jul 2 16:52:26.155850465 (223h40m53.082145153s) [noise=true , address=45.91.116.5:4298 , piece_id=JUYTY35XKYPNJYE7X3R7UMINTVMXDMNM6YZSZORJWATTVDBZFOMQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.156432201 (223h40m53.082726892s) [noise=true , address=77.90.31.14:4661 , piece_id=B5E2MZ5JW7Z5AG4GFQ4QGMAFW7ZMKWKRFU276Z6556JFWKYVLXTQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.156943265 (223h40m53.083237956s) [noise=true , address=104.36.82.16:4294 , piece_id=36S2PXQADEAQPWSBMQC4XEAH6H5PDA4AO34JI37VD4KPJOU2WVTA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.158591570 (223h40m53.084886251s) [noise=true , address=104.36.82.3:4767 , piece_id=U4L7ULC2R5OKDDL6BGDSEQUZNTVMQSGXJ56CGVKHNCCHTMIIDXDQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.159458953 (223h40m53.085753634s) [noise=true , address=45.91.116.8:4047 , piece_id=IJZDAO2K7KADVBNZGHLACI2ARTDDYOHIC742L2IUJT2WFAMMIGSQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.161102375 (223h40m53.087397055s) [noise=true , address=45.91.116.12:4226 , piece_id=NFB2MT72D7MLPOY3A6MVO2QQBUFVYJPKN7D5XJQLQCCMYNQ6WJOA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.161556163 (223h40m53.087850838s) [noise=true , address=104.36.82.12:4149 , piece_id=V5265CLKUYZQN5B663WB4CFR3SXR2OA2BT53OAVCLFZJJ2YY6U4A, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.162013197 (223h40m53.088307874s) [noise=true , address=104.36.82.14:4222 , piece_id=3HZK5YRWL2KXKY4E7XASRI2UIPCJA72JSJSTGBLPA4ZBP3MGBEOQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.162459436 (223h40m53.088754112s) [noise=true , address=77.90.31.15:4698 , piece_id=VDVQDBYDQYRMVP7ZUPSEFC2R5UF2LVW45G23JURMYKS5IT6RBQFQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.162920549 (223h40m53.089215241s) [noise=true , address=77.90.31.21:4878 , piece_id=QJARHOYLDCFGYTTCD2EYCH43X3EARFK3Q3C7SCZ33X5T6CTLYDMQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.163747035 (223h40m53.090041713s) [noise=true , address=104.36.82.11:4115 , piece_id=LOK75B4Y2M3DMRJAHNYCVE46DELAAVN4SFP35AWPFESLUJT2V6KQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.167885784 (223h40m53.094180467s) [noise=true , address=104.36.82.6:4874 , piece_id=XVVQN3VX4I2KS4CIXEADBSFYBVYTVEWS5GMXAHE2LWDJGD2A2KAA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.168494969 (223h40m53.094789668s) [noise=true , address=104.36.82.10:4079 , piece_id=JELVGAGG2NF2OSMKXQAV6DIXDTUFIER5WX4KTCJFQQ5P35EAMOEA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.168525706 (223h40m53.094820396s) [seg_pos={0 0}, upload=179165]: Segment reached optimal threshold of 13 pieces.
Jul 2 16:52:26.168715378 (223h40m53.09501007s) [noise=true , address=77.90.31.19:4843 , piece_id=J74IERT5EP2VDSXY5PNWWBVTHQEXT4CWGLWP72Y3CQLOTSI6XZDA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: expected piece hash; context canceled; EOF
Jul 2 16:52:26.168766821 (223h40m53.095061509s) [noise=true , address=104.36.82.8:4970 , piece_id=X7KKQJYHVNI5LJDKGYNV5JHHCEGF4QSHFRKMSBLRE5W3W2ITAAHQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe): protocol: expected piece hash; context canceled; EOF
Jul 2 16:52:26.168816209 (223h40m53.095110907s) [noise=true , address=45.91.116.11:4118 , piece_id=CSQO6I5J4BW4NUAOFH3QDF6VEJGACWOSYNTNGXVELK2HLXIMX4YQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF
Jul 2 16:52:26.168851649 (223h40m53.095146335s) [seg_pos={0 0}, upload=179165]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 2 16:52:26.169058251 (223h40m53.095352944s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 16:52:26.169066272 (223h40m53.095360967s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 16:52:26.220481259 (223h40m53.146775936s) [upload=179165]: Done uploading segments...
Jul 2 16:52:26.220500093 (223h40m53.14679477s) [upload=179165]: Upload finished. err: <nil>
Duration: 477.864145ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........] duration:
1.904479348s
FAIL (2025-07-02 17:23:48 UTC)
orbiter warning: uplink: metaclient: manager closed: closed: read tcp 159.223.143.39:49282->34.150.199.48:7777: read: connection reset by peer
Duration: 1.904509262s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............X............................................] duration:
605.376797ms
FAIL (2025-07-02 17:23:37 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
led; ecclient: upload cut due to slow connection (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN): protocol: expected piece hash; context canceled; EOF
Jul 2 16:51:39.852164074 (223h40m6.778458767s) [noise=true , address=104.36.82.12:4150 , piece_id=QHXVON3LSBWYG5AWPVZ45KOTJJPUXIIAXGED7OHG65V5V3IJPULA, seg_pos={0 0}, upload=179153]: Done uploading piece. err: <nil>
Jul 2 16:51:39.852220651 (223h40m6.778515346s) [seg_pos={0 0}, upload=179153]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 2 16:51:39.852324711 (223h40m6.778619405s) [upload=179153]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 16:51:39.852329895 (223h40m6.778624588s) [upload=179153]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 16:51:39.945155283 (223h40m6.871449972s) [upload=179153]: Done uploading segments...
Jul 2 16:51:39.945173208 (223h40m6.871467885s) [upload=179153]: Upload finished. err: <nil>
Jul 2 16:52:25.999553770 (223h40m52.925848461s) [upload=179165]: Starting upload
Jul 2 16:52:25.999676452 (223h40m52.92597114s) [upload=179165]: Done starting upload
Jul 2 16:52:25.999808777 (223h40m52.926103458s) [upload=179165]: Uploading segments...
Jul 2 16:52:26.000049412 (223h40m52.926344119s) [upload=179165]: Got next segment. Inline: false
Jul 2 16:52:26.000208665 (223h40m52.926503358s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 2 16:52:26.000216887 (223h40m52.926511587s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 2 16:52:26.029546629 (223h40m52.95584132s) [seg_pos={0 0}, upload=179165]: Begin upload segment...
Jul 2 16:52:26.029596311 (223h40m52.955890985s) [seg_pos={0 0}, upload=179165]: Done begin upload segment.
Jul 2 16:52:26.029630299 (223h40m52.955924986s) [noise=true , address=45.91.116.12:4226 , piece_id=NFB2MT72D7MLPOY3A6MVO2QQBUFVYJPKN7D5XJQLQCCMYNQ6WJOA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.030073195 (223h40m52.956367894s) [noise=true , address=77.90.31.14:4661 , piece_id=B5E2MZ5JW7Z5AG4GFQ4QGMAFW7ZMKWKRFU276Z6556JFWKYVLXTQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.030398264 (223h40m52.956692955s) [noise=true , address=77.90.31.15:4698 , piece_id=VDVQDBYDQYRMVP7ZUPSEFC2R5UF2LVW45G23JURMYKS5IT6RBQFQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.030716003 (223h40m52.957010692s) [noise=true , address=45.91.116.11:4118 , piece_id=CSQO6I5J4BW4NUAOFH3QDF6VEJGACWOSYNTNGXVELK2HLXIMX4YQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.031068785 (223h40m52.957363472s) [noise=true , address=45.91.116.5:4298 , piece_id=JUYTY35XKYPNJYE7X3R7UMINTVMXDMNM6YZSZORJWATTVDBZFOMQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.031405588 (223h40m52.957700282s) [noise=true , address=104.36.82.8:4970 , piece_id=X7KKQJYHVNI5LJDKGYNV5JHHCEGF4QSHFRKMSBLRE5W3W2ITAAHQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.031730040 (223h40m52.958024715s) [noise=true , address=104.36.82.12:4149 , piece_id=V5265CLKUYZQN5B663WB4CFR3SXR2OA2BT53OAVCLFZJJ2YY6U4A, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.032049179 (223h40m52.958343866s) [noise=true , address=104.36.82.6:4874 , piece_id=XVVQN3VX4I2KS4CIXEADBSFYBVYTVEWS5GMXAHE2LWDJGD2A2KAA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.032432775 (223h40m52.958727467s) [noise=true , address=104.36.82.16:4294 , piece_id=36S2PXQADEAQPWSBMQC4XEAH6H5PDA4AO34JI37VD4KPJOU2WVTA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.032754164 (223h40m52.959048852s) [noise=true , address=77.90.31.19:4843 , piece_id=J74IERT5EP2VDSXY5PNWWBVTHQEXT4CWGLWP72Y3CQLOTSI6XZDA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.033080515 (223h40m52.959375213s) [noise=true , address=104.36.82.11:4115 , piece_id=LOK75B4Y2M3DMRJAHNYCVE46DELAAVN4SFP35AWPFESLUJT2V6KQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.033386908 (223h40m52.959681597s) [noise=true , address=104.36.82.14:4222 , piece_id=3HZK5YRWL2KXKY4E7XASRI2UIPCJA72JSJSTGBLPA4ZBP3MGBEOQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.033709080 (223h40m52.960003768s) [noise=true , address=77.90.31.21:4878 , piece_id=QJARHOYLDCFGYTTCD2EYCH43X3EARFK3Q3C7SCZ33X5T6CTLYDMQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.034030732 (223h40m52.96032542s) [noise=true , address=104.36.82.3:4767 , piece_id=U4L7ULC2R5OKDDL6BGDSEQUZNTVMQSGXJ56CGVKHNCCHTMIIDXDQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.034335911 (223h40m52.960630599s) [noise=true , address=45.91.116.8:4047 , piece_id=IJZDAO2K7KADVBNZGHLACI2ARTDDYOHIC742L2IUJT2WFAMMIGSQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.034644696 (223h40m52.960939386s) [noise=true , address=104.36.82.10:4079 , piece_id=JELVGAGG2NF2OSMKXQAV6DIXDTUFIER5WX4KTCJFQQ5P35EAMOEA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.148244089 (223h40m53.074538765s) [upload=179165]: Next returned nil segment
Jul 2 16:52:26.148268128 (223h40m53.074562811s) [upload=179165]: Waiting for error group managing segments...
Jul 2 16:52:26.155850465 (223h40m53.082145153s) [noise=true , address=45.91.116.5:4298 , piece_id=JUYTY35XKYPNJYE7X3R7UMINTVMXDMNM6YZSZORJWATTVDBZFOMQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.156432201 (223h40m53.082726892s) [noise=true , address=77.90.31.14:4661 , piece_id=B5E2MZ5JW7Z5AG4GFQ4QGMAFW7ZMKWKRFU276Z6556JFWKYVLXTQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.156943265 (223h40m53.083237956s) [noise=true , address=104.36.82.16:4294 , piece_id=36S2PXQADEAQPWSBMQC4XEAH6H5PDA4AO34JI37VD4KPJOU2WVTA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.158591570 (223h40m53.084886251s) [noise=true , address=104.36.82.3:4767 , piece_id=U4L7ULC2R5OKDDL6BGDSEQUZNTVMQSGXJ56CGVKHNCCHTMIIDXDQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.159458953 (223h40m53.085753634s) [noise=true , address=45.91.116.8:4047 , piece_id=IJZDAO2K7KADVBNZGHLACI2ARTDDYOHIC742L2IUJT2WFAMMIGSQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.161102375 (223h40m53.087397055s) [noise=true , address=45.91.116.12:4226 , piece_id=NFB2MT72D7MLPOY3A6MVO2QQBUFVYJPKN7D5XJQLQCCMYNQ6WJOA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.161556163 (223h40m53.087850838s) [noise=true , address=104.36.82.12:4149 , piece_id=V5265CLKUYZQN5B663WB4CFR3SXR2OA2BT53OAVCLFZJJ2YY6U4A, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.162013197 (223h40m53.088307874s) [noise=true , address=104.36.82.14:4222 , piece_id=3HZK5YRWL2KXKY4E7XASRI2UIPCJA72JSJSTGBLPA4ZBP3MGBEOQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.162459436 (223h40m53.088754112s) [noise=true , address=77.90.31.15:4698 , piece_id=VDVQDBYDQYRMVP7ZUPSEFC2R5UF2LVW45G23JURMYKS5IT6RBQFQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.162920549 (223h40m53.089215241s) [noise=true , address=77.90.31.21:4878 , piece_id=QJARHOYLDCFGYTTCD2EYCH43X3EARFK3Q3C7SCZ33X5T6CTLYDMQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.163747035 (223h40m53.090041713s) [noise=true , address=104.36.82.11:4115 , piece_id=LOK75B4Y2M3DMRJAHNYCVE46DELAAVN4SFP35AWPFESLUJT2V6KQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.167885784 (223h40m53.094180467s) [noise=true , address=104.36.82.6:4874 , piece_id=XVVQN3VX4I2KS4CIXEADBSFYBVYTVEWS5GMXAHE2LWDJGD2A2KAA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.168494969 (223h40m53.094789668s) [noise=true , address=104.36.82.10:4079 , piece_id=JELVGAGG2NF2OSMKXQAV6DIXDTUFIER5WX4KTCJFQQ5P35EAMOEA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.168525706 (223h40m53.094820396s) [seg_pos={0 0}, upload=179165]: Segment reached optimal threshold of 13 pieces.
Jul 2 16:52:26.168715378 (223h40m53.09501007s) [noise=true , address=77.90.31.19:4843 , piece_id=J74IERT5EP2VDSXY5PNWWBVTHQEXT4CWGLWP72Y3CQLOTSI6XZDA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: expected piece hash; context canceled; EOF
Jul 2 16:52:26.168766821 (223h40m53.095061509s) [noise=true , address=104.36.82.8:4970 , piece_id=X7KKQJYHVNI5LJDKGYNV5JHHCEGF4QSHFRKMSBLRE5W3W2ITAAHQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe): protocol: expected piece hash; context canceled; EOF
Jul 2 16:52:26.168816209 (223h40m53.095110907s) [noise=true , address=45.91.116.11:4118 , piece_id=CSQO6I5J4BW4NUAOFH3QDF6VEJGACWOSYNTNGXVELK2HLXIMX4YQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF
Jul 2 16:52:26.168851649 (223h40m53.095146335s) [seg_pos={0 0}, upload=179165]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 2 16:52:26.169058251 (223h40m53.095352944s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 16:52:26.169066272 (223h40m53.095360967s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 16:52:26.220481259 (223h40m53.146775936s) [upload=179165]: Done uploading segments...
Jul 2 16:52:26.220500093 (223h40m53.14679477s) [upload=179165]: Upload finished. err: <nil>
Duration: 605.407469ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
1.727188727s
FAIL (2025-07-02 17:23:21 UTC)
orbiter warning: uplink: metaclient: manager closed: closed: read tcp 159.223.143.39:49282->34.150.199.48:7777: read: connection reset by peer
Duration: 1.72721446s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [................X...........................................] duration:
446.557689ms
FAIL (2025-07-02 17:22:39 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
led; ecclient: upload cut due to slow connection (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN): protocol: expected piece hash; context canceled; EOF
Jul 2 16:51:39.852164074 (223h40m6.778458767s) [noise=true , address=104.36.82.12:4150 , piece_id=QHXVON3LSBWYG5AWPVZ45KOTJJPUXIIAXGED7OHG65V5V3IJPULA, seg_pos={0 0}, upload=179153]: Done uploading piece. err: <nil>
Jul 2 16:51:39.852220651 (223h40m6.778515346s) [seg_pos={0 0}, upload=179153]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 2 16:51:39.852324711 (223h40m6.778619405s) [upload=179153]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 16:51:39.852329895 (223h40m6.778624588s) [upload=179153]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 16:51:39.945155283 (223h40m6.871449972s) [upload=179153]: Done uploading segments...
Jul 2 16:51:39.945173208 (223h40m6.871467885s) [upload=179153]: Upload finished. err: <nil>
Jul 2 16:52:25.999553770 (223h40m52.925848461s) [upload=179165]: Starting upload
Jul 2 16:52:25.999676452 (223h40m52.92597114s) [upload=179165]: Done starting upload
Jul 2 16:52:25.999808777 (223h40m52.926103458s) [upload=179165]: Uploading segments...
Jul 2 16:52:26.000049412 (223h40m52.926344119s) [upload=179165]: Got next segment. Inline: false
Jul 2 16:52:26.000208665 (223h40m52.926503358s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 2 16:52:26.000216887 (223h40m52.926511587s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 2 16:52:26.029546629 (223h40m52.95584132s) [seg_pos={0 0}, upload=179165]: Begin upload segment...
Jul 2 16:52:26.029596311 (223h40m52.955890985s) [seg_pos={0 0}, upload=179165]: Done begin upload segment.
Jul 2 16:52:26.029630299 (223h40m52.955924986s) [noise=true , address=45.91.116.12:4226 , piece_id=NFB2MT72D7MLPOY3A6MVO2QQBUFVYJPKN7D5XJQLQCCMYNQ6WJOA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.030073195 (223h40m52.956367894s) [noise=true , address=77.90.31.14:4661 , piece_id=B5E2MZ5JW7Z5AG4GFQ4QGMAFW7ZMKWKRFU276Z6556JFWKYVLXTQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.030398264 (223h40m52.956692955s) [noise=true , address=77.90.31.15:4698 , piece_id=VDVQDBYDQYRMVP7ZUPSEFC2R5UF2LVW45G23JURMYKS5IT6RBQFQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.030716003 (223h40m52.957010692s) [noise=true , address=45.91.116.11:4118 , piece_id=CSQO6I5J4BW4NUAOFH3QDF6VEJGACWOSYNTNGXVELK2HLXIMX4YQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.031068785 (223h40m52.957363472s) [noise=true , address=45.91.116.5:4298 , piece_id=JUYTY35XKYPNJYE7X3R7UMINTVMXDMNM6YZSZORJWATTVDBZFOMQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.031405588 (223h40m52.957700282s) [noise=true , address=104.36.82.8:4970 , piece_id=X7KKQJYHVNI5LJDKGYNV5JHHCEGF4QSHFRKMSBLRE5W3W2ITAAHQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.031730040 (223h40m52.958024715s) [noise=true , address=104.36.82.12:4149 , piece_id=V5265CLKUYZQN5B663WB4CFR3SXR2OA2BT53OAVCLFZJJ2YY6U4A, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.032049179 (223h40m52.958343866s) [noise=true , address=104.36.82.6:4874 , piece_id=XVVQN3VX4I2KS4CIXEADBSFYBVYTVEWS5GMXAHE2LWDJGD2A2KAA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.032432775 (223h40m52.958727467s) [noise=true , address=104.36.82.16:4294 , piece_id=36S2PXQADEAQPWSBMQC4XEAH6H5PDA4AO34JI37VD4KPJOU2WVTA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.032754164 (223h40m52.959048852s) [noise=true , address=77.90.31.19:4843 , piece_id=J74IERT5EP2VDSXY5PNWWBVTHQEXT4CWGLWP72Y3CQLOTSI6XZDA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.033080515 (223h40m52.959375213s) [noise=true , address=104.36.82.11:4115 , piece_id=LOK75B4Y2M3DMRJAHNYCVE46DELAAVN4SFP35AWPFESLUJT2V6KQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.033386908 (223h40m52.959681597s) [noise=true , address=104.36.82.14:4222 , piece_id=3HZK5YRWL2KXKY4E7XASRI2UIPCJA72JSJSTGBLPA4ZBP3MGBEOQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.033709080 (223h40m52.960003768s) [noise=true , address=77.90.31.21:4878 , piece_id=QJARHOYLDCFGYTTCD2EYCH43X3EARFK3Q3C7SCZ33X5T6CTLYDMQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.034030732 (223h40m52.96032542s) [noise=true , address=104.36.82.3:4767 , piece_id=U4L7ULC2R5OKDDL6BGDSEQUZNTVMQSGXJ56CGVKHNCCHTMIIDXDQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.034335911 (223h40m52.960630599s) [noise=true , address=45.91.116.8:4047 , piece_id=IJZDAO2K7KADVBNZGHLACI2ARTDDYOHIC742L2IUJT2WFAMMIGSQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.034644696 (223h40m52.960939386s) [noise=true , address=104.36.82.10:4079 , piece_id=JELVGAGG2NF2OSMKXQAV6DIXDTUFIER5WX4KTCJFQQ5P35EAMOEA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.148244089 (223h40m53.074538765s) [upload=179165]: Next returned nil segment
Jul 2 16:52:26.148268128 (223h40m53.074562811s) [upload=179165]: Waiting for error group managing segments...
Jul 2 16:52:26.155850465 (223h40m53.082145153s) [noise=true , address=45.91.116.5:4298 , piece_id=JUYTY35XKYPNJYE7X3R7UMINTVMXDMNM6YZSZORJWATTVDBZFOMQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.156432201 (223h40m53.082726892s) [noise=true , address=77.90.31.14:4661 , piece_id=B5E2MZ5JW7Z5AG4GFQ4QGMAFW7ZMKWKRFU276Z6556JFWKYVLXTQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.156943265 (223h40m53.083237956s) [noise=true , address=104.36.82.16:4294 , piece_id=36S2PXQADEAQPWSBMQC4XEAH6H5PDA4AO34JI37VD4KPJOU2WVTA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.158591570 (223h40m53.084886251s) [noise=true , address=104.36.82.3:4767 , piece_id=U4L7ULC2R5OKDDL6BGDSEQUZNTVMQSGXJ56CGVKHNCCHTMIIDXDQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.159458953 (223h40m53.085753634s) [noise=true , address=45.91.116.8:4047 , piece_id=IJZDAO2K7KADVBNZGHLACI2ARTDDYOHIC742L2IUJT2WFAMMIGSQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.161102375 (223h40m53.087397055s) [noise=true , address=45.91.116.12:4226 , piece_id=NFB2MT72D7MLPOY3A6MVO2QQBUFVYJPKN7D5XJQLQCCMYNQ6WJOA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.161556163 (223h40m53.087850838s) [noise=true , address=104.36.82.12:4149 , piece_id=V5265CLKUYZQN5B663WB4CFR3SXR2OA2BT53OAVCLFZJJ2YY6U4A, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.162013197 (223h40m53.088307874s) [noise=true , address=104.36.82.14:4222 , piece_id=3HZK5YRWL2KXKY4E7XASRI2UIPCJA72JSJSTGBLPA4ZBP3MGBEOQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.162459436 (223h40m53.088754112s) [noise=true , address=77.90.31.15:4698 , piece_id=VDVQDBYDQYRMVP7ZUPSEFC2R5UF2LVW45G23JURMYKS5IT6RBQFQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.162920549 (223h40m53.089215241s) [noise=true , address=77.90.31.21:4878 , piece_id=QJARHOYLDCFGYTTCD2EYCH43X3EARFK3Q3C7SCZ33X5T6CTLYDMQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.163747035 (223h40m53.090041713s) [noise=true , address=104.36.82.11:4115 , piece_id=LOK75B4Y2M3DMRJAHNYCVE46DELAAVN4SFP35AWPFESLUJT2V6KQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.167885784 (223h40m53.094180467s) [noise=true , address=104.36.82.6:4874 , piece_id=XVVQN3VX4I2KS4CIXEADBSFYBVYTVEWS5GMXAHE2LWDJGD2A2KAA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.168494969 (223h40m53.094789668s) [noise=true , address=104.36.82.10:4079 , piece_id=JELVGAGG2NF2OSMKXQAV6DIXDTUFIER5WX4KTCJFQQ5P35EAMOEA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.168525706 (223h40m53.094820396s) [seg_pos={0 0}, upload=179165]: Segment reached optimal threshold of 13 pieces.
Jul 2 16:52:26.168715378 (223h40m53.09501007s) [noise=true , address=77.90.31.19:4843 , piece_id=J74IERT5EP2VDSXY5PNWWBVTHQEXT4CWGLWP72Y3CQLOTSI6XZDA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: expected piece hash; context canceled; EOF
Jul 2 16:52:26.168766821 (223h40m53.095061509s) [noise=true , address=104.36.82.8:4970 , piece_id=X7KKQJYHVNI5LJDKGYNV5JHHCEGF4QSHFRKMSBLRE5W3W2ITAAHQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe): protocol: expected piece hash; context canceled; EOF
Jul 2 16:52:26.168816209 (223h40m53.095110907s) [noise=true , address=45.91.116.11:4118 , piece_id=CSQO6I5J4BW4NUAOFH3QDF6VEJGACWOSYNTNGXVELK2HLXIMX4YQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF
Jul 2 16:52:26.168851649 (223h40m53.095146335s) [seg_pos={0 0}, upload=179165]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 2 16:52:26.169058251 (223h40m53.095352944s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 16:52:26.169066272 (223h40m53.095360967s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 16:52:26.220481259 (223h40m53.146775936s) [upload=179165]: Done uploading segments...
Jul 2 16:52:26.220500093 (223h40m53.14679477s) [upload=179165]: Upload finished. err: <nil>
Duration: 446.58876ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........] duration:
1.733883673s
FAIL (2025-07-02 17:22:22 UTC)
orbiter warning: uplink: metaclient: manager closed: closed: read tcp 159.223.143.39:49282->34.150.199.48:7777: read: connection reset by peer
Duration: 1.733919893s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................X..........................................] duration:
463.239136ms
FAIL (2025-07-02 17:21:59 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
led; ecclient: upload cut due to slow connection (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN): protocol: expected piece hash; context canceled; EOF
Jul 2 16:51:39.852164074 (223h40m6.778458767s) [noise=true , address=104.36.82.12:4150 , piece_id=QHXVON3LSBWYG5AWPVZ45KOTJJPUXIIAXGED7OHG65V5V3IJPULA, seg_pos={0 0}, upload=179153]: Done uploading piece. err: <nil>
Jul 2 16:51:39.852220651 (223h40m6.778515346s) [seg_pos={0 0}, upload=179153]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 2 16:51:39.852324711 (223h40m6.778619405s) [upload=179153]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 16:51:39.852329895 (223h40m6.778624588s) [upload=179153]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 16:51:39.945155283 (223h40m6.871449972s) [upload=179153]: Done uploading segments...
Jul 2 16:51:39.945173208 (223h40m6.871467885s) [upload=179153]: Upload finished. err: <nil>
Jul 2 16:52:25.999553770 (223h40m52.925848461s) [upload=179165]: Starting upload
Jul 2 16:52:25.999676452 (223h40m52.92597114s) [upload=179165]: Done starting upload
Jul 2 16:52:25.999808777 (223h40m52.926103458s) [upload=179165]: Uploading segments...
Jul 2 16:52:26.000049412 (223h40m52.926344119s) [upload=179165]: Got next segment. Inline: false
Jul 2 16:52:26.000208665 (223h40m52.926503358s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 2 16:52:26.000216887 (223h40m52.926511587s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 2 16:52:26.029546629 (223h40m52.95584132s) [seg_pos={0 0}, upload=179165]: Begin upload segment...
Jul 2 16:52:26.029596311 (223h40m52.955890985s) [seg_pos={0 0}, upload=179165]: Done begin upload segment.
Jul 2 16:52:26.029630299 (223h40m52.955924986s) [noise=true , address=45.91.116.12:4226 , piece_id=NFB2MT72D7MLPOY3A6MVO2QQBUFVYJPKN7D5XJQLQCCMYNQ6WJOA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.030073195 (223h40m52.956367894s) [noise=true , address=77.90.31.14:4661 , piece_id=B5E2MZ5JW7Z5AG4GFQ4QGMAFW7ZMKWKRFU276Z6556JFWKYVLXTQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.030398264 (223h40m52.956692955s) [noise=true , address=77.90.31.15:4698 , piece_id=VDVQDBYDQYRMVP7ZUPSEFC2R5UF2LVW45G23JURMYKS5IT6RBQFQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.030716003 (223h40m52.957010692s) [noise=true , address=45.91.116.11:4118 , piece_id=CSQO6I5J4BW4NUAOFH3QDF6VEJGACWOSYNTNGXVELK2HLXIMX4YQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.031068785 (223h40m52.957363472s) [noise=true , address=45.91.116.5:4298 , piece_id=JUYTY35XKYPNJYE7X3R7UMINTVMXDMNM6YZSZORJWATTVDBZFOMQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.031405588 (223h40m52.957700282s) [noise=true , address=104.36.82.8:4970 , piece_id=X7KKQJYHVNI5LJDKGYNV5JHHCEGF4QSHFRKMSBLRE5W3W2ITAAHQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.031730040 (223h40m52.958024715s) [noise=true , address=104.36.82.12:4149 , piece_id=V5265CLKUYZQN5B663WB4CFR3SXR2OA2BT53OAVCLFZJJ2YY6U4A, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.032049179 (223h40m52.958343866s) [noise=true , address=104.36.82.6:4874 , piece_id=XVVQN3VX4I2KS4CIXEADBSFYBVYTVEWS5GMXAHE2LWDJGD2A2KAA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.032432775 (223h40m52.958727467s) [noise=true , address=104.36.82.16:4294 , piece_id=36S2PXQADEAQPWSBMQC4XEAH6H5PDA4AO34JI37VD4KPJOU2WVTA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.032754164 (223h40m52.959048852s) [noise=true , address=77.90.31.19:4843 , piece_id=J74IERT5EP2VDSXY5PNWWBVTHQEXT4CWGLWP72Y3CQLOTSI6XZDA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.033080515 (223h40m52.959375213s) [noise=true , address=104.36.82.11:4115 , piece_id=LOK75B4Y2M3DMRJAHNYCVE46DELAAVN4SFP35AWPFESLUJT2V6KQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.033386908 (223h40m52.959681597s) [noise=true , address=104.36.82.14:4222 , piece_id=3HZK5YRWL2KXKY4E7XASRI2UIPCJA72JSJSTGBLPA4ZBP3MGBEOQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.033709080 (223h40m52.960003768s) [noise=true , address=77.90.31.21:4878 , piece_id=QJARHOYLDCFGYTTCD2EYCH43X3EARFK3Q3C7SCZ33X5T6CTLYDMQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.034030732 (223h40m52.96032542s) [noise=true , address=104.36.82.3:4767 , piece_id=U4L7ULC2R5OKDDL6BGDSEQUZNTVMQSGXJ56CGVKHNCCHTMIIDXDQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.034335911 (223h40m52.960630599s) [noise=true , address=45.91.116.8:4047 , piece_id=IJZDAO2K7KADVBNZGHLACI2ARTDDYOHIC742L2IUJT2WFAMMIGSQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.034644696 (223h40m52.960939386s) [noise=true , address=104.36.82.10:4079 , piece_id=JELVGAGG2NF2OSMKXQAV6DIXDTUFIER5WX4KTCJFQQ5P35EAMOEA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.148244089 (223h40m53.074538765s) [upload=179165]: Next returned nil segment
Jul 2 16:52:26.148268128 (223h40m53.074562811s) [upload=179165]: Waiting for error group managing segments...
Jul 2 16:52:26.155850465 (223h40m53.082145153s) [noise=true , address=45.91.116.5:4298 , piece_id=JUYTY35XKYPNJYE7X3R7UMINTVMXDMNM6YZSZORJWATTVDBZFOMQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.156432201 (223h40m53.082726892s) [noise=true , address=77.90.31.14:4661 , piece_id=B5E2MZ5JW7Z5AG4GFQ4QGMAFW7ZMKWKRFU276Z6556JFWKYVLXTQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.156943265 (223h40m53.083237956s) [noise=true , address=104.36.82.16:4294 , piece_id=36S2PXQADEAQPWSBMQC4XEAH6H5PDA4AO34JI37VD4KPJOU2WVTA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.158591570 (223h40m53.084886251s) [noise=true , address=104.36.82.3:4767 , piece_id=U4L7ULC2R5OKDDL6BGDSEQUZNTVMQSGXJ56CGVKHNCCHTMIIDXDQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.159458953 (223h40m53.085753634s) [noise=true , address=45.91.116.8:4047 , piece_id=IJZDAO2K7KADVBNZGHLACI2ARTDDYOHIC742L2IUJT2WFAMMIGSQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.161102375 (223h40m53.087397055s) [noise=true , address=45.91.116.12:4226 , piece_id=NFB2MT72D7MLPOY3A6MVO2QQBUFVYJPKN7D5XJQLQCCMYNQ6WJOA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.161556163 (223h40m53.087850838s) [noise=true , address=104.36.82.12:4149 , piece_id=V5265CLKUYZQN5B663WB4CFR3SXR2OA2BT53OAVCLFZJJ2YY6U4A, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.162013197 (223h40m53.088307874s) [noise=true , address=104.36.82.14:4222 , piece_id=3HZK5YRWL2KXKY4E7XASRI2UIPCJA72JSJSTGBLPA4ZBP3MGBEOQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.162459436 (223h40m53.088754112s) [noise=true , address=77.90.31.15:4698 , piece_id=VDVQDBYDQYRMVP7ZUPSEFC2R5UF2LVW45G23JURMYKS5IT6RBQFQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.162920549 (223h40m53.089215241s) [noise=true , address=77.90.31.21:4878 , piece_id=QJARHOYLDCFGYTTCD2EYCH43X3EARFK3Q3C7SCZ33X5T6CTLYDMQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.163747035 (223h40m53.090041713s) [noise=true , address=104.36.82.11:4115 , piece_id=LOK75B4Y2M3DMRJAHNYCVE46DELAAVN4SFP35AWPFESLUJT2V6KQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.167885784 (223h40m53.094180467s) [noise=true , address=104.36.82.6:4874 , piece_id=XVVQN3VX4I2KS4CIXEADBSFYBVYTVEWS5GMXAHE2LWDJGD2A2KAA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.168494969 (223h40m53.094789668s) [noise=true , address=104.36.82.10:4079 , piece_id=JELVGAGG2NF2OSMKXQAV6DIXDTUFIER5WX4KTCJFQQ5P35EAMOEA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.168525706 (223h40m53.094820396s) [seg_pos={0 0}, upload=179165]: Segment reached optimal threshold of 13 pieces.
Jul 2 16:52:26.168715378 (223h40m53.09501007s) [noise=true , address=77.90.31.19:4843 , piece_id=J74IERT5EP2VDSXY5PNWWBVTHQEXT4CWGLWP72Y3CQLOTSI6XZDA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: expected piece hash; context canceled; EOF
Jul 2 16:52:26.168766821 (223h40m53.095061509s) [noise=true , address=104.36.82.8:4970 , piece_id=X7KKQJYHVNI5LJDKGYNV5JHHCEGF4QSHFRKMSBLRE5W3W2ITAAHQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe): protocol: expected piece hash; context canceled; EOF
Jul 2 16:52:26.168816209 (223h40m53.095110907s) [noise=true , address=45.91.116.11:4118 , piece_id=CSQO6I5J4BW4NUAOFH3QDF6VEJGACWOSYNTNGXVELK2HLXIMX4YQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF
Jul 2 16:52:26.168851649 (223h40m53.095146335s) [seg_pos={0 0}, upload=179165]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 2 16:52:26.169058251 (223h40m53.095352944s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 16:52:26.169066272 (223h40m53.095360967s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 16:52:26.220481259 (223h40m53.146775936s) [upload=179165]: Done uploading segments...
Jul 2 16:52:26.220500093 (223h40m53.14679477s) [upload=179165]: Upload finished. err: <nil>
Duration: 463.268016ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......] duration:
2.067616281s
FAIL (2025-07-02 17:21:48 UTC)
orbiter warning: uplink: metaclient: manager closed: closed: read tcp 159.223.143.39:49282->34.150.199.48:7777: read: connection reset by peer
Duration: 2.067650174s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................X.........................................] duration:
957.462365ms
FAIL (2025-07-02 17:21:09 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
led; ecclient: upload cut due to slow connection (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN): protocol: expected piece hash; context canceled; EOF
Jul 2 16:51:39.852164074 (223h40m6.778458767s) [noise=true , address=104.36.82.12:4150 , piece_id=QHXVON3LSBWYG5AWPVZ45KOTJJPUXIIAXGED7OHG65V5V3IJPULA, seg_pos={0 0}, upload=179153]: Done uploading piece. err: <nil>
Jul 2 16:51:39.852220651 (223h40m6.778515346s) [seg_pos={0 0}, upload=179153]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 2 16:51:39.852324711 (223h40m6.778619405s) [upload=179153]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 16:51:39.852329895 (223h40m6.778624588s) [upload=179153]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 16:51:39.945155283 (223h40m6.871449972s) [upload=179153]: Done uploading segments...
Jul 2 16:51:39.945173208 (223h40m6.871467885s) [upload=179153]: Upload finished. err: <nil>
Jul 2 16:52:25.999553770 (223h40m52.925848461s) [upload=179165]: Starting upload
Jul 2 16:52:25.999676452 (223h40m52.92597114s) [upload=179165]: Done starting upload
Jul 2 16:52:25.999808777 (223h40m52.926103458s) [upload=179165]: Uploading segments...
Jul 2 16:52:26.000049412 (223h40m52.926344119s) [upload=179165]: Got next segment. Inline: false
Jul 2 16:52:26.000208665 (223h40m52.926503358s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 2 16:52:26.000216887 (223h40m52.926511587s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 2 16:52:26.029546629 (223h40m52.95584132s) [seg_pos={0 0}, upload=179165]: Begin upload segment...
Jul 2 16:52:26.029596311 (223h40m52.955890985s) [seg_pos={0 0}, upload=179165]: Done begin upload segment.
Jul 2 16:52:26.029630299 (223h40m52.955924986s) [noise=true , address=45.91.116.12:4226 , piece_id=NFB2MT72D7MLPOY3A6MVO2QQBUFVYJPKN7D5XJQLQCCMYNQ6WJOA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.030073195 (223h40m52.956367894s) [noise=true , address=77.90.31.14:4661 , piece_id=B5E2MZ5JW7Z5AG4GFQ4QGMAFW7ZMKWKRFU276Z6556JFWKYVLXTQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.030398264 (223h40m52.956692955s) [noise=true , address=77.90.31.15:4698 , piece_id=VDVQDBYDQYRMVP7ZUPSEFC2R5UF2LVW45G23JURMYKS5IT6RBQFQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.030716003 (223h40m52.957010692s) [noise=true , address=45.91.116.11:4118 , piece_id=CSQO6I5J4BW4NUAOFH3QDF6VEJGACWOSYNTNGXVELK2HLXIMX4YQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.031068785 (223h40m52.957363472s) [noise=true , address=45.91.116.5:4298 , piece_id=JUYTY35XKYPNJYE7X3R7UMINTVMXDMNM6YZSZORJWATTVDBZFOMQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.031405588 (223h40m52.957700282s) [noise=true , address=104.36.82.8:4970 , piece_id=X7KKQJYHVNI5LJDKGYNV5JHHCEGF4QSHFRKMSBLRE5W3W2ITAAHQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.031730040 (223h40m52.958024715s) [noise=true , address=104.36.82.12:4149 , piece_id=V5265CLKUYZQN5B663WB4CFR3SXR2OA2BT53OAVCLFZJJ2YY6U4A, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.032049179 (223h40m52.958343866s) [noise=true , address=104.36.82.6:4874 , piece_id=XVVQN3VX4I2KS4CIXEADBSFYBVYTVEWS5GMXAHE2LWDJGD2A2KAA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.032432775 (223h40m52.958727467s) [noise=true , address=104.36.82.16:4294 , piece_id=36S2PXQADEAQPWSBMQC4XEAH6H5PDA4AO34JI37VD4KPJOU2WVTA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.032754164 (223h40m52.959048852s) [noise=true , address=77.90.31.19:4843 , piece_id=J74IERT5EP2VDSXY5PNWWBVTHQEXT4CWGLWP72Y3CQLOTSI6XZDA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.033080515 (223h40m52.959375213s) [noise=true , address=104.36.82.11:4115 , piece_id=LOK75B4Y2M3DMRJAHNYCVE46DELAAVN4SFP35AWPFESLUJT2V6KQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.033386908 (223h40m52.959681597s) [noise=true , address=104.36.82.14:4222 , piece_id=3HZK5YRWL2KXKY4E7XASRI2UIPCJA72JSJSTGBLPA4ZBP3MGBEOQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.033709080 (223h40m52.960003768s) [noise=true , address=77.90.31.21:4878 , piece_id=QJARHOYLDCFGYTTCD2EYCH43X3EARFK3Q3C7SCZ33X5T6CTLYDMQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.034030732 (223h40m52.96032542s) [noise=true , address=104.36.82.3:4767 , piece_id=U4L7ULC2R5OKDDL6BGDSEQUZNTVMQSGXJ56CGVKHNCCHTMIIDXDQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.034335911 (223h40m52.960630599s) [noise=true , address=45.91.116.8:4047 , piece_id=IJZDAO2K7KADVBNZGHLACI2ARTDDYOHIC742L2IUJT2WFAMMIGSQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.034644696 (223h40m52.960939386s) [noise=true , address=104.36.82.10:4079 , piece_id=JELVGAGG2NF2OSMKXQAV6DIXDTUFIER5WX4KTCJFQQ5P35EAMOEA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.148244089 (223h40m53.074538765s) [upload=179165]: Next returned nil segment
Jul 2 16:52:26.148268128 (223h40m53.074562811s) [upload=179165]: Waiting for error group managing segments...
Jul 2 16:52:26.155850465 (223h40m53.082145153s) [noise=true , address=45.91.116.5:4298 , piece_id=JUYTY35XKYPNJYE7X3R7UMINTVMXDMNM6YZSZORJWATTVDBZFOMQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.156432201 (223h40m53.082726892s) [noise=true , address=77.90.31.14:4661 , piece_id=B5E2MZ5JW7Z5AG4GFQ4QGMAFW7ZMKWKRFU276Z6556JFWKYVLXTQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.156943265 (223h40m53.083237956s) [noise=true , address=104.36.82.16:4294 , piece_id=36S2PXQADEAQPWSBMQC4XEAH6H5PDA4AO34JI37VD4KPJOU2WVTA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.158591570 (223h40m53.084886251s) [noise=true , address=104.36.82.3:4767 , piece_id=U4L7ULC2R5OKDDL6BGDSEQUZNTVMQSGXJ56CGVKHNCCHTMIIDXDQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.159458953 (223h40m53.085753634s) [noise=true , address=45.91.116.8:4047 , piece_id=IJZDAO2K7KADVBNZGHLACI2ARTDDYOHIC742L2IUJT2WFAMMIGSQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.161102375 (223h40m53.087397055s) [noise=true , address=45.91.116.12:4226 , piece_id=NFB2MT72D7MLPOY3A6MVO2QQBUFVYJPKN7D5XJQLQCCMYNQ6WJOA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.161556163 (223h40m53.087850838s) [noise=true , address=104.36.82.12:4149 , piece_id=V5265CLKUYZQN5B663WB4CFR3SXR2OA2BT53OAVCLFZJJ2YY6U4A, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.162013197 (223h40m53.088307874s) [noise=true , address=104.36.82.14:4222 , piece_id=3HZK5YRWL2KXKY4E7XASRI2UIPCJA72JSJSTGBLPA4ZBP3MGBEOQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.162459436 (223h40m53.088754112s) [noise=true , address=77.90.31.15:4698 , piece_id=VDVQDBYDQYRMVP7ZUPSEFC2R5UF2LVW45G23JURMYKS5IT6RBQFQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.162920549 (223h40m53.089215241s) [noise=true , address=77.90.31.21:4878 , piece_id=QJARHOYLDCFGYTTCD2EYCH43X3EARFK3Q3C7SCZ33X5T6CTLYDMQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.163747035 (223h40m53.090041713s) [noise=true , address=104.36.82.11:4115 , piece_id=LOK75B4Y2M3DMRJAHNYCVE46DELAAVN4SFP35AWPFESLUJT2V6KQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.167885784 (223h40m53.094180467s) [noise=true , address=104.36.82.6:4874 , piece_id=XVVQN3VX4I2KS4CIXEADBSFYBVYTVEWS5GMXAHE2LWDJGD2A2KAA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.168494969 (223h40m53.094789668s) [noise=true , address=104.36.82.10:4079 , piece_id=JELVGAGG2NF2OSMKXQAV6DIXDTUFIER5WX4KTCJFQQ5P35EAMOEA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.168525706 (223h40m53.094820396s) [seg_pos={0 0}, upload=179165]: Segment reached optimal threshold of 13 pieces.
Jul 2 16:52:26.168715378 (223h40m53.09501007s) [noise=true , address=77.90.31.19:4843 , piece_id=J74IERT5EP2VDSXY5PNWWBVTHQEXT4CWGLWP72Y3CQLOTSI6XZDA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: expected piece hash; context canceled; EOF
Jul 2 16:52:26.168766821 (223h40m53.095061509s) [noise=true , address=104.36.82.8:4970 , piece_id=X7KKQJYHVNI5LJDKGYNV5JHHCEGF4QSHFRKMSBLRE5W3W2ITAAHQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe): protocol: expected piece hash; context canceled; EOF
Jul 2 16:52:26.168816209 (223h40m53.095110907s) [noise=true , address=45.91.116.11:4118 , piece_id=CSQO6I5J4BW4NUAOFH3QDF6VEJGACWOSYNTNGXVELK2HLXIMX4YQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF
Jul 2 16:52:26.168851649 (223h40m53.095146335s) [seg_pos={0 0}, upload=179165]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 2 16:52:26.169058251 (223h40m53.095352944s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 16:52:26.169066272 (223h40m53.095360967s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 16:52:26.220481259 (223h40m53.146775936s) [upload=179165]: Done uploading segments...
Jul 2 16:52:26.220500093 (223h40m53.14679477s) [upload=179165]: Upload finished. err: <nil>
Duration: 957.49001ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......] duration:
2.075410137s
FAIL (2025-07-02 17:20:39 UTC)
orbiter warning: uplink: metaclient: manager closed: closed: read tcp 159.223.143.39:49282->34.150.199.48:7777: read: connection reset by peer
Duration: 2.075437756s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................X........................................] duration:
1.528480029s
FAIL (2025-07-02 17:20:19 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
led; ecclient: upload cut due to slow connection (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN): protocol: expected piece hash; context canceled; EOF
Jul 2 16:51:39.852164074 (223h40m6.778458767s) [noise=true , address=104.36.82.12:4150 , piece_id=QHXVON3LSBWYG5AWPVZ45KOTJJPUXIIAXGED7OHG65V5V3IJPULA, seg_pos={0 0}, upload=179153]: Done uploading piece. err: <nil>
Jul 2 16:51:39.852220651 (223h40m6.778515346s) [seg_pos={0 0}, upload=179153]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 2 16:51:39.852324711 (223h40m6.778619405s) [upload=179153]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 16:51:39.852329895 (223h40m6.778624588s) [upload=179153]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 16:51:39.945155283 (223h40m6.871449972s) [upload=179153]: Done uploading segments...
Jul 2 16:51:39.945173208 (223h40m6.871467885s) [upload=179153]: Upload finished. err: <nil>
Jul 2 16:52:25.999553770 (223h40m52.925848461s) [upload=179165]: Starting upload
Jul 2 16:52:25.999676452 (223h40m52.92597114s) [upload=179165]: Done starting upload
Jul 2 16:52:25.999808777 (223h40m52.926103458s) [upload=179165]: Uploading segments...
Jul 2 16:52:26.000049412 (223h40m52.926344119s) [upload=179165]: Got next segment. Inline: false
Jul 2 16:52:26.000208665 (223h40m52.926503358s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 2 16:52:26.000216887 (223h40m52.926511587s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 2 16:52:26.029546629 (223h40m52.95584132s) [seg_pos={0 0}, upload=179165]: Begin upload segment...
Jul 2 16:52:26.029596311 (223h40m52.955890985s) [seg_pos={0 0}, upload=179165]: Done begin upload segment.
Jul 2 16:52:26.029630299 (223h40m52.955924986s) [noise=true , address=45.91.116.12:4226 , piece_id=NFB2MT72D7MLPOY3A6MVO2QQBUFVYJPKN7D5XJQLQCCMYNQ6WJOA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.030073195 (223h40m52.956367894s) [noise=true , address=77.90.31.14:4661 , piece_id=B5E2MZ5JW7Z5AG4GFQ4QGMAFW7ZMKWKRFU276Z6556JFWKYVLXTQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.030398264 (223h40m52.956692955s) [noise=true , address=77.90.31.15:4698 , piece_id=VDVQDBYDQYRMVP7ZUPSEFC2R5UF2LVW45G23JURMYKS5IT6RBQFQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.030716003 (223h40m52.957010692s) [noise=true , address=45.91.116.11:4118 , piece_id=CSQO6I5J4BW4NUAOFH3QDF6VEJGACWOSYNTNGXVELK2HLXIMX4YQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.031068785 (223h40m52.957363472s) [noise=true , address=45.91.116.5:4298 , piece_id=JUYTY35XKYPNJYE7X3R7UMINTVMXDMNM6YZSZORJWATTVDBZFOMQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.031405588 (223h40m52.957700282s) [noise=true , address=104.36.82.8:4970 , piece_id=X7KKQJYHVNI5LJDKGYNV5JHHCEGF4QSHFRKMSBLRE5W3W2ITAAHQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.031730040 (223h40m52.958024715s) [noise=true , address=104.36.82.12:4149 , piece_id=V5265CLKUYZQN5B663WB4CFR3SXR2OA2BT53OAVCLFZJJ2YY6U4A, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.032049179 (223h40m52.958343866s) [noise=true , address=104.36.82.6:4874 , piece_id=XVVQN3VX4I2KS4CIXEADBSFYBVYTVEWS5GMXAHE2LWDJGD2A2KAA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.032432775 (223h40m52.958727467s) [noise=true , address=104.36.82.16:4294 , piece_id=36S2PXQADEAQPWSBMQC4XEAH6H5PDA4AO34JI37VD4KPJOU2WVTA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.032754164 (223h40m52.959048852s) [noise=true , address=77.90.31.19:4843 , piece_id=J74IERT5EP2VDSXY5PNWWBVTHQEXT4CWGLWP72Y3CQLOTSI6XZDA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.033080515 (223h40m52.959375213s) [noise=true , address=104.36.82.11:4115 , piece_id=LOK75B4Y2M3DMRJAHNYCVE46DELAAVN4SFP35AWPFESLUJT2V6KQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.033386908 (223h40m52.959681597s) [noise=true , address=104.36.82.14:4222 , piece_id=3HZK5YRWL2KXKY4E7XASRI2UIPCJA72JSJSTGBLPA4ZBP3MGBEOQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.033709080 (223h40m52.960003768s) [noise=true , address=77.90.31.21:4878 , piece_id=QJARHOYLDCFGYTTCD2EYCH43X3EARFK3Q3C7SCZ33X5T6CTLYDMQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.034030732 (223h40m52.96032542s) [noise=true , address=104.36.82.3:4767 , piece_id=U4L7ULC2R5OKDDL6BGDSEQUZNTVMQSGXJ56CGVKHNCCHTMIIDXDQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.034335911 (223h40m52.960630599s) [noise=true , address=45.91.116.8:4047 , piece_id=IJZDAO2K7KADVBNZGHLACI2ARTDDYOHIC742L2IUJT2WFAMMIGSQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.034644696 (223h40m52.960939386s) [noise=true , address=104.36.82.10:4079 , piece_id=JELVGAGG2NF2OSMKXQAV6DIXDTUFIER5WX4KTCJFQQ5P35EAMOEA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.148244089 (223h40m53.074538765s) [upload=179165]: Next returned nil segment
Jul 2 16:52:26.148268128 (223h40m53.074562811s) [upload=179165]: Waiting for error group managing segments...
Jul 2 16:52:26.155850465 (223h40m53.082145153s) [noise=true , address=45.91.116.5:4298 , piece_id=JUYTY35XKYPNJYE7X3R7UMINTVMXDMNM6YZSZORJWATTVDBZFOMQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.156432201 (223h40m53.082726892s) [noise=true , address=77.90.31.14:4661 , piece_id=B5E2MZ5JW7Z5AG4GFQ4QGMAFW7ZMKWKRFU276Z6556JFWKYVLXTQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.156943265 (223h40m53.083237956s) [noise=true , address=104.36.82.16:4294 , piece_id=36S2PXQADEAQPWSBMQC4XEAH6H5PDA4AO34JI37VD4KPJOU2WVTA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.158591570 (223h40m53.084886251s) [noise=true , address=104.36.82.3:4767 , piece_id=U4L7ULC2R5OKDDL6BGDSEQUZNTVMQSGXJ56CGVKHNCCHTMIIDXDQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.159458953 (223h40m53.085753634s) [noise=true , address=45.91.116.8:4047 , piece_id=IJZDAO2K7KADVBNZGHLACI2ARTDDYOHIC742L2IUJT2WFAMMIGSQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.161102375 (223h40m53.087397055s) [noise=true , address=45.91.116.12:4226 , piece_id=NFB2MT72D7MLPOY3A6MVO2QQBUFVYJPKN7D5XJQLQCCMYNQ6WJOA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.161556163 (223h40m53.087850838s) [noise=true , address=104.36.82.12:4149 , piece_id=V5265CLKUYZQN5B663WB4CFR3SXR2OA2BT53OAVCLFZJJ2YY6U4A, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.162013197 (223h40m53.088307874s) [noise=true , address=104.36.82.14:4222 , piece_id=3HZK5YRWL2KXKY4E7XASRI2UIPCJA72JSJSTGBLPA4ZBP3MGBEOQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.162459436 (223h40m53.088754112s) [noise=true , address=77.90.31.15:4698 , piece_id=VDVQDBYDQYRMVP7ZUPSEFC2R5UF2LVW45G23JURMYKS5IT6RBQFQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.162920549 (223h40m53.089215241s) [noise=true , address=77.90.31.21:4878 , piece_id=QJARHOYLDCFGYTTCD2EYCH43X3EARFK3Q3C7SCZ33X5T6CTLYDMQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.163747035 (223h40m53.090041713s) [noise=true , address=104.36.82.11:4115 , piece_id=LOK75B4Y2M3DMRJAHNYCVE46DELAAVN4SFP35AWPFESLUJT2V6KQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.167885784 (223h40m53.094180467s) [noise=true , address=104.36.82.6:4874 , piece_id=XVVQN3VX4I2KS4CIXEADBSFYBVYTVEWS5GMXAHE2LWDJGD2A2KAA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.168494969 (223h40m53.094789668s) [noise=true , address=104.36.82.10:4079 , piece_id=JELVGAGG2NF2OSMKXQAV6DIXDTUFIER5WX4KTCJFQQ5P35EAMOEA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.168525706 (223h40m53.094820396s) [seg_pos={0 0}, upload=179165]: Segment reached optimal threshold of 13 pieces.
Jul 2 16:52:26.168715378 (223h40m53.09501007s) [noise=true , address=77.90.31.19:4843 , piece_id=J74IERT5EP2VDSXY5PNWWBVTHQEXT4CWGLWP72Y3CQLOTSI6XZDA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: expected piece hash; context canceled; EOF
Jul 2 16:52:26.168766821 (223h40m53.095061509s) [noise=true , address=104.36.82.8:4970 , piece_id=X7KKQJYHVNI5LJDKGYNV5JHHCEGF4QSHFRKMSBLRE5W3W2ITAAHQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe): protocol: expected piece hash; context canceled; EOF
Jul 2 16:52:26.168816209 (223h40m53.095110907s) [noise=true , address=45.91.116.11:4118 , piece_id=CSQO6I5J4BW4NUAOFH3QDF6VEJGACWOSYNTNGXVELK2HLXIMX4YQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF
Jul 2 16:52:26.168851649 (223h40m53.095146335s) [seg_pos={0 0}, upload=179165]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 2 16:52:26.169058251 (223h40m53.095352944s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 16:52:26.169066272 (223h40m53.095360967s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 16:52:26.220481259 (223h40m53.146775936s) [upload=179165]: Done uploading segments...
Jul 2 16:52:26.220500093 (223h40m53.14679477s) [upload=179165]: Upload finished. err: <nil>
Duration: 1.52852s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................X.......................................] duration:
914.39351ms
FAIL (2025-07-02 17:20:00 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
led; ecclient: upload cut due to slow connection (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN): protocol: expected piece hash; context canceled; EOF
Jul 2 16:51:39.852164074 (223h40m6.778458767s) [noise=true , address=104.36.82.12:4150 , piece_id=QHXVON3LSBWYG5AWPVZ45KOTJJPUXIIAXGED7OHG65V5V3IJPULA, seg_pos={0 0}, upload=179153]: Done uploading piece. err: <nil>
Jul 2 16:51:39.852220651 (223h40m6.778515346s) [seg_pos={0 0}, upload=179153]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 2 16:51:39.852324711 (223h40m6.778619405s) [upload=179153]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 16:51:39.852329895 (223h40m6.778624588s) [upload=179153]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 16:51:39.945155283 (223h40m6.871449972s) [upload=179153]: Done uploading segments...
Jul 2 16:51:39.945173208 (223h40m6.871467885s) [upload=179153]: Upload finished. err: <nil>
Jul 2 16:52:25.999553770 (223h40m52.925848461s) [upload=179165]: Starting upload
Jul 2 16:52:25.999676452 (223h40m52.92597114s) [upload=179165]: Done starting upload
Jul 2 16:52:25.999808777 (223h40m52.926103458s) [upload=179165]: Uploading segments...
Jul 2 16:52:26.000049412 (223h40m52.926344119s) [upload=179165]: Got next segment. Inline: false
Jul 2 16:52:26.000208665 (223h40m52.926503358s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 2 16:52:26.000216887 (223h40m52.926511587s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 2 16:52:26.029546629 (223h40m52.95584132s) [seg_pos={0 0}, upload=179165]: Begin upload segment...
Jul 2 16:52:26.029596311 (223h40m52.955890985s) [seg_pos={0 0}, upload=179165]: Done begin upload segment.
Jul 2 16:52:26.029630299 (223h40m52.955924986s) [noise=true , address=45.91.116.12:4226 , piece_id=NFB2MT72D7MLPOY3A6MVO2QQBUFVYJPKN7D5XJQLQCCMYNQ6WJOA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.030073195 (223h40m52.956367894s) [noise=true , address=77.90.31.14:4661 , piece_id=B5E2MZ5JW7Z5AG4GFQ4QGMAFW7ZMKWKRFU276Z6556JFWKYVLXTQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.030398264 (223h40m52.956692955s) [noise=true , address=77.90.31.15:4698 , piece_id=VDVQDBYDQYRMVP7ZUPSEFC2R5UF2LVW45G23JURMYKS5IT6RBQFQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.030716003 (223h40m52.957010692s) [noise=true , address=45.91.116.11:4118 , piece_id=CSQO6I5J4BW4NUAOFH3QDF6VEJGACWOSYNTNGXVELK2HLXIMX4YQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.031068785 (223h40m52.957363472s) [noise=true , address=45.91.116.5:4298 , piece_id=JUYTY35XKYPNJYE7X3R7UMINTVMXDMNM6YZSZORJWATTVDBZFOMQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.031405588 (223h40m52.957700282s) [noise=true , address=104.36.82.8:4970 , piece_id=X7KKQJYHVNI5LJDKGYNV5JHHCEGF4QSHFRKMSBLRE5W3W2ITAAHQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.031730040 (223h40m52.958024715s) [noise=true , address=104.36.82.12:4149 , piece_id=V5265CLKUYZQN5B663WB4CFR3SXR2OA2BT53OAVCLFZJJ2YY6U4A, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.032049179 (223h40m52.958343866s) [noise=true , address=104.36.82.6:4874 , piece_id=XVVQN3VX4I2KS4CIXEADBSFYBVYTVEWS5GMXAHE2LWDJGD2A2KAA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.032432775 (223h40m52.958727467s) [noise=true , address=104.36.82.16:4294 , piece_id=36S2PXQADEAQPWSBMQC4XEAH6H5PDA4AO34JI37VD4KPJOU2WVTA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.032754164 (223h40m52.959048852s) [noise=true , address=77.90.31.19:4843 , piece_id=J74IERT5EP2VDSXY5PNWWBVTHQEXT4CWGLWP72Y3CQLOTSI6XZDA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.033080515 (223h40m52.959375213s) [noise=true , address=104.36.82.11:4115 , piece_id=LOK75B4Y2M3DMRJAHNYCVE46DELAAVN4SFP35AWPFESLUJT2V6KQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.033386908 (223h40m52.959681597s) [noise=true , address=104.36.82.14:4222 , piece_id=3HZK5YRWL2KXKY4E7XASRI2UIPCJA72JSJSTGBLPA4ZBP3MGBEOQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.033709080 (223h40m52.960003768s) [noise=true , address=77.90.31.21:4878 , piece_id=QJARHOYLDCFGYTTCD2EYCH43X3EARFK3Q3C7SCZ33X5T6CTLYDMQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.034030732 (223h40m52.96032542s) [noise=true , address=104.36.82.3:4767 , piece_id=U4L7ULC2R5OKDDL6BGDSEQUZNTVMQSGXJ56CGVKHNCCHTMIIDXDQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.034335911 (223h40m52.960630599s) [noise=true , address=45.91.116.8:4047 , piece_id=IJZDAO2K7KADVBNZGHLACI2ARTDDYOHIC742L2IUJT2WFAMMIGSQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.034644696 (223h40m52.960939386s) [noise=true , address=104.36.82.10:4079 , piece_id=JELVGAGG2NF2OSMKXQAV6DIXDTUFIER5WX4KTCJFQQ5P35EAMOEA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.148244089 (223h40m53.074538765s) [upload=179165]: Next returned nil segment
Jul 2 16:52:26.148268128 (223h40m53.074562811s) [upload=179165]: Waiting for error group managing segments...
Jul 2 16:52:26.155850465 (223h40m53.082145153s) [noise=true , address=45.91.116.5:4298 , piece_id=JUYTY35XKYPNJYE7X3R7UMINTVMXDMNM6YZSZORJWATTVDBZFOMQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.156432201 (223h40m53.082726892s) [noise=true , address=77.90.31.14:4661 , piece_id=B5E2MZ5JW7Z5AG4GFQ4QGMAFW7ZMKWKRFU276Z6556JFWKYVLXTQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.156943265 (223h40m53.083237956s) [noise=true , address=104.36.82.16:4294 , piece_id=36S2PXQADEAQPWSBMQC4XEAH6H5PDA4AO34JI37VD4KPJOU2WVTA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.158591570 (223h40m53.084886251s) [noise=true , address=104.36.82.3:4767 , piece_id=U4L7ULC2R5OKDDL6BGDSEQUZNTVMQSGXJ56CGVKHNCCHTMIIDXDQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.159458953 (223h40m53.085753634s) [noise=true , address=45.91.116.8:4047 , piece_id=IJZDAO2K7KADVBNZGHLACI2ARTDDYOHIC742L2IUJT2WFAMMIGSQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.161102375 (223h40m53.087397055s) [noise=true , address=45.91.116.12:4226 , piece_id=NFB2MT72D7MLPOY3A6MVO2QQBUFVYJPKN7D5XJQLQCCMYNQ6WJOA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.161556163 (223h40m53.087850838s) [noise=true , address=104.36.82.12:4149 , piece_id=V5265CLKUYZQN5B663WB4CFR3SXR2OA2BT53OAVCLFZJJ2YY6U4A, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.162013197 (223h40m53.088307874s) [noise=true , address=104.36.82.14:4222 , piece_id=3HZK5YRWL2KXKY4E7XASRI2UIPCJA72JSJSTGBLPA4ZBP3MGBEOQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.162459436 (223h40m53.088754112s) [noise=true , address=77.90.31.15:4698 , piece_id=VDVQDBYDQYRMVP7ZUPSEFC2R5UF2LVW45G23JURMYKS5IT6RBQFQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.162920549 (223h40m53.089215241s) [noise=true , address=77.90.31.21:4878 , piece_id=QJARHOYLDCFGYTTCD2EYCH43X3EARFK3Q3C7SCZ33X5T6CTLYDMQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.163747035 (223h40m53.090041713s) [noise=true , address=104.36.82.11:4115 , piece_id=LOK75B4Y2M3DMRJAHNYCVE46DELAAVN4SFP35AWPFESLUJT2V6KQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.167885784 (223h40m53.094180467s) [noise=true , address=104.36.82.6:4874 , piece_id=XVVQN3VX4I2KS4CIXEADBSFYBVYTVEWS5GMXAHE2LWDJGD2A2KAA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.168494969 (223h40m53.094789668s) [noise=true , address=104.36.82.10:4079 , piece_id=JELVGAGG2NF2OSMKXQAV6DIXDTUFIER5WX4KTCJFQQ5P35EAMOEA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.168525706 (223h40m53.094820396s) [seg_pos={0 0}, upload=179165]: Segment reached optimal threshold of 13 pieces.
Jul 2 16:52:26.168715378 (223h40m53.09501007s) [noise=true , address=77.90.31.19:4843 , piece_id=J74IERT5EP2VDSXY5PNWWBVTHQEXT4CWGLWP72Y3CQLOTSI6XZDA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: expected piece hash; context canceled; EOF
Jul 2 16:52:26.168766821 (223h40m53.095061509s) [noise=true , address=104.36.82.8:4970 , piece_id=X7KKQJYHVNI5LJDKGYNV5JHHCEGF4QSHFRKMSBLRE5W3W2ITAAHQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe): protocol: expected piece hash; context canceled; EOF
Jul 2 16:52:26.168816209 (223h40m53.095110907s) [noise=true , address=45.91.116.11:4118 , piece_id=CSQO6I5J4BW4NUAOFH3QDF6VEJGACWOSYNTNGXVELK2HLXIMX4YQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF
Jul 2 16:52:26.168851649 (223h40m53.095146335s) [seg_pos={0 0}, upload=179165]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 2 16:52:26.169058251 (223h40m53.095352944s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 16:52:26.169066272 (223h40m53.095360967s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 16:52:26.220481259 (223h40m53.146775936s) [upload=179165]: Done uploading segments...
Jul 2 16:52:26.220500093 (223h40m53.14679477s) [upload=179165]: Upload finished. err: <nil>
Duration: 914.428692ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....] duration:
2.169003844s
FAIL (2025-07-02 17:19:50 UTC)
orbiter warning: uplink: metaclient: manager closed: closed: read tcp 159.223.143.39:49282->34.150.199.48:7777: read: connection reset by peer
Duration: 2.169035018s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................X......................................] duration:
798.278827ms
FAIL (2025-07-02 17:19:25 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
led; ecclient: upload cut due to slow connection (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN): protocol: expected piece hash; context canceled; EOF
Jul 2 16:51:39.852164074 (223h40m6.778458767s) [noise=true , address=104.36.82.12:4150 , piece_id=QHXVON3LSBWYG5AWPVZ45KOTJJPUXIIAXGED7OHG65V5V3IJPULA, seg_pos={0 0}, upload=179153]: Done uploading piece. err: <nil>
Jul 2 16:51:39.852220651 (223h40m6.778515346s) [seg_pos={0 0}, upload=179153]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 2 16:51:39.852324711 (223h40m6.778619405s) [upload=179153]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 16:51:39.852329895 (223h40m6.778624588s) [upload=179153]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 16:51:39.945155283 (223h40m6.871449972s) [upload=179153]: Done uploading segments...
Jul 2 16:51:39.945173208 (223h40m6.871467885s) [upload=179153]: Upload finished. err: <nil>
Jul 2 16:52:25.999553770 (223h40m52.925848461s) [upload=179165]: Starting upload
Jul 2 16:52:25.999676452 (223h40m52.92597114s) [upload=179165]: Done starting upload
Jul 2 16:52:25.999808777 (223h40m52.926103458s) [upload=179165]: Uploading segments...
Jul 2 16:52:26.000049412 (223h40m52.926344119s) [upload=179165]: Got next segment. Inline: false
Jul 2 16:52:26.000208665 (223h40m52.926503358s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 2 16:52:26.000216887 (223h40m52.926511587s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 2 16:52:26.029546629 (223h40m52.95584132s) [seg_pos={0 0}, upload=179165]: Begin upload segment...
Jul 2 16:52:26.029596311 (223h40m52.955890985s) [seg_pos={0 0}, upload=179165]: Done begin upload segment.
Jul 2 16:52:26.029630299 (223h40m52.955924986s) [noise=true , address=45.91.116.12:4226 , piece_id=NFB2MT72D7MLPOY3A6MVO2QQBUFVYJPKN7D5XJQLQCCMYNQ6WJOA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.030073195 (223h40m52.956367894s) [noise=true , address=77.90.31.14:4661 , piece_id=B5E2MZ5JW7Z5AG4GFQ4QGMAFW7ZMKWKRFU276Z6556JFWKYVLXTQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.030398264 (223h40m52.956692955s) [noise=true , address=77.90.31.15:4698 , piece_id=VDVQDBYDQYRMVP7ZUPSEFC2R5UF2LVW45G23JURMYKS5IT6RBQFQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.030716003 (223h40m52.957010692s) [noise=true , address=45.91.116.11:4118 , piece_id=CSQO6I5J4BW4NUAOFH3QDF6VEJGACWOSYNTNGXVELK2HLXIMX4YQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.031068785 (223h40m52.957363472s) [noise=true , address=45.91.116.5:4298 , piece_id=JUYTY35XKYPNJYE7X3R7UMINTVMXDMNM6YZSZORJWATTVDBZFOMQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.031405588 (223h40m52.957700282s) [noise=true , address=104.36.82.8:4970 , piece_id=X7KKQJYHVNI5LJDKGYNV5JHHCEGF4QSHFRKMSBLRE5W3W2ITAAHQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.031730040 (223h40m52.958024715s) [noise=true , address=104.36.82.12:4149 , piece_id=V5265CLKUYZQN5B663WB4CFR3SXR2OA2BT53OAVCLFZJJ2YY6U4A, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.032049179 (223h40m52.958343866s) [noise=true , address=104.36.82.6:4874 , piece_id=XVVQN3VX4I2KS4CIXEADBSFYBVYTVEWS5GMXAHE2LWDJGD2A2KAA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.032432775 (223h40m52.958727467s) [noise=true , address=104.36.82.16:4294 , piece_id=36S2PXQADEAQPWSBMQC4XEAH6H5PDA4AO34JI37VD4KPJOU2WVTA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.032754164 (223h40m52.959048852s) [noise=true , address=77.90.31.19:4843 , piece_id=J74IERT5EP2VDSXY5PNWWBVTHQEXT4CWGLWP72Y3CQLOTSI6XZDA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.033080515 (223h40m52.959375213s) [noise=true , address=104.36.82.11:4115 , piece_id=LOK75B4Y2M3DMRJAHNYCVE46DELAAVN4SFP35AWPFESLUJT2V6KQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.033386908 (223h40m52.959681597s) [noise=true , address=104.36.82.14:4222 , piece_id=3HZK5YRWL2KXKY4E7XASRI2UIPCJA72JSJSTGBLPA4ZBP3MGBEOQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.033709080 (223h40m52.960003768s) [noise=true , address=77.90.31.21:4878 , piece_id=QJARHOYLDCFGYTTCD2EYCH43X3EARFK3Q3C7SCZ33X5T6CTLYDMQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.034030732 (223h40m52.96032542s) [noise=true , address=104.36.82.3:4767 , piece_id=U4L7ULC2R5OKDDL6BGDSEQUZNTVMQSGXJ56CGVKHNCCHTMIIDXDQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.034335911 (223h40m52.960630599s) [noise=true , address=45.91.116.8:4047 , piece_id=IJZDAO2K7KADVBNZGHLACI2ARTDDYOHIC742L2IUJT2WFAMMIGSQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.034644696 (223h40m52.960939386s) [noise=true , address=104.36.82.10:4079 , piece_id=JELVGAGG2NF2OSMKXQAV6DIXDTUFIER5WX4KTCJFQQ5P35EAMOEA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.148244089 (223h40m53.074538765s) [upload=179165]: Next returned nil segment
Jul 2 16:52:26.148268128 (223h40m53.074562811s) [upload=179165]: Waiting for error group managing segments...
Jul 2 16:52:26.155850465 (223h40m53.082145153s) [noise=true , address=45.91.116.5:4298 , piece_id=JUYTY35XKYPNJYE7X3R7UMINTVMXDMNM6YZSZORJWATTVDBZFOMQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.156432201 (223h40m53.082726892s) [noise=true , address=77.90.31.14:4661 , piece_id=B5E2MZ5JW7Z5AG4GFQ4QGMAFW7ZMKWKRFU276Z6556JFWKYVLXTQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.156943265 (223h40m53.083237956s) [noise=true , address=104.36.82.16:4294 , piece_id=36S2PXQADEAQPWSBMQC4XEAH6H5PDA4AO34JI37VD4KPJOU2WVTA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.158591570 (223h40m53.084886251s) [noise=true , address=104.36.82.3:4767 , piece_id=U4L7ULC2R5OKDDL6BGDSEQUZNTVMQSGXJ56CGVKHNCCHTMIIDXDQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.159458953 (223h40m53.085753634s) [noise=true , address=45.91.116.8:4047 , piece_id=IJZDAO2K7KADVBNZGHLACI2ARTDDYOHIC742L2IUJT2WFAMMIGSQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.161102375 (223h40m53.087397055s) [noise=true , address=45.91.116.12:4226 , piece_id=NFB2MT72D7MLPOY3A6MVO2QQBUFVYJPKN7D5XJQLQCCMYNQ6WJOA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.161556163 (223h40m53.087850838s) [noise=true , address=104.36.82.12:4149 , piece_id=V5265CLKUYZQN5B663WB4CFR3SXR2OA2BT53OAVCLFZJJ2YY6U4A, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.162013197 (223h40m53.088307874s) [noise=true , address=104.36.82.14:4222 , piece_id=3HZK5YRWL2KXKY4E7XASRI2UIPCJA72JSJSTGBLPA4ZBP3MGBEOQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.162459436 (223h40m53.088754112s) [noise=true , address=77.90.31.15:4698 , piece_id=VDVQDBYDQYRMVP7ZUPSEFC2R5UF2LVW45G23JURMYKS5IT6RBQFQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.162920549 (223h40m53.089215241s) [noise=true , address=77.90.31.21:4878 , piece_id=QJARHOYLDCFGYTTCD2EYCH43X3EARFK3Q3C7SCZ33X5T6CTLYDMQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.163747035 (223h40m53.090041713s) [noise=true , address=104.36.82.11:4115 , piece_id=LOK75B4Y2M3DMRJAHNYCVE46DELAAVN4SFP35AWPFESLUJT2V6KQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.167885784 (223h40m53.094180467s) [noise=true , address=104.36.82.6:4874 , piece_id=XVVQN3VX4I2KS4CIXEADBSFYBVYTVEWS5GMXAHE2LWDJGD2A2KAA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.168494969 (223h40m53.094789668s) [noise=true , address=104.36.82.10:4079 , piece_id=JELVGAGG2NF2OSMKXQAV6DIXDTUFIER5WX4KTCJFQQ5P35EAMOEA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.168525706 (223h40m53.094820396s) [seg_pos={0 0}, upload=179165]: Segment reached optimal threshold of 13 pieces.
Jul 2 16:52:26.168715378 (223h40m53.09501007s) [noise=true , address=77.90.31.19:4843 , piece_id=J74IERT5EP2VDSXY5PNWWBVTHQEXT4CWGLWP72Y3CQLOTSI6XZDA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: expected piece hash; context canceled; EOF
Jul 2 16:52:26.168766821 (223h40m53.095061509s) [noise=true , address=104.36.82.8:4970 , piece_id=X7KKQJYHVNI5LJDKGYNV5JHHCEGF4QSHFRKMSBLRE5W3W2ITAAHQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe): protocol: expected piece hash; context canceled; EOF
Jul 2 16:52:26.168816209 (223h40m53.095110907s) [noise=true , address=45.91.116.11:4118 , piece_id=CSQO6I5J4BW4NUAOFH3QDF6VEJGACWOSYNTNGXVELK2HLXIMX4YQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF
Jul 2 16:52:26.168851649 (223h40m53.095146335s) [seg_pos={0 0}, upload=179165]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 2 16:52:26.169058251 (223h40m53.095352944s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 16:52:26.169066272 (223h40m53.095360967s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 16:52:26.220481259 (223h40m53.146775936s) [upload=179165]: Done uploading segments...
Jul 2 16:52:26.220500093 (223h40m53.14679477s) [upload=179165]: Upload finished. err: <nil>
Duration: 798.309411ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................X.....................................] duration:
986.725907ms
FAIL (2025-07-02 17:19:09 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
led; ecclient: upload cut due to slow connection (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN): protocol: expected piece hash; context canceled; EOF
Jul 2 16:51:39.852164074 (223h40m6.778458767s) [noise=true , address=104.36.82.12:4150 , piece_id=QHXVON3LSBWYG5AWPVZ45KOTJJPUXIIAXGED7OHG65V5V3IJPULA, seg_pos={0 0}, upload=179153]: Done uploading piece. err: <nil>
Jul 2 16:51:39.852220651 (223h40m6.778515346s) [seg_pos={0 0}, upload=179153]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 2 16:51:39.852324711 (223h40m6.778619405s) [upload=179153]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 16:51:39.852329895 (223h40m6.778624588s) [upload=179153]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 16:51:39.945155283 (223h40m6.871449972s) [upload=179153]: Done uploading segments...
Jul 2 16:51:39.945173208 (223h40m6.871467885s) [upload=179153]: Upload finished. err: <nil>
Jul 2 16:52:25.999553770 (223h40m52.925848461s) [upload=179165]: Starting upload
Jul 2 16:52:25.999676452 (223h40m52.92597114s) [upload=179165]: Done starting upload
Jul 2 16:52:25.999808777 (223h40m52.926103458s) [upload=179165]: Uploading segments...
Jul 2 16:52:26.000049412 (223h40m52.926344119s) [upload=179165]: Got next segment. Inline: false
Jul 2 16:52:26.000208665 (223h40m52.926503358s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 2 16:52:26.000216887 (223h40m52.926511587s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 2 16:52:26.029546629 (223h40m52.95584132s) [seg_pos={0 0}, upload=179165]: Begin upload segment...
Jul 2 16:52:26.029596311 (223h40m52.955890985s) [seg_pos={0 0}, upload=179165]: Done begin upload segment.
Jul 2 16:52:26.029630299 (223h40m52.955924986s) [noise=true , address=45.91.116.12:4226 , piece_id=NFB2MT72D7MLPOY3A6MVO2QQBUFVYJPKN7D5XJQLQCCMYNQ6WJOA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.030073195 (223h40m52.956367894s) [noise=true , address=77.90.31.14:4661 , piece_id=B5E2MZ5JW7Z5AG4GFQ4QGMAFW7ZMKWKRFU276Z6556JFWKYVLXTQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.030398264 (223h40m52.956692955s) [noise=true , address=77.90.31.15:4698 , piece_id=VDVQDBYDQYRMVP7ZUPSEFC2R5UF2LVW45G23JURMYKS5IT6RBQFQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.030716003 (223h40m52.957010692s) [noise=true , address=45.91.116.11:4118 , piece_id=CSQO6I5J4BW4NUAOFH3QDF6VEJGACWOSYNTNGXVELK2HLXIMX4YQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.031068785 (223h40m52.957363472s) [noise=true , address=45.91.116.5:4298 , piece_id=JUYTY35XKYPNJYE7X3R7UMINTVMXDMNM6YZSZORJWATTVDBZFOMQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.031405588 (223h40m52.957700282s) [noise=true , address=104.36.82.8:4970 , piece_id=X7KKQJYHVNI5LJDKGYNV5JHHCEGF4QSHFRKMSBLRE5W3W2ITAAHQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.031730040 (223h40m52.958024715s) [noise=true , address=104.36.82.12:4149 , piece_id=V5265CLKUYZQN5B663WB4CFR3SXR2OA2BT53OAVCLFZJJ2YY6U4A, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.032049179 (223h40m52.958343866s) [noise=true , address=104.36.82.6:4874 , piece_id=XVVQN3VX4I2KS4CIXEADBSFYBVYTVEWS5GMXAHE2LWDJGD2A2KAA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.032432775 (223h40m52.958727467s) [noise=true , address=104.36.82.16:4294 , piece_id=36S2PXQADEAQPWSBMQC4XEAH6H5PDA4AO34JI37VD4KPJOU2WVTA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.032754164 (223h40m52.959048852s) [noise=true , address=77.90.31.19:4843 , piece_id=J74IERT5EP2VDSXY5PNWWBVTHQEXT4CWGLWP72Y3CQLOTSI6XZDA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.033080515 (223h40m52.959375213s) [noise=true , address=104.36.82.11:4115 , piece_id=LOK75B4Y2M3DMRJAHNYCVE46DELAAVN4SFP35AWPFESLUJT2V6KQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.033386908 (223h40m52.959681597s) [noise=true , address=104.36.82.14:4222 , piece_id=3HZK5YRWL2KXKY4E7XASRI2UIPCJA72JSJSTGBLPA4ZBP3MGBEOQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.033709080 (223h40m52.960003768s) [noise=true , address=77.90.31.21:4878 , piece_id=QJARHOYLDCFGYTTCD2EYCH43X3EARFK3Q3C7SCZ33X5T6CTLYDMQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.034030732 (223h40m52.96032542s) [noise=true , address=104.36.82.3:4767 , piece_id=U4L7ULC2R5OKDDL6BGDSEQUZNTVMQSGXJ56CGVKHNCCHTMIIDXDQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.034335911 (223h40m52.960630599s) [noise=true , address=45.91.116.8:4047 , piece_id=IJZDAO2K7KADVBNZGHLACI2ARTDDYOHIC742L2IUJT2WFAMMIGSQ, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.034644696 (223h40m52.960939386s) [noise=true , address=104.36.82.10:4079 , piece_id=JELVGAGG2NF2OSMKXQAV6DIXDTUFIER5WX4KTCJFQQ5P35EAMOEA, seg_pos={0 0}, upload=179165]: Uploading piece...
Jul 2 16:52:26.148244089 (223h40m53.074538765s) [upload=179165]: Next returned nil segment
Jul 2 16:52:26.148268128 (223h40m53.074562811s) [upload=179165]: Waiting for error group managing segments...
Jul 2 16:52:26.155850465 (223h40m53.082145153s) [noise=true , address=45.91.116.5:4298 , piece_id=JUYTY35XKYPNJYE7X3R7UMINTVMXDMNM6YZSZORJWATTVDBZFOMQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.156432201 (223h40m53.082726892s) [noise=true , address=77.90.31.14:4661 , piece_id=B5E2MZ5JW7Z5AG4GFQ4QGMAFW7ZMKWKRFU276Z6556JFWKYVLXTQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.156943265 (223h40m53.083237956s) [noise=true , address=104.36.82.16:4294 , piece_id=36S2PXQADEAQPWSBMQC4XEAH6H5PDA4AO34JI37VD4KPJOU2WVTA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.158591570 (223h40m53.084886251s) [noise=true , address=104.36.82.3:4767 , piece_id=U4L7ULC2R5OKDDL6BGDSEQUZNTVMQSGXJ56CGVKHNCCHTMIIDXDQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.159458953 (223h40m53.085753634s) [noise=true , address=45.91.116.8:4047 , piece_id=IJZDAO2K7KADVBNZGHLACI2ARTDDYOHIC742L2IUJT2WFAMMIGSQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.161102375 (223h40m53.087397055s) [noise=true , address=45.91.116.12:4226 , piece_id=NFB2MT72D7MLPOY3A6MVO2QQBUFVYJPKN7D5XJQLQCCMYNQ6WJOA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.161556163 (223h40m53.087850838s) [noise=true , address=104.36.82.12:4149 , piece_id=V5265CLKUYZQN5B663WB4CFR3SXR2OA2BT53OAVCLFZJJ2YY6U4A, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.162013197 (223h40m53.088307874s) [noise=true , address=104.36.82.14:4222 , piece_id=3HZK5YRWL2KXKY4E7XASRI2UIPCJA72JSJSTGBLPA4ZBP3MGBEOQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.162459436 (223h40m53.088754112s) [noise=true , address=77.90.31.15:4698 , piece_id=VDVQDBYDQYRMVP7ZUPSEFC2R5UF2LVW45G23JURMYKS5IT6RBQFQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.162920549 (223h40m53.089215241s) [noise=true , address=77.90.31.21:4878 , piece_id=QJARHOYLDCFGYTTCD2EYCH43X3EARFK3Q3C7SCZ33X5T6CTLYDMQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.163747035 (223h40m53.090041713s) [noise=true , address=104.36.82.11:4115 , piece_id=LOK75B4Y2M3DMRJAHNYCVE46DELAAVN4SFP35AWPFESLUJT2V6KQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.167885784 (223h40m53.094180467s) [noise=true , address=104.36.82.6:4874 , piece_id=XVVQN3VX4I2KS4CIXEADBSFYBVYTVEWS5GMXAHE2LWDJGD2A2KAA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.168494969 (223h40m53.094789668s) [noise=true , address=104.36.82.10:4079 , piece_id=JELVGAGG2NF2OSMKXQAV6DIXDTUFIER5WX4KTCJFQQ5P35EAMOEA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: <nil>
Jul 2 16:52:26.168525706 (223h40m53.094820396s) [seg_pos={0 0}, upload=179165]: Segment reached optimal threshold of 13 pieces.
Jul 2 16:52:26.168715378 (223h40m53.09501007s) [noise=true , address=77.90.31.19:4843 , piece_id=J74IERT5EP2VDSXY5PNWWBVTHQEXT4CWGLWP72Y3CQLOTSI6XZDA, seg_pos={0 0}, upload=179165]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: expected piece hash; context canceled; EOF
Jul 2 16:52:26.168766821 (223h40m53.095061509s) [noise=true , address=104.36.82.8:4970 , piece_id=X7KKQJYHVNI5LJDKGYNV5JHHCEGF4QSHFRKMSBLRE5W3W2ITAAHQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe): protocol: expected piece hash; context canceled; EOF
Jul 2 16:52:26.168816209 (223h40m53.095110907s) [noise=true , address=45.91.116.11:4118 , piece_id=CSQO6I5J4BW4NUAOFH3QDF6VEJGACWOSYNTNGXVELK2HLXIMX4YQ, seg_pos={0 0}, upload=179165]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF
Jul 2 16:52:26.168851649 (223h40m53.095146335s) [seg_pos={0 0}, upload=179165]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 2 16:52:26.169058251 (223h40m53.095352944s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 16:52:26.169066272 (223h40m53.095360967s) [upload=179165]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 16:52:26.220481259 (223h40m53.146775936s) [upload=179165]: Done uploading segments...
Jul 2 16:52:26.220500093 (223h40m53.14679477s) [upload=179165]: Upload finished. err: <nil>
Duration: 986.758182ms
core
uplink
us-select-1
|