Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [..............................................X.............] duration:
10.346420901s
FAIL (2025-06-22 03:49:50 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
4 (83h17m33.572378185s) [upload=66665]: Uploading segments...
Jun 22 03:40:58.009397890 (83h17m33.572720878s) [upload=66665]: Got next segment. Inline: false
Jun 22 03:40:58.009451249 (83h17m33.572774249s) [upload=66665]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 22 03:40:58.009457748 (83h17m33.572780755s) [upload=66665]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 22 03:40:58.044661723 (83h17m33.607984722s) [seg_pos={0 0}, upload=66665]: Begin upload segment...
Jun 22 03:40:58.044722729 (83h17m33.608045718s) [seg_pos={0 0}, upload=66665]: Done begin upload segment.
Jun 22 03:40:58.044746618 (83h17m33.608069618s) [noise=true , address=104.36.82.5:5856 , piece_id=ZATJRQE4NARTU5JDY3XMUDSW5E6DTQZPTD5VQV5OIANWC2CSNARA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.044784017 (83h17m33.608107034s) [noise=true , address=79.127.220.97:4363 , piece_id=5HMOMDEQKEOHRF67UVYFYSKIMH2IRLMME6S7XKDNVAMAFHE5EO6Q, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.045240611 (83h17m33.608563622s) [noise=true , address=104.36.82.52:4094 , piece_id=7AXGQ6CXE6H7GTURGOCHG3UBTV3VWLVNIXPNM2NAN7IR2RQCTRIA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.045699560 (83h17m33.609022565s) [noise=true , address=104.36.82.12:6158 , piece_id=VWTJS7XRTMBJFUNAYMZCJQ2EEMT5CVILS4JWMPUK5GIYPLQADAJQ, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.046153432 (83h17m33.609476443s) [noise=true , address=104.36.82.11:6125 , piece_id=NP2722BQYMG5H7RQD3TLWYO23TFPT3HSBZFS6JRYYBFYBZWHEB5A, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.046516869 (83h17m33.609839878s) [noise=true , address=64.49.13.169:5711 , piece_id=ROPI2UEYUINLDKQ4IEVWXTYGSU3PXWGXFI7TC2L675L2KENKWWEQ, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.046897154 (83h17m33.610220195s) [noise=true , address=104.36.82.54:4167 , piece_id=I4FIVN2M4OF7EARPGIY3X72D2DOARBQGIRU2W7KCCOMUEEMZ776Q, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.047287027 (83h17m33.610610033s) [noise=true , address=104.36.82.3:5792 , piece_id=W2WE2U645R765EZI5CGYAOLJWIS6LQIGEFVNZUK4PS2IC6HEUILA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.047564954 (83h17m33.610887959s) [noise=true , address=45.91.116.8:5058 , piece_id=FVHGWLW7PVRET7FKWLXJ6V6MUJKWVTFNOKLGIEQQI72ZXZTW74BA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.047691900 (83h17m33.611014907s) [noise=true , address=104.36.82.55:4217 , piece_id=6DKIHWLLTMCGBT63TAG234VGEPOX7QCTXLOBVQSG56LJS5YYSPPA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.047925386 (83h17m33.611248395s) [noise=true , address=104.36.82.10:6112 , piece_id=56XUDWBP5SSSKOWDY2N2JV6PV63VXTROBOJQLZUUKGLI2ZBE7YMA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.048123429 (83h17m33.611446434s) [noise=true , address=104.36.82.14:6227 , piece_id=B4YO53N2MW6VCC5KHU5LWN6PQNGAJ4ULDPFLHL3OFRP3IBDUSVCA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.048331201 (83h17m33.61165421s) [noise=true , address=104.36.82.9:6016 , piece_id=MLPYRXQYKDFRHRQ4BKELVLTJD6ZMOHM7YSCS5O5HH6525IQOBPGQ, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.048701328 (83h17m33.612024333s) [noise=true , address=104.36.82.42:6832 , piece_id=TRXGPKOEU7HPFSEDKPJN3C6GJOBUZPBM6CNPZWS6ZLOHO2F7SSLA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.049050929 (83h17m33.612373939s) [noise=true , address=104.36.82.56:4252 , piece_id=KHT4ZUAQRO7NJ2IWH6FORUJSH5V2UFKTGYJETRL5SLYRU2D57V4A, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.049392658 (83h17m33.612715669s) [noise=true , address=79.127.219.48:4076 , piece_id=UQM6DRGET6FQFND5HU4CTJDIC4337XX6MK5NJV5XQPOK7AFCPSMQ, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.049754448 (83h17m33.613077451s) [noise=true , address=77.90.31.14:4688 , piece_id=GHO7ZNMRBXPWN34PPDFGKPOOA7L65YVO6AYHFFP5R4XCUGACY3DQ, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.050127175 (83h17m33.613450178s) [noise=true , address=104.36.82.53:4136 , piece_id=QK4OSCIQAD6IB3I4QK4KYU4MTZEOPDJC5GG6HLMCUAXGDNLMSFYA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.091864444 (83h17m33.655187435s) [noise=true , address=79.127.219.48:4076 , piece_id=UQM6DRGET6FQFND5HU4CTJDIC4337XX6MK5NJV5XQPOK7AFCPSMQ, seg_pos={0 0}, upload=66665]: Done uploading piece. err: ecclient: upload failed (node:1QeLZQsubmwYFwTPC9ipEgKJJvFjJ7VrXLC7QncaYoPyyFM4Fn, address:79.127.219.48:4076): protocol: storage node overloaded, request limit: 2000
Jun 22 03:40:58.161661930 (83h17m33.72498492s) [upload=66665]: Next returned nil segment
Jun 22 03:40:58.161681692 (83h17m33.725004695s) [upload=66665]: Waiting for error group managing segments...
Jun 22 03:40:58.190491904 (83h17m33.753814895s) [noise=true , address=104.36.82.11:6125 , piece_id=NP2722BQYMG5H7RQD3TLWYO23TFPT3HSBZFS6JRYYBFYBZWHEB5A, seg_pos={0 0}, upload=66665]: Done uploading piece. err: ecclient: upload failed (node:12vricofwx67FRaxAGLbmgT1TYyKkqfJbxdsyeEPL4R7kg8pKzT, address:104.36.82.11:6125): protocol: storage node overloaded, request limit: 2000
Jun 22 03:40:58.207750455 (83h17m33.77107344s) [noise=true , address=77.90.31.14:4688 , piece_id=GHO7ZNMRBXPWN34PPDFGKPOOA7L65YVO6AYHFFP5R4XCUGACY3DQ, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:40:58.531973385 (83h17m34.095296379s) [noise=true , address=64.49.13.169:5711 , piece_id=ROPI2UEYUINLDKQ4IEVWXTYGSU3PXWGXFI7TC2L675L2KENKWWEQ, seg_pos={0 0}, upload=66665]: Done uploading piece. err: ecclient: upload failed (node:1nxT1Ldr46HPrDRHQ2nrwEBZT5bEVoPAAfBz86ZYwUH13sqwHu, address:64.49.13.169:5711): protocol: storage node overloaded, request limit: 2000
Jun 22 03:40:58.990571568 (83h17m34.553894561s) [noise=true , address=104.36.82.9:6016 , piece_id=MLPYRXQYKDFRHRQ4BKELVLTJD6ZMOHM7YSCS5O5HH6525IQOBPGQ, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:00.325760684 (83h17m35.889083676s) [noise=true , address=104.36.82.14:6227 , piece_id=B4YO53N2MW6VCC5KHU5LWN6PQNGAJ4ULDPFLHL3OFRP3IBDUSVCA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:00.953756837 (83h17m36.517079837s) [noise=true , address=45.91.116.8:5058 , piece_id=FVHGWLW7PVRET7FKWLXJ6V6MUJKWVTFNOKLGIEQQI72ZXZTW74BA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:01.386979509 (83h17m36.950302499s) [noise=true , address=104.36.82.10:6112 , piece_id=56XUDWBP5SSSKOWDY2N2JV6PV63VXTROBOJQLZUUKGLI2ZBE7YMA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:01.689801010 (83h17m37.253124004s) [noise=true , address=104.36.82.3:5792 , piece_id=W2WE2U645R765EZI5CGYAOLJWIS6LQIGEFVNZUK4PS2IC6HEUILA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:01.856352835 (83h17m37.419675823s) [noise=true , address=104.36.82.5:5856 , piece_id=ZATJRQE4NARTU5JDY3XMUDSW5E6DTQZPTD5VQV5OIANWC2CSNARA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:03.717835535 (83h17m39.281158548s) [noise=true , address=104.36.82.42:6832 , piece_id=TRXGPKOEU7HPFSEDKPJN3C6GJOBUZPBM6CNPZWS6ZLOHO2F7SSLA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:03.961658935 (83h17m39.524981945s) [noise=true , address=79.127.220.97:4363 , piece_id=5HMOMDEQKEOHRF67UVYFYSKIMH2IRLMME6S7XKDNVAMAFHE5EO6Q, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:09.244412959 (83h17m44.807735964s) [noise=true , address=104.36.82.56:4252 , piece_id=KHT4ZUAQRO7NJ2IWH6FORUJSH5V2UFKTGYJETRL5SLYRU2D57V4A, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:14.056547620 (83h17m49.619870606s) [noise=true , address=104.36.82.53:4136 , piece_id=QK4OSCIQAD6IB3I4QK4KYU4MTZEOPDJC5GG6HLMCUAXGDNLMSFYA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:19.882456778 (83h17m55.445779771s) [noise=true , address=104.36.82.55:4217 , piece_id=6DKIHWLLTMCGBT63TAG234VGEPOX7QCTXLOBVQSG56LJS5YYSPPA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:28.035171487 (83h18m3.598494478s) [noise=true , address=104.36.82.52:4094 , piece_id=7AXGQ6CXE6H7GTURGOCHG3UBTV3VWLVNIXPNM2NAN7IR2RQCTRIA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:57.717732997 (83h18m33.281055985s) [noise=true , address=104.36.82.12:6158 , piece_id=VWTJS7XRTMBJFUNAYMZCJQ2EEMT5CVILS4JWMPUK5GIYPLQADAJQ, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:57.717762830 (83h18m33.281085829s) [seg_pos={0 0}, upload=66665]: Segment reached optimal threshold of 14 pieces.
Jun 22 03:41:57.718005914 (83h18m33.281328913s) [noise=true , address=104.36.82.54:4167 , piece_id=I4FIVN2M4OF7EARPGIY3X72D2DOARBQGIRU2W7KCCOMUEEMZ776Q, seg_pos={0 0}, upload=66665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hhT46NW5CQ7dWHRxy5MSjyKc3PiQqBifThT7dBmx2sgNvn8GZ): protocol: write tcp 159.223.143.39:56114->104.36.82.54:4167: use of closed network connection; write tcp 159.223.143.39:56114->104.36.82.54:4167: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:56114->104.36.82.54:4167: use of closed network connection
Jun 22 03:41:57.718019794 (83h18m33.281342793s) [seg_pos={0 0}, upload=66665]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jun 22 03:41:57.718095039 (83h18m33.281418038s) [upload=66665]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 22 03:41:57.718100732 (83h18m33.281423731s) [upload=66665]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 22 03:41:57.767433584 (83h18m33.330756651s) [upload=66665]: Done uploading segments...
Jun 22 03:41:57.767452412 (83h18m33.330775402s) [upload=66665]: Upload finished. err: <nil>
Duration: 10.346447183s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............] duration:
13.789180132s
FAIL (2025-06-22 03:49:33 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
4 (83h17m33.572378185s) [upload=66665]: Uploading segments...
Jun 22 03:40:58.009397890 (83h17m33.572720878s) [upload=66665]: Got next segment. Inline: false
Jun 22 03:40:58.009451249 (83h17m33.572774249s) [upload=66665]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 22 03:40:58.009457748 (83h17m33.572780755s) [upload=66665]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 22 03:40:58.044661723 (83h17m33.607984722s) [seg_pos={0 0}, upload=66665]: Begin upload segment...
Jun 22 03:40:58.044722729 (83h17m33.608045718s) [seg_pos={0 0}, upload=66665]: Done begin upload segment.
Jun 22 03:40:58.044746618 (83h17m33.608069618s) [noise=true , address=104.36.82.5:5856 , piece_id=ZATJRQE4NARTU5JDY3XMUDSW5E6DTQZPTD5VQV5OIANWC2CSNARA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.044784017 (83h17m33.608107034s) [noise=true , address=79.127.220.97:4363 , piece_id=5HMOMDEQKEOHRF67UVYFYSKIMH2IRLMME6S7XKDNVAMAFHE5EO6Q, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.045240611 (83h17m33.608563622s) [noise=true , address=104.36.82.52:4094 , piece_id=7AXGQ6CXE6H7GTURGOCHG3UBTV3VWLVNIXPNM2NAN7IR2RQCTRIA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.045699560 (83h17m33.609022565s) [noise=true , address=104.36.82.12:6158 , piece_id=VWTJS7XRTMBJFUNAYMZCJQ2EEMT5CVILS4JWMPUK5GIYPLQADAJQ, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.046153432 (83h17m33.609476443s) [noise=true , address=104.36.82.11:6125 , piece_id=NP2722BQYMG5H7RQD3TLWYO23TFPT3HSBZFS6JRYYBFYBZWHEB5A, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.046516869 (83h17m33.609839878s) [noise=true , address=64.49.13.169:5711 , piece_id=ROPI2UEYUINLDKQ4IEVWXTYGSU3PXWGXFI7TC2L675L2KENKWWEQ, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.046897154 (83h17m33.610220195s) [noise=true , address=104.36.82.54:4167 , piece_id=I4FIVN2M4OF7EARPGIY3X72D2DOARBQGIRU2W7KCCOMUEEMZ776Q, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.047287027 (83h17m33.610610033s) [noise=true , address=104.36.82.3:5792 , piece_id=W2WE2U645R765EZI5CGYAOLJWIS6LQIGEFVNZUK4PS2IC6HEUILA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.047564954 (83h17m33.610887959s) [noise=true , address=45.91.116.8:5058 , piece_id=FVHGWLW7PVRET7FKWLXJ6V6MUJKWVTFNOKLGIEQQI72ZXZTW74BA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.047691900 (83h17m33.611014907s) [noise=true , address=104.36.82.55:4217 , piece_id=6DKIHWLLTMCGBT63TAG234VGEPOX7QCTXLOBVQSG56LJS5YYSPPA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.047925386 (83h17m33.611248395s) [noise=true , address=104.36.82.10:6112 , piece_id=56XUDWBP5SSSKOWDY2N2JV6PV63VXTROBOJQLZUUKGLI2ZBE7YMA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.048123429 (83h17m33.611446434s) [noise=true , address=104.36.82.14:6227 , piece_id=B4YO53N2MW6VCC5KHU5LWN6PQNGAJ4ULDPFLHL3OFRP3IBDUSVCA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.048331201 (83h17m33.61165421s) [noise=true , address=104.36.82.9:6016 , piece_id=MLPYRXQYKDFRHRQ4BKELVLTJD6ZMOHM7YSCS5O5HH6525IQOBPGQ, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.048701328 (83h17m33.612024333s) [noise=true , address=104.36.82.42:6832 , piece_id=TRXGPKOEU7HPFSEDKPJN3C6GJOBUZPBM6CNPZWS6ZLOHO2F7SSLA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.049050929 (83h17m33.612373939s) [noise=true , address=104.36.82.56:4252 , piece_id=KHT4ZUAQRO7NJ2IWH6FORUJSH5V2UFKTGYJETRL5SLYRU2D57V4A, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.049392658 (83h17m33.612715669s) [noise=true , address=79.127.219.48:4076 , piece_id=UQM6DRGET6FQFND5HU4CTJDIC4337XX6MK5NJV5XQPOK7AFCPSMQ, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.049754448 (83h17m33.613077451s) [noise=true , address=77.90.31.14:4688 , piece_id=GHO7ZNMRBXPWN34PPDFGKPOOA7L65YVO6AYHFFP5R4XCUGACY3DQ, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.050127175 (83h17m33.613450178s) [noise=true , address=104.36.82.53:4136 , piece_id=QK4OSCIQAD6IB3I4QK4KYU4MTZEOPDJC5GG6HLMCUAXGDNLMSFYA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.091864444 (83h17m33.655187435s) [noise=true , address=79.127.219.48:4076 , piece_id=UQM6DRGET6FQFND5HU4CTJDIC4337XX6MK5NJV5XQPOK7AFCPSMQ, seg_pos={0 0}, upload=66665]: Done uploading piece. err: ecclient: upload failed (node:1QeLZQsubmwYFwTPC9ipEgKJJvFjJ7VrXLC7QncaYoPyyFM4Fn, address:79.127.219.48:4076): protocol: storage node overloaded, request limit: 2000
Jun 22 03:40:58.161661930 (83h17m33.72498492s) [upload=66665]: Next returned nil segment
Jun 22 03:40:58.161681692 (83h17m33.725004695s) [upload=66665]: Waiting for error group managing segments...
Jun 22 03:40:58.190491904 (83h17m33.753814895s) [noise=true , address=104.36.82.11:6125 , piece_id=NP2722BQYMG5H7RQD3TLWYO23TFPT3HSBZFS6JRYYBFYBZWHEB5A, seg_pos={0 0}, upload=66665]: Done uploading piece. err: ecclient: upload failed (node:12vricofwx67FRaxAGLbmgT1TYyKkqfJbxdsyeEPL4R7kg8pKzT, address:104.36.82.11:6125): protocol: storage node overloaded, request limit: 2000
Jun 22 03:40:58.207750455 (83h17m33.77107344s) [noise=true , address=77.90.31.14:4688 , piece_id=GHO7ZNMRBXPWN34PPDFGKPOOA7L65YVO6AYHFFP5R4XCUGACY3DQ, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:40:58.531973385 (83h17m34.095296379s) [noise=true , address=64.49.13.169:5711 , piece_id=ROPI2UEYUINLDKQ4IEVWXTYGSU3PXWGXFI7TC2L675L2KENKWWEQ, seg_pos={0 0}, upload=66665]: Done uploading piece. err: ecclient: upload failed (node:1nxT1Ldr46HPrDRHQ2nrwEBZT5bEVoPAAfBz86ZYwUH13sqwHu, address:64.49.13.169:5711): protocol: storage node overloaded, request limit: 2000
Jun 22 03:40:58.990571568 (83h17m34.553894561s) [noise=true , address=104.36.82.9:6016 , piece_id=MLPYRXQYKDFRHRQ4BKELVLTJD6ZMOHM7YSCS5O5HH6525IQOBPGQ, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:00.325760684 (83h17m35.889083676s) [noise=true , address=104.36.82.14:6227 , piece_id=B4YO53N2MW6VCC5KHU5LWN6PQNGAJ4ULDPFLHL3OFRP3IBDUSVCA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:00.953756837 (83h17m36.517079837s) [noise=true , address=45.91.116.8:5058 , piece_id=FVHGWLW7PVRET7FKWLXJ6V6MUJKWVTFNOKLGIEQQI72ZXZTW74BA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:01.386979509 (83h17m36.950302499s) [noise=true , address=104.36.82.10:6112 , piece_id=56XUDWBP5SSSKOWDY2N2JV6PV63VXTROBOJQLZUUKGLI2ZBE7YMA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:01.689801010 (83h17m37.253124004s) [noise=true , address=104.36.82.3:5792 , piece_id=W2WE2U645R765EZI5CGYAOLJWIS6LQIGEFVNZUK4PS2IC6HEUILA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:01.856352835 (83h17m37.419675823s) [noise=true , address=104.36.82.5:5856 , piece_id=ZATJRQE4NARTU5JDY3XMUDSW5E6DTQZPTD5VQV5OIANWC2CSNARA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:03.717835535 (83h17m39.281158548s) [noise=true , address=104.36.82.42:6832 , piece_id=TRXGPKOEU7HPFSEDKPJN3C6GJOBUZPBM6CNPZWS6ZLOHO2F7SSLA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:03.961658935 (83h17m39.524981945s) [noise=true , address=79.127.220.97:4363 , piece_id=5HMOMDEQKEOHRF67UVYFYSKIMH2IRLMME6S7XKDNVAMAFHE5EO6Q, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:09.244412959 (83h17m44.807735964s) [noise=true , address=104.36.82.56:4252 , piece_id=KHT4ZUAQRO7NJ2IWH6FORUJSH5V2UFKTGYJETRL5SLYRU2D57V4A, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:14.056547620 (83h17m49.619870606s) [noise=true , address=104.36.82.53:4136 , piece_id=QK4OSCIQAD6IB3I4QK4KYU4MTZEOPDJC5GG6HLMCUAXGDNLMSFYA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:19.882456778 (83h17m55.445779771s) [noise=true , address=104.36.82.55:4217 , piece_id=6DKIHWLLTMCGBT63TAG234VGEPOX7QCTXLOBVQSG56LJS5YYSPPA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:28.035171487 (83h18m3.598494478s) [noise=true , address=104.36.82.52:4094 , piece_id=7AXGQ6CXE6H7GTURGOCHG3UBTV3VWLVNIXPNM2NAN7IR2RQCTRIA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:57.717732997 (83h18m33.281055985s) [noise=true , address=104.36.82.12:6158 , piece_id=VWTJS7XRTMBJFUNAYMZCJQ2EEMT5CVILS4JWMPUK5GIYPLQADAJQ, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:57.717762830 (83h18m33.281085829s) [seg_pos={0 0}, upload=66665]: Segment reached optimal threshold of 14 pieces.
Jun 22 03:41:57.718005914 (83h18m33.281328913s) [noise=true , address=104.36.82.54:4167 , piece_id=I4FIVN2M4OF7EARPGIY3X72D2DOARBQGIRU2W7KCCOMUEEMZ776Q, seg_pos={0 0}, upload=66665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hhT46NW5CQ7dWHRxy5MSjyKc3PiQqBifThT7dBmx2sgNvn8GZ): protocol: write tcp 159.223.143.39:56114->104.36.82.54:4167: use of closed network connection; write tcp 159.223.143.39:56114->104.36.82.54:4167: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:56114->104.36.82.54:4167: use of closed network connection
Jun 22 03:41:57.718019794 (83h18m33.281342793s) [seg_pos={0 0}, upload=66665]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jun 22 03:41:57.718095039 (83h18m33.281418038s) [upload=66665]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 22 03:41:57.718100732 (83h18m33.281423731s) [upload=66665]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 22 03:41:57.767433584 (83h18m33.330756651s) [upload=66665]: Done uploading segments...
Jun 22 03:41:57.767452412 (83h18m33.330775402s) [upload=66665]: Upload finished. err: <nil>
Duration: 13.789208974s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........] duration:
8.080076404s
FAIL (2025-06-22 03:48:45 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
4 (83h17m33.572378185s) [upload=66665]: Uploading segments...
Jun 22 03:40:58.009397890 (83h17m33.572720878s) [upload=66665]: Got next segment. Inline: false
Jun 22 03:40:58.009451249 (83h17m33.572774249s) [upload=66665]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 22 03:40:58.009457748 (83h17m33.572780755s) [upload=66665]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 22 03:40:58.044661723 (83h17m33.607984722s) [seg_pos={0 0}, upload=66665]: Begin upload segment...
Jun 22 03:40:58.044722729 (83h17m33.608045718s) [seg_pos={0 0}, upload=66665]: Done begin upload segment.
Jun 22 03:40:58.044746618 (83h17m33.608069618s) [noise=true , address=104.36.82.5:5856 , piece_id=ZATJRQE4NARTU5JDY3XMUDSW5E6DTQZPTD5VQV5OIANWC2CSNARA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.044784017 (83h17m33.608107034s) [noise=true , address=79.127.220.97:4363 , piece_id=5HMOMDEQKEOHRF67UVYFYSKIMH2IRLMME6S7XKDNVAMAFHE5EO6Q, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.045240611 (83h17m33.608563622s) [noise=true , address=104.36.82.52:4094 , piece_id=7AXGQ6CXE6H7GTURGOCHG3UBTV3VWLVNIXPNM2NAN7IR2RQCTRIA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.045699560 (83h17m33.609022565s) [noise=true , address=104.36.82.12:6158 , piece_id=VWTJS7XRTMBJFUNAYMZCJQ2EEMT5CVILS4JWMPUK5GIYPLQADAJQ, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.046153432 (83h17m33.609476443s) [noise=true , address=104.36.82.11:6125 , piece_id=NP2722BQYMG5H7RQD3TLWYO23TFPT3HSBZFS6JRYYBFYBZWHEB5A, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.046516869 (83h17m33.609839878s) [noise=true , address=64.49.13.169:5711 , piece_id=ROPI2UEYUINLDKQ4IEVWXTYGSU3PXWGXFI7TC2L675L2KENKWWEQ, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.046897154 (83h17m33.610220195s) [noise=true , address=104.36.82.54:4167 , piece_id=I4FIVN2M4OF7EARPGIY3X72D2DOARBQGIRU2W7KCCOMUEEMZ776Q, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.047287027 (83h17m33.610610033s) [noise=true , address=104.36.82.3:5792 , piece_id=W2WE2U645R765EZI5CGYAOLJWIS6LQIGEFVNZUK4PS2IC6HEUILA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.047564954 (83h17m33.610887959s) [noise=true , address=45.91.116.8:5058 , piece_id=FVHGWLW7PVRET7FKWLXJ6V6MUJKWVTFNOKLGIEQQI72ZXZTW74BA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.047691900 (83h17m33.611014907s) [noise=true , address=104.36.82.55:4217 , piece_id=6DKIHWLLTMCGBT63TAG234VGEPOX7QCTXLOBVQSG56LJS5YYSPPA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.047925386 (83h17m33.611248395s) [noise=true , address=104.36.82.10:6112 , piece_id=56XUDWBP5SSSKOWDY2N2JV6PV63VXTROBOJQLZUUKGLI2ZBE7YMA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.048123429 (83h17m33.611446434s) [noise=true , address=104.36.82.14:6227 , piece_id=B4YO53N2MW6VCC5KHU5LWN6PQNGAJ4ULDPFLHL3OFRP3IBDUSVCA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.048331201 (83h17m33.61165421s) [noise=true , address=104.36.82.9:6016 , piece_id=MLPYRXQYKDFRHRQ4BKELVLTJD6ZMOHM7YSCS5O5HH6525IQOBPGQ, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.048701328 (83h17m33.612024333s) [noise=true , address=104.36.82.42:6832 , piece_id=TRXGPKOEU7HPFSEDKPJN3C6GJOBUZPBM6CNPZWS6ZLOHO2F7SSLA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.049050929 (83h17m33.612373939s) [noise=true , address=104.36.82.56:4252 , piece_id=KHT4ZUAQRO7NJ2IWH6FORUJSH5V2UFKTGYJETRL5SLYRU2D57V4A, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.049392658 (83h17m33.612715669s) [noise=true , address=79.127.219.48:4076 , piece_id=UQM6DRGET6FQFND5HU4CTJDIC4337XX6MK5NJV5XQPOK7AFCPSMQ, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.049754448 (83h17m33.613077451s) [noise=true , address=77.90.31.14:4688 , piece_id=GHO7ZNMRBXPWN34PPDFGKPOOA7L65YVO6AYHFFP5R4XCUGACY3DQ, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.050127175 (83h17m33.613450178s) [noise=true , address=104.36.82.53:4136 , piece_id=QK4OSCIQAD6IB3I4QK4KYU4MTZEOPDJC5GG6HLMCUAXGDNLMSFYA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.091864444 (83h17m33.655187435s) [noise=true , address=79.127.219.48:4076 , piece_id=UQM6DRGET6FQFND5HU4CTJDIC4337XX6MK5NJV5XQPOK7AFCPSMQ, seg_pos={0 0}, upload=66665]: Done uploading piece. err: ecclient: upload failed (node:1QeLZQsubmwYFwTPC9ipEgKJJvFjJ7VrXLC7QncaYoPyyFM4Fn, address:79.127.219.48:4076): protocol: storage node overloaded, request limit: 2000
Jun 22 03:40:58.161661930 (83h17m33.72498492s) [upload=66665]: Next returned nil segment
Jun 22 03:40:58.161681692 (83h17m33.725004695s) [upload=66665]: Waiting for error group managing segments...
Jun 22 03:40:58.190491904 (83h17m33.753814895s) [noise=true , address=104.36.82.11:6125 , piece_id=NP2722BQYMG5H7RQD3TLWYO23TFPT3HSBZFS6JRYYBFYBZWHEB5A, seg_pos={0 0}, upload=66665]: Done uploading piece. err: ecclient: upload failed (node:12vricofwx67FRaxAGLbmgT1TYyKkqfJbxdsyeEPL4R7kg8pKzT, address:104.36.82.11:6125): protocol: storage node overloaded, request limit: 2000
Jun 22 03:40:58.207750455 (83h17m33.77107344s) [noise=true , address=77.90.31.14:4688 , piece_id=GHO7ZNMRBXPWN34PPDFGKPOOA7L65YVO6AYHFFP5R4XCUGACY3DQ, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:40:58.531973385 (83h17m34.095296379s) [noise=true , address=64.49.13.169:5711 , piece_id=ROPI2UEYUINLDKQ4IEVWXTYGSU3PXWGXFI7TC2L675L2KENKWWEQ, seg_pos={0 0}, upload=66665]: Done uploading piece. err: ecclient: upload failed (node:1nxT1Ldr46HPrDRHQ2nrwEBZT5bEVoPAAfBz86ZYwUH13sqwHu, address:64.49.13.169:5711): protocol: storage node overloaded, request limit: 2000
Jun 22 03:40:58.990571568 (83h17m34.553894561s) [noise=true , address=104.36.82.9:6016 , piece_id=MLPYRXQYKDFRHRQ4BKELVLTJD6ZMOHM7YSCS5O5HH6525IQOBPGQ, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:00.325760684 (83h17m35.889083676s) [noise=true , address=104.36.82.14:6227 , piece_id=B4YO53N2MW6VCC5KHU5LWN6PQNGAJ4ULDPFLHL3OFRP3IBDUSVCA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:00.953756837 (83h17m36.517079837s) [noise=true , address=45.91.116.8:5058 , piece_id=FVHGWLW7PVRET7FKWLXJ6V6MUJKWVTFNOKLGIEQQI72ZXZTW74BA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:01.386979509 (83h17m36.950302499s) [noise=true , address=104.36.82.10:6112 , piece_id=56XUDWBP5SSSKOWDY2N2JV6PV63VXTROBOJQLZUUKGLI2ZBE7YMA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:01.689801010 (83h17m37.253124004s) [noise=true , address=104.36.82.3:5792 , piece_id=W2WE2U645R765EZI5CGYAOLJWIS6LQIGEFVNZUK4PS2IC6HEUILA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:01.856352835 (83h17m37.419675823s) [noise=true , address=104.36.82.5:5856 , piece_id=ZATJRQE4NARTU5JDY3XMUDSW5E6DTQZPTD5VQV5OIANWC2CSNARA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:03.717835535 (83h17m39.281158548s) [noise=true , address=104.36.82.42:6832 , piece_id=TRXGPKOEU7HPFSEDKPJN3C6GJOBUZPBM6CNPZWS6ZLOHO2F7SSLA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:03.961658935 (83h17m39.524981945s) [noise=true , address=79.127.220.97:4363 , piece_id=5HMOMDEQKEOHRF67UVYFYSKIMH2IRLMME6S7XKDNVAMAFHE5EO6Q, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:09.244412959 (83h17m44.807735964s) [noise=true , address=104.36.82.56:4252 , piece_id=KHT4ZUAQRO7NJ2IWH6FORUJSH5V2UFKTGYJETRL5SLYRU2D57V4A, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:14.056547620 (83h17m49.619870606s) [noise=true , address=104.36.82.53:4136 , piece_id=QK4OSCIQAD6IB3I4QK4KYU4MTZEOPDJC5GG6HLMCUAXGDNLMSFYA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:19.882456778 (83h17m55.445779771s) [noise=true , address=104.36.82.55:4217 , piece_id=6DKIHWLLTMCGBT63TAG234VGEPOX7QCTXLOBVQSG56LJS5YYSPPA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:28.035171487 (83h18m3.598494478s) [noise=true , address=104.36.82.52:4094 , piece_id=7AXGQ6CXE6H7GTURGOCHG3UBTV3VWLVNIXPNM2NAN7IR2RQCTRIA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:57.717732997 (83h18m33.281055985s) [noise=true , address=104.36.82.12:6158 , piece_id=VWTJS7XRTMBJFUNAYMZCJQ2EEMT5CVILS4JWMPUK5GIYPLQADAJQ, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:57.717762830 (83h18m33.281085829s) [seg_pos={0 0}, upload=66665]: Segment reached optimal threshold of 14 pieces.
Jun 22 03:41:57.718005914 (83h18m33.281328913s) [noise=true , address=104.36.82.54:4167 , piece_id=I4FIVN2M4OF7EARPGIY3X72D2DOARBQGIRU2W7KCCOMUEEMZ776Q, seg_pos={0 0}, upload=66665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hhT46NW5CQ7dWHRxy5MSjyKc3PiQqBifThT7dBmx2sgNvn8GZ): protocol: write tcp 159.223.143.39:56114->104.36.82.54:4167: use of closed network connection; write tcp 159.223.143.39:56114->104.36.82.54:4167: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:56114->104.36.82.54:4167: use of closed network connection
Jun 22 03:41:57.718019794 (83h18m33.281342793s) [seg_pos={0 0}, upload=66665]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jun 22 03:41:57.718095039 (83h18m33.281418038s) [upload=66665]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 22 03:41:57.718100732 (83h18m33.281423731s) [upload=66665]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 22 03:41:57.767433584 (83h18m33.330756651s) [upload=66665]: Done uploading segments...
Jun 22 03:41:57.767452412 (83h18m33.330775402s) [upload=66665]: Upload finished. err: <nil>
Duration: 8.08010303s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........] duration:
15.925608558s
FAIL (2025-06-22 03:47:59 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
4 (83h17m33.572378185s) [upload=66665]: Uploading segments...
Jun 22 03:40:58.009397890 (83h17m33.572720878s) [upload=66665]: Got next segment. Inline: false
Jun 22 03:40:58.009451249 (83h17m33.572774249s) [upload=66665]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 22 03:40:58.009457748 (83h17m33.572780755s) [upload=66665]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 22 03:40:58.044661723 (83h17m33.607984722s) [seg_pos={0 0}, upload=66665]: Begin upload segment...
Jun 22 03:40:58.044722729 (83h17m33.608045718s) [seg_pos={0 0}, upload=66665]: Done begin upload segment.
Jun 22 03:40:58.044746618 (83h17m33.608069618s) [noise=true , address=104.36.82.5:5856 , piece_id=ZATJRQE4NARTU5JDY3XMUDSW5E6DTQZPTD5VQV5OIANWC2CSNARA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.044784017 (83h17m33.608107034s) [noise=true , address=79.127.220.97:4363 , piece_id=5HMOMDEQKEOHRF67UVYFYSKIMH2IRLMME6S7XKDNVAMAFHE5EO6Q, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.045240611 (83h17m33.608563622s) [noise=true , address=104.36.82.52:4094 , piece_id=7AXGQ6CXE6H7GTURGOCHG3UBTV3VWLVNIXPNM2NAN7IR2RQCTRIA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.045699560 (83h17m33.609022565s) [noise=true , address=104.36.82.12:6158 , piece_id=VWTJS7XRTMBJFUNAYMZCJQ2EEMT5CVILS4JWMPUK5GIYPLQADAJQ, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.046153432 (83h17m33.609476443s) [noise=true , address=104.36.82.11:6125 , piece_id=NP2722BQYMG5H7RQD3TLWYO23TFPT3HSBZFS6JRYYBFYBZWHEB5A, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.046516869 (83h17m33.609839878s) [noise=true , address=64.49.13.169:5711 , piece_id=ROPI2UEYUINLDKQ4IEVWXTYGSU3PXWGXFI7TC2L675L2KENKWWEQ, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.046897154 (83h17m33.610220195s) [noise=true , address=104.36.82.54:4167 , piece_id=I4FIVN2M4OF7EARPGIY3X72D2DOARBQGIRU2W7KCCOMUEEMZ776Q, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.047287027 (83h17m33.610610033s) [noise=true , address=104.36.82.3:5792 , piece_id=W2WE2U645R765EZI5CGYAOLJWIS6LQIGEFVNZUK4PS2IC6HEUILA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.047564954 (83h17m33.610887959s) [noise=true , address=45.91.116.8:5058 , piece_id=FVHGWLW7PVRET7FKWLXJ6V6MUJKWVTFNOKLGIEQQI72ZXZTW74BA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.047691900 (83h17m33.611014907s) [noise=true , address=104.36.82.55:4217 , piece_id=6DKIHWLLTMCGBT63TAG234VGEPOX7QCTXLOBVQSG56LJS5YYSPPA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.047925386 (83h17m33.611248395s) [noise=true , address=104.36.82.10:6112 , piece_id=56XUDWBP5SSSKOWDY2N2JV6PV63VXTROBOJQLZUUKGLI2ZBE7YMA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.048123429 (83h17m33.611446434s) [noise=true , address=104.36.82.14:6227 , piece_id=B4YO53N2MW6VCC5KHU5LWN6PQNGAJ4ULDPFLHL3OFRP3IBDUSVCA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.048331201 (83h17m33.61165421s) [noise=true , address=104.36.82.9:6016 , piece_id=MLPYRXQYKDFRHRQ4BKELVLTJD6ZMOHM7YSCS5O5HH6525IQOBPGQ, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.048701328 (83h17m33.612024333s) [noise=true , address=104.36.82.42:6832 , piece_id=TRXGPKOEU7HPFSEDKPJN3C6GJOBUZPBM6CNPZWS6ZLOHO2F7SSLA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.049050929 (83h17m33.612373939s) [noise=true , address=104.36.82.56:4252 , piece_id=KHT4ZUAQRO7NJ2IWH6FORUJSH5V2UFKTGYJETRL5SLYRU2D57V4A, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.049392658 (83h17m33.612715669s) [noise=true , address=79.127.219.48:4076 , piece_id=UQM6DRGET6FQFND5HU4CTJDIC4337XX6MK5NJV5XQPOK7AFCPSMQ, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.049754448 (83h17m33.613077451s) [noise=true , address=77.90.31.14:4688 , piece_id=GHO7ZNMRBXPWN34PPDFGKPOOA7L65YVO6AYHFFP5R4XCUGACY3DQ, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.050127175 (83h17m33.613450178s) [noise=true , address=104.36.82.53:4136 , piece_id=QK4OSCIQAD6IB3I4QK4KYU4MTZEOPDJC5GG6HLMCUAXGDNLMSFYA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.091864444 (83h17m33.655187435s) [noise=true , address=79.127.219.48:4076 , piece_id=UQM6DRGET6FQFND5HU4CTJDIC4337XX6MK5NJV5XQPOK7AFCPSMQ, seg_pos={0 0}, upload=66665]: Done uploading piece. err: ecclient: upload failed (node:1QeLZQsubmwYFwTPC9ipEgKJJvFjJ7VrXLC7QncaYoPyyFM4Fn, address:79.127.219.48:4076): protocol: storage node overloaded, request limit: 2000
Jun 22 03:40:58.161661930 (83h17m33.72498492s) [upload=66665]: Next returned nil segment
Jun 22 03:40:58.161681692 (83h17m33.725004695s) [upload=66665]: Waiting for error group managing segments...
Jun 22 03:40:58.190491904 (83h17m33.753814895s) [noise=true , address=104.36.82.11:6125 , piece_id=NP2722BQYMG5H7RQD3TLWYO23TFPT3HSBZFS6JRYYBFYBZWHEB5A, seg_pos={0 0}, upload=66665]: Done uploading piece. err: ecclient: upload failed (node:12vricofwx67FRaxAGLbmgT1TYyKkqfJbxdsyeEPL4R7kg8pKzT, address:104.36.82.11:6125): protocol: storage node overloaded, request limit: 2000
Jun 22 03:40:58.207750455 (83h17m33.77107344s) [noise=true , address=77.90.31.14:4688 , piece_id=GHO7ZNMRBXPWN34PPDFGKPOOA7L65YVO6AYHFFP5R4XCUGACY3DQ, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:40:58.531973385 (83h17m34.095296379s) [noise=true , address=64.49.13.169:5711 , piece_id=ROPI2UEYUINLDKQ4IEVWXTYGSU3PXWGXFI7TC2L675L2KENKWWEQ, seg_pos={0 0}, upload=66665]: Done uploading piece. err: ecclient: upload failed (node:1nxT1Ldr46HPrDRHQ2nrwEBZT5bEVoPAAfBz86ZYwUH13sqwHu, address:64.49.13.169:5711): protocol: storage node overloaded, request limit: 2000
Jun 22 03:40:58.990571568 (83h17m34.553894561s) [noise=true , address=104.36.82.9:6016 , piece_id=MLPYRXQYKDFRHRQ4BKELVLTJD6ZMOHM7YSCS5O5HH6525IQOBPGQ, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:00.325760684 (83h17m35.889083676s) [noise=true , address=104.36.82.14:6227 , piece_id=B4YO53N2MW6VCC5KHU5LWN6PQNGAJ4ULDPFLHL3OFRP3IBDUSVCA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:00.953756837 (83h17m36.517079837s) [noise=true , address=45.91.116.8:5058 , piece_id=FVHGWLW7PVRET7FKWLXJ6V6MUJKWVTFNOKLGIEQQI72ZXZTW74BA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:01.386979509 (83h17m36.950302499s) [noise=true , address=104.36.82.10:6112 , piece_id=56XUDWBP5SSSKOWDY2N2JV6PV63VXTROBOJQLZUUKGLI2ZBE7YMA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:01.689801010 (83h17m37.253124004s) [noise=true , address=104.36.82.3:5792 , piece_id=W2WE2U645R765EZI5CGYAOLJWIS6LQIGEFVNZUK4PS2IC6HEUILA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:01.856352835 (83h17m37.419675823s) [noise=true , address=104.36.82.5:5856 , piece_id=ZATJRQE4NARTU5JDY3XMUDSW5E6DTQZPTD5VQV5OIANWC2CSNARA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:03.717835535 (83h17m39.281158548s) [noise=true , address=104.36.82.42:6832 , piece_id=TRXGPKOEU7HPFSEDKPJN3C6GJOBUZPBM6CNPZWS6ZLOHO2F7SSLA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:03.961658935 (83h17m39.524981945s) [noise=true , address=79.127.220.97:4363 , piece_id=5HMOMDEQKEOHRF67UVYFYSKIMH2IRLMME6S7XKDNVAMAFHE5EO6Q, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:09.244412959 (83h17m44.807735964s) [noise=true , address=104.36.82.56:4252 , piece_id=KHT4ZUAQRO7NJ2IWH6FORUJSH5V2UFKTGYJETRL5SLYRU2D57V4A, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:14.056547620 (83h17m49.619870606s) [noise=true , address=104.36.82.53:4136 , piece_id=QK4OSCIQAD6IB3I4QK4KYU4MTZEOPDJC5GG6HLMCUAXGDNLMSFYA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:19.882456778 (83h17m55.445779771s) [noise=true , address=104.36.82.55:4217 , piece_id=6DKIHWLLTMCGBT63TAG234VGEPOX7QCTXLOBVQSG56LJS5YYSPPA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:28.035171487 (83h18m3.598494478s) [noise=true , address=104.36.82.52:4094 , piece_id=7AXGQ6CXE6H7GTURGOCHG3UBTV3VWLVNIXPNM2NAN7IR2RQCTRIA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:57.717732997 (83h18m33.281055985s) [noise=true , address=104.36.82.12:6158 , piece_id=VWTJS7XRTMBJFUNAYMZCJQ2EEMT5CVILS4JWMPUK5GIYPLQADAJQ, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:57.717762830 (83h18m33.281085829s) [seg_pos={0 0}, upload=66665]: Segment reached optimal threshold of 14 pieces.
Jun 22 03:41:57.718005914 (83h18m33.281328913s) [noise=true , address=104.36.82.54:4167 , piece_id=I4FIVN2M4OF7EARPGIY3X72D2DOARBQGIRU2W7KCCOMUEEMZ776Q, seg_pos={0 0}, upload=66665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hhT46NW5CQ7dWHRxy5MSjyKc3PiQqBifThT7dBmx2sgNvn8GZ): protocol: write tcp 159.223.143.39:56114->104.36.82.54:4167: use of closed network connection; write tcp 159.223.143.39:56114->104.36.82.54:4167: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:56114->104.36.82.54:4167: use of closed network connection
Jun 22 03:41:57.718019794 (83h18m33.281342793s) [seg_pos={0 0}, upload=66665]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jun 22 03:41:57.718095039 (83h18m33.281418038s) [upload=66665]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 22 03:41:57.718100732 (83h18m33.281423731s) [upload=66665]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 22 03:41:57.767433584 (83h18m33.330756651s) [upload=66665]: Done uploading segments...
Jun 22 03:41:57.767452412 (83h18m33.330775402s) [upload=66665]: Upload finished. err: <nil>
Duration: 15.925652178s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
16.059348697s
FAIL (2025-06-22 03:47:28 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
4 (83h17m33.572378185s) [upload=66665]: Uploading segments...
Jun 22 03:40:58.009397890 (83h17m33.572720878s) [upload=66665]: Got next segment. Inline: false
Jun 22 03:40:58.009451249 (83h17m33.572774249s) [upload=66665]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 22 03:40:58.009457748 (83h17m33.572780755s) [upload=66665]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 22 03:40:58.044661723 (83h17m33.607984722s) [seg_pos={0 0}, upload=66665]: Begin upload segment...
Jun 22 03:40:58.044722729 (83h17m33.608045718s) [seg_pos={0 0}, upload=66665]: Done begin upload segment.
Jun 22 03:40:58.044746618 (83h17m33.608069618s) [noise=true , address=104.36.82.5:5856 , piece_id=ZATJRQE4NARTU5JDY3XMUDSW5E6DTQZPTD5VQV5OIANWC2CSNARA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.044784017 (83h17m33.608107034s) [noise=true , address=79.127.220.97:4363 , piece_id=5HMOMDEQKEOHRF67UVYFYSKIMH2IRLMME6S7XKDNVAMAFHE5EO6Q, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.045240611 (83h17m33.608563622s) [noise=true , address=104.36.82.52:4094 , piece_id=7AXGQ6CXE6H7GTURGOCHG3UBTV3VWLVNIXPNM2NAN7IR2RQCTRIA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.045699560 (83h17m33.609022565s) [noise=true , address=104.36.82.12:6158 , piece_id=VWTJS7XRTMBJFUNAYMZCJQ2EEMT5CVILS4JWMPUK5GIYPLQADAJQ, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.046153432 (83h17m33.609476443s) [noise=true , address=104.36.82.11:6125 , piece_id=NP2722BQYMG5H7RQD3TLWYO23TFPT3HSBZFS6JRYYBFYBZWHEB5A, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.046516869 (83h17m33.609839878s) [noise=true , address=64.49.13.169:5711 , piece_id=ROPI2UEYUINLDKQ4IEVWXTYGSU3PXWGXFI7TC2L675L2KENKWWEQ, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.046897154 (83h17m33.610220195s) [noise=true , address=104.36.82.54:4167 , piece_id=I4FIVN2M4OF7EARPGIY3X72D2DOARBQGIRU2W7KCCOMUEEMZ776Q, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.047287027 (83h17m33.610610033s) [noise=true , address=104.36.82.3:5792 , piece_id=W2WE2U645R765EZI5CGYAOLJWIS6LQIGEFVNZUK4PS2IC6HEUILA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.047564954 (83h17m33.610887959s) [noise=true , address=45.91.116.8:5058 , piece_id=FVHGWLW7PVRET7FKWLXJ6V6MUJKWVTFNOKLGIEQQI72ZXZTW74BA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.047691900 (83h17m33.611014907s) [noise=true , address=104.36.82.55:4217 , piece_id=6DKIHWLLTMCGBT63TAG234VGEPOX7QCTXLOBVQSG56LJS5YYSPPA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.047925386 (83h17m33.611248395s) [noise=true , address=104.36.82.10:6112 , piece_id=56XUDWBP5SSSKOWDY2N2JV6PV63VXTROBOJQLZUUKGLI2ZBE7YMA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.048123429 (83h17m33.611446434s) [noise=true , address=104.36.82.14:6227 , piece_id=B4YO53N2MW6VCC5KHU5LWN6PQNGAJ4ULDPFLHL3OFRP3IBDUSVCA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.048331201 (83h17m33.61165421s) [noise=true , address=104.36.82.9:6016 , piece_id=MLPYRXQYKDFRHRQ4BKELVLTJD6ZMOHM7YSCS5O5HH6525IQOBPGQ, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.048701328 (83h17m33.612024333s) [noise=true , address=104.36.82.42:6832 , piece_id=TRXGPKOEU7HPFSEDKPJN3C6GJOBUZPBM6CNPZWS6ZLOHO2F7SSLA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.049050929 (83h17m33.612373939s) [noise=true , address=104.36.82.56:4252 , piece_id=KHT4ZUAQRO7NJ2IWH6FORUJSH5V2UFKTGYJETRL5SLYRU2D57V4A, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.049392658 (83h17m33.612715669s) [noise=true , address=79.127.219.48:4076 , piece_id=UQM6DRGET6FQFND5HU4CTJDIC4337XX6MK5NJV5XQPOK7AFCPSMQ, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.049754448 (83h17m33.613077451s) [noise=true , address=77.90.31.14:4688 , piece_id=GHO7ZNMRBXPWN34PPDFGKPOOA7L65YVO6AYHFFP5R4XCUGACY3DQ, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.050127175 (83h17m33.613450178s) [noise=true , address=104.36.82.53:4136 , piece_id=QK4OSCIQAD6IB3I4QK4KYU4MTZEOPDJC5GG6HLMCUAXGDNLMSFYA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.091864444 (83h17m33.655187435s) [noise=true , address=79.127.219.48:4076 , piece_id=UQM6DRGET6FQFND5HU4CTJDIC4337XX6MK5NJV5XQPOK7AFCPSMQ, seg_pos={0 0}, upload=66665]: Done uploading piece. err: ecclient: upload failed (node:1QeLZQsubmwYFwTPC9ipEgKJJvFjJ7VrXLC7QncaYoPyyFM4Fn, address:79.127.219.48:4076): protocol: storage node overloaded, request limit: 2000
Jun 22 03:40:58.161661930 (83h17m33.72498492s) [upload=66665]: Next returned nil segment
Jun 22 03:40:58.161681692 (83h17m33.725004695s) [upload=66665]: Waiting for error group managing segments...
Jun 22 03:40:58.190491904 (83h17m33.753814895s) [noise=true , address=104.36.82.11:6125 , piece_id=NP2722BQYMG5H7RQD3TLWYO23TFPT3HSBZFS6JRYYBFYBZWHEB5A, seg_pos={0 0}, upload=66665]: Done uploading piece. err: ecclient: upload failed (node:12vricofwx67FRaxAGLbmgT1TYyKkqfJbxdsyeEPL4R7kg8pKzT, address:104.36.82.11:6125): protocol: storage node overloaded, request limit: 2000
Jun 22 03:40:58.207750455 (83h17m33.77107344s) [noise=true , address=77.90.31.14:4688 , piece_id=GHO7ZNMRBXPWN34PPDFGKPOOA7L65YVO6AYHFFP5R4XCUGACY3DQ, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:40:58.531973385 (83h17m34.095296379s) [noise=true , address=64.49.13.169:5711 , piece_id=ROPI2UEYUINLDKQ4IEVWXTYGSU3PXWGXFI7TC2L675L2KENKWWEQ, seg_pos={0 0}, upload=66665]: Done uploading piece. err: ecclient: upload failed (node:1nxT1Ldr46HPrDRHQ2nrwEBZT5bEVoPAAfBz86ZYwUH13sqwHu, address:64.49.13.169:5711): protocol: storage node overloaded, request limit: 2000
Jun 22 03:40:58.990571568 (83h17m34.553894561s) [noise=true , address=104.36.82.9:6016 , piece_id=MLPYRXQYKDFRHRQ4BKELVLTJD6ZMOHM7YSCS5O5HH6525IQOBPGQ, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:00.325760684 (83h17m35.889083676s) [noise=true , address=104.36.82.14:6227 , piece_id=B4YO53N2MW6VCC5KHU5LWN6PQNGAJ4ULDPFLHL3OFRP3IBDUSVCA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:00.953756837 (83h17m36.517079837s) [noise=true , address=45.91.116.8:5058 , piece_id=FVHGWLW7PVRET7FKWLXJ6V6MUJKWVTFNOKLGIEQQI72ZXZTW74BA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:01.386979509 (83h17m36.950302499s) [noise=true , address=104.36.82.10:6112 , piece_id=56XUDWBP5SSSKOWDY2N2JV6PV63VXTROBOJQLZUUKGLI2ZBE7YMA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:01.689801010 (83h17m37.253124004s) [noise=true , address=104.36.82.3:5792 , piece_id=W2WE2U645R765EZI5CGYAOLJWIS6LQIGEFVNZUK4PS2IC6HEUILA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:01.856352835 (83h17m37.419675823s) [noise=true , address=104.36.82.5:5856 , piece_id=ZATJRQE4NARTU5JDY3XMUDSW5E6DTQZPTD5VQV5OIANWC2CSNARA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:03.717835535 (83h17m39.281158548s) [noise=true , address=104.36.82.42:6832 , piece_id=TRXGPKOEU7HPFSEDKPJN3C6GJOBUZPBM6CNPZWS6ZLOHO2F7SSLA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:03.961658935 (83h17m39.524981945s) [noise=true , address=79.127.220.97:4363 , piece_id=5HMOMDEQKEOHRF67UVYFYSKIMH2IRLMME6S7XKDNVAMAFHE5EO6Q, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:09.244412959 (83h17m44.807735964s) [noise=true , address=104.36.82.56:4252 , piece_id=KHT4ZUAQRO7NJ2IWH6FORUJSH5V2UFKTGYJETRL5SLYRU2D57V4A, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:14.056547620 (83h17m49.619870606s) [noise=true , address=104.36.82.53:4136 , piece_id=QK4OSCIQAD6IB3I4QK4KYU4MTZEOPDJC5GG6HLMCUAXGDNLMSFYA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:19.882456778 (83h17m55.445779771s) [noise=true , address=104.36.82.55:4217 , piece_id=6DKIHWLLTMCGBT63TAG234VGEPOX7QCTXLOBVQSG56LJS5YYSPPA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:28.035171487 (83h18m3.598494478s) [noise=true , address=104.36.82.52:4094 , piece_id=7AXGQ6CXE6H7GTURGOCHG3UBTV3VWLVNIXPNM2NAN7IR2RQCTRIA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:57.717732997 (83h18m33.281055985s) [noise=true , address=104.36.82.12:6158 , piece_id=VWTJS7XRTMBJFUNAYMZCJQ2EEMT5CVILS4JWMPUK5GIYPLQADAJQ, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:57.717762830 (83h18m33.281085829s) [seg_pos={0 0}, upload=66665]: Segment reached optimal threshold of 14 pieces.
Jun 22 03:41:57.718005914 (83h18m33.281328913s) [noise=true , address=104.36.82.54:4167 , piece_id=I4FIVN2M4OF7EARPGIY3X72D2DOARBQGIRU2W7KCCOMUEEMZ776Q, seg_pos={0 0}, upload=66665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hhT46NW5CQ7dWHRxy5MSjyKc3PiQqBifThT7dBmx2sgNvn8GZ): protocol: write tcp 159.223.143.39:56114->104.36.82.54:4167: use of closed network connection; write tcp 159.223.143.39:56114->104.36.82.54:4167: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:56114->104.36.82.54:4167: use of closed network connection
Jun 22 03:41:57.718019794 (83h18m33.281342793s) [seg_pos={0 0}, upload=66665]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jun 22 03:41:57.718095039 (83h18m33.281418038s) [upload=66665]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 22 03:41:57.718100732 (83h18m33.281423731s) [upload=66665]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 22 03:41:57.767433584 (83h18m33.330756651s) [upload=66665]: Done uploading segments...
Jun 22 03:41:57.767452412 (83h18m33.330775402s) [upload=66665]: Upload finished. err: <nil>
Duration: 16.059375026s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........] duration:
11.538479325s
FAIL (2025-06-22 03:46:36 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
4 (83h17m33.572378185s) [upload=66665]: Uploading segments...
Jun 22 03:40:58.009397890 (83h17m33.572720878s) [upload=66665]: Got next segment. Inline: false
Jun 22 03:40:58.009451249 (83h17m33.572774249s) [upload=66665]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 22 03:40:58.009457748 (83h17m33.572780755s) [upload=66665]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 22 03:40:58.044661723 (83h17m33.607984722s) [seg_pos={0 0}, upload=66665]: Begin upload segment...
Jun 22 03:40:58.044722729 (83h17m33.608045718s) [seg_pos={0 0}, upload=66665]: Done begin upload segment.
Jun 22 03:40:58.044746618 (83h17m33.608069618s) [noise=true , address=104.36.82.5:5856 , piece_id=ZATJRQE4NARTU5JDY3XMUDSW5E6DTQZPTD5VQV5OIANWC2CSNARA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.044784017 (83h17m33.608107034s) [noise=true , address=79.127.220.97:4363 , piece_id=5HMOMDEQKEOHRF67UVYFYSKIMH2IRLMME6S7XKDNVAMAFHE5EO6Q, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.045240611 (83h17m33.608563622s) [noise=true , address=104.36.82.52:4094 , piece_id=7AXGQ6CXE6H7GTURGOCHG3UBTV3VWLVNIXPNM2NAN7IR2RQCTRIA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.045699560 (83h17m33.609022565s) [noise=true , address=104.36.82.12:6158 , piece_id=VWTJS7XRTMBJFUNAYMZCJQ2EEMT5CVILS4JWMPUK5GIYPLQADAJQ, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.046153432 (83h17m33.609476443s) [noise=true , address=104.36.82.11:6125 , piece_id=NP2722BQYMG5H7RQD3TLWYO23TFPT3HSBZFS6JRYYBFYBZWHEB5A, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.046516869 (83h17m33.609839878s) [noise=true , address=64.49.13.169:5711 , piece_id=ROPI2UEYUINLDKQ4IEVWXTYGSU3PXWGXFI7TC2L675L2KENKWWEQ, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.046897154 (83h17m33.610220195s) [noise=true , address=104.36.82.54:4167 , piece_id=I4FIVN2M4OF7EARPGIY3X72D2DOARBQGIRU2W7KCCOMUEEMZ776Q, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.047287027 (83h17m33.610610033s) [noise=true , address=104.36.82.3:5792 , piece_id=W2WE2U645R765EZI5CGYAOLJWIS6LQIGEFVNZUK4PS2IC6HEUILA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.047564954 (83h17m33.610887959s) [noise=true , address=45.91.116.8:5058 , piece_id=FVHGWLW7PVRET7FKWLXJ6V6MUJKWVTFNOKLGIEQQI72ZXZTW74BA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.047691900 (83h17m33.611014907s) [noise=true , address=104.36.82.55:4217 , piece_id=6DKIHWLLTMCGBT63TAG234VGEPOX7QCTXLOBVQSG56LJS5YYSPPA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.047925386 (83h17m33.611248395s) [noise=true , address=104.36.82.10:6112 , piece_id=56XUDWBP5SSSKOWDY2N2JV6PV63VXTROBOJQLZUUKGLI2ZBE7YMA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.048123429 (83h17m33.611446434s) [noise=true , address=104.36.82.14:6227 , piece_id=B4YO53N2MW6VCC5KHU5LWN6PQNGAJ4ULDPFLHL3OFRP3IBDUSVCA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.048331201 (83h17m33.61165421s) [noise=true , address=104.36.82.9:6016 , piece_id=MLPYRXQYKDFRHRQ4BKELVLTJD6ZMOHM7YSCS5O5HH6525IQOBPGQ, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.048701328 (83h17m33.612024333s) [noise=true , address=104.36.82.42:6832 , piece_id=TRXGPKOEU7HPFSEDKPJN3C6GJOBUZPBM6CNPZWS6ZLOHO2F7SSLA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.049050929 (83h17m33.612373939s) [noise=true , address=104.36.82.56:4252 , piece_id=KHT4ZUAQRO7NJ2IWH6FORUJSH5V2UFKTGYJETRL5SLYRU2D57V4A, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.049392658 (83h17m33.612715669s) [noise=true , address=79.127.219.48:4076 , piece_id=UQM6DRGET6FQFND5HU4CTJDIC4337XX6MK5NJV5XQPOK7AFCPSMQ, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.049754448 (83h17m33.613077451s) [noise=true , address=77.90.31.14:4688 , piece_id=GHO7ZNMRBXPWN34PPDFGKPOOA7L65YVO6AYHFFP5R4XCUGACY3DQ, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.050127175 (83h17m33.613450178s) [noise=true , address=104.36.82.53:4136 , piece_id=QK4OSCIQAD6IB3I4QK4KYU4MTZEOPDJC5GG6HLMCUAXGDNLMSFYA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.091864444 (83h17m33.655187435s) [noise=true , address=79.127.219.48:4076 , piece_id=UQM6DRGET6FQFND5HU4CTJDIC4337XX6MK5NJV5XQPOK7AFCPSMQ, seg_pos={0 0}, upload=66665]: Done uploading piece. err: ecclient: upload failed (node:1QeLZQsubmwYFwTPC9ipEgKJJvFjJ7VrXLC7QncaYoPyyFM4Fn, address:79.127.219.48:4076): protocol: storage node overloaded, request limit: 2000
Jun 22 03:40:58.161661930 (83h17m33.72498492s) [upload=66665]: Next returned nil segment
Jun 22 03:40:58.161681692 (83h17m33.725004695s) [upload=66665]: Waiting for error group managing segments...
Jun 22 03:40:58.190491904 (83h17m33.753814895s) [noise=true , address=104.36.82.11:6125 , piece_id=NP2722BQYMG5H7RQD3TLWYO23TFPT3HSBZFS6JRYYBFYBZWHEB5A, seg_pos={0 0}, upload=66665]: Done uploading piece. err: ecclient: upload failed (node:12vricofwx67FRaxAGLbmgT1TYyKkqfJbxdsyeEPL4R7kg8pKzT, address:104.36.82.11:6125): protocol: storage node overloaded, request limit: 2000
Jun 22 03:40:58.207750455 (83h17m33.77107344s) [noise=true , address=77.90.31.14:4688 , piece_id=GHO7ZNMRBXPWN34PPDFGKPOOA7L65YVO6AYHFFP5R4XCUGACY3DQ, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:40:58.531973385 (83h17m34.095296379s) [noise=true , address=64.49.13.169:5711 , piece_id=ROPI2UEYUINLDKQ4IEVWXTYGSU3PXWGXFI7TC2L675L2KENKWWEQ, seg_pos={0 0}, upload=66665]: Done uploading piece. err: ecclient: upload failed (node:1nxT1Ldr46HPrDRHQ2nrwEBZT5bEVoPAAfBz86ZYwUH13sqwHu, address:64.49.13.169:5711): protocol: storage node overloaded, request limit: 2000
Jun 22 03:40:58.990571568 (83h17m34.553894561s) [noise=true , address=104.36.82.9:6016 , piece_id=MLPYRXQYKDFRHRQ4BKELVLTJD6ZMOHM7YSCS5O5HH6525IQOBPGQ, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:00.325760684 (83h17m35.889083676s) [noise=true , address=104.36.82.14:6227 , piece_id=B4YO53N2MW6VCC5KHU5LWN6PQNGAJ4ULDPFLHL3OFRP3IBDUSVCA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:00.953756837 (83h17m36.517079837s) [noise=true , address=45.91.116.8:5058 , piece_id=FVHGWLW7PVRET7FKWLXJ6V6MUJKWVTFNOKLGIEQQI72ZXZTW74BA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:01.386979509 (83h17m36.950302499s) [noise=true , address=104.36.82.10:6112 , piece_id=56XUDWBP5SSSKOWDY2N2JV6PV63VXTROBOJQLZUUKGLI2ZBE7YMA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:01.689801010 (83h17m37.253124004s) [noise=true , address=104.36.82.3:5792 , piece_id=W2WE2U645R765EZI5CGYAOLJWIS6LQIGEFVNZUK4PS2IC6HEUILA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:01.856352835 (83h17m37.419675823s) [noise=true , address=104.36.82.5:5856 , piece_id=ZATJRQE4NARTU5JDY3XMUDSW5E6DTQZPTD5VQV5OIANWC2CSNARA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:03.717835535 (83h17m39.281158548s) [noise=true , address=104.36.82.42:6832 , piece_id=TRXGPKOEU7HPFSEDKPJN3C6GJOBUZPBM6CNPZWS6ZLOHO2F7SSLA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:03.961658935 (83h17m39.524981945s) [noise=true , address=79.127.220.97:4363 , piece_id=5HMOMDEQKEOHRF67UVYFYSKIMH2IRLMME6S7XKDNVAMAFHE5EO6Q, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:09.244412959 (83h17m44.807735964s) [noise=true , address=104.36.82.56:4252 , piece_id=KHT4ZUAQRO7NJ2IWH6FORUJSH5V2UFKTGYJETRL5SLYRU2D57V4A, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:14.056547620 (83h17m49.619870606s) [noise=true , address=104.36.82.53:4136 , piece_id=QK4OSCIQAD6IB3I4QK4KYU4MTZEOPDJC5GG6HLMCUAXGDNLMSFYA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:19.882456778 (83h17m55.445779771s) [noise=true , address=104.36.82.55:4217 , piece_id=6DKIHWLLTMCGBT63TAG234VGEPOX7QCTXLOBVQSG56LJS5YYSPPA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:28.035171487 (83h18m3.598494478s) [noise=true , address=104.36.82.52:4094 , piece_id=7AXGQ6CXE6H7GTURGOCHG3UBTV3VWLVNIXPNM2NAN7IR2RQCTRIA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:57.717732997 (83h18m33.281055985s) [noise=true , address=104.36.82.12:6158 , piece_id=VWTJS7XRTMBJFUNAYMZCJQ2EEMT5CVILS4JWMPUK5GIYPLQADAJQ, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:57.717762830 (83h18m33.281085829s) [seg_pos={0 0}, upload=66665]: Segment reached optimal threshold of 14 pieces.
Jun 22 03:41:57.718005914 (83h18m33.281328913s) [noise=true , address=104.36.82.54:4167 , piece_id=I4FIVN2M4OF7EARPGIY3X72D2DOARBQGIRU2W7KCCOMUEEMZ776Q, seg_pos={0 0}, upload=66665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hhT46NW5CQ7dWHRxy5MSjyKc3PiQqBifThT7dBmx2sgNvn8GZ): protocol: write tcp 159.223.143.39:56114->104.36.82.54:4167: use of closed network connection; write tcp 159.223.143.39:56114->104.36.82.54:4167: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:56114->104.36.82.54:4167: use of closed network connection
Jun 22 03:41:57.718019794 (83h18m33.281342793s) [seg_pos={0 0}, upload=66665]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jun 22 03:41:57.718095039 (83h18m33.281418038s) [upload=66665]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 22 03:41:57.718100732 (83h18m33.281423731s) [upload=66665]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 22 03:41:57.767433584 (83h18m33.330756651s) [upload=66665]: Done uploading segments...
Jun 22 03:41:57.767452412 (83h18m33.330775402s) [upload=66665]: Upload finished. err: <nil>
Duration: 11.538500882s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......] duration:
19.950635738s
FAIL (2025-06-22 03:46:15 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
4 (83h17m33.572378185s) [upload=66665]: Uploading segments...
Jun 22 03:40:58.009397890 (83h17m33.572720878s) [upload=66665]: Got next segment. Inline: false
Jun 22 03:40:58.009451249 (83h17m33.572774249s) [upload=66665]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 22 03:40:58.009457748 (83h17m33.572780755s) [upload=66665]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 22 03:40:58.044661723 (83h17m33.607984722s) [seg_pos={0 0}, upload=66665]: Begin upload segment...
Jun 22 03:40:58.044722729 (83h17m33.608045718s) [seg_pos={0 0}, upload=66665]: Done begin upload segment.
Jun 22 03:40:58.044746618 (83h17m33.608069618s) [noise=true , address=104.36.82.5:5856 , piece_id=ZATJRQE4NARTU5JDY3XMUDSW5E6DTQZPTD5VQV5OIANWC2CSNARA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.044784017 (83h17m33.608107034s) [noise=true , address=79.127.220.97:4363 , piece_id=5HMOMDEQKEOHRF67UVYFYSKIMH2IRLMME6S7XKDNVAMAFHE5EO6Q, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.045240611 (83h17m33.608563622s) [noise=true , address=104.36.82.52:4094 , piece_id=7AXGQ6CXE6H7GTURGOCHG3UBTV3VWLVNIXPNM2NAN7IR2RQCTRIA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.045699560 (83h17m33.609022565s) [noise=true , address=104.36.82.12:6158 , piece_id=VWTJS7XRTMBJFUNAYMZCJQ2EEMT5CVILS4JWMPUK5GIYPLQADAJQ, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.046153432 (83h17m33.609476443s) [noise=true , address=104.36.82.11:6125 , piece_id=NP2722BQYMG5H7RQD3TLWYO23TFPT3HSBZFS6JRYYBFYBZWHEB5A, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.046516869 (83h17m33.609839878s) [noise=true , address=64.49.13.169:5711 , piece_id=ROPI2UEYUINLDKQ4IEVWXTYGSU3PXWGXFI7TC2L675L2KENKWWEQ, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.046897154 (83h17m33.610220195s) [noise=true , address=104.36.82.54:4167 , piece_id=I4FIVN2M4OF7EARPGIY3X72D2DOARBQGIRU2W7KCCOMUEEMZ776Q, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.047287027 (83h17m33.610610033s) [noise=true , address=104.36.82.3:5792 , piece_id=W2WE2U645R765EZI5CGYAOLJWIS6LQIGEFVNZUK4PS2IC6HEUILA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.047564954 (83h17m33.610887959s) [noise=true , address=45.91.116.8:5058 , piece_id=FVHGWLW7PVRET7FKWLXJ6V6MUJKWVTFNOKLGIEQQI72ZXZTW74BA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.047691900 (83h17m33.611014907s) [noise=true , address=104.36.82.55:4217 , piece_id=6DKIHWLLTMCGBT63TAG234VGEPOX7QCTXLOBVQSG56LJS5YYSPPA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.047925386 (83h17m33.611248395s) [noise=true , address=104.36.82.10:6112 , piece_id=56XUDWBP5SSSKOWDY2N2JV6PV63VXTROBOJQLZUUKGLI2ZBE7YMA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.048123429 (83h17m33.611446434s) [noise=true , address=104.36.82.14:6227 , piece_id=B4YO53N2MW6VCC5KHU5LWN6PQNGAJ4ULDPFLHL3OFRP3IBDUSVCA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.048331201 (83h17m33.61165421s) [noise=true , address=104.36.82.9:6016 , piece_id=MLPYRXQYKDFRHRQ4BKELVLTJD6ZMOHM7YSCS5O5HH6525IQOBPGQ, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.048701328 (83h17m33.612024333s) [noise=true , address=104.36.82.42:6832 , piece_id=TRXGPKOEU7HPFSEDKPJN3C6GJOBUZPBM6CNPZWS6ZLOHO2F7SSLA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.049050929 (83h17m33.612373939s) [noise=true , address=104.36.82.56:4252 , piece_id=KHT4ZUAQRO7NJ2IWH6FORUJSH5V2UFKTGYJETRL5SLYRU2D57V4A, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.049392658 (83h17m33.612715669s) [noise=true , address=79.127.219.48:4076 , piece_id=UQM6DRGET6FQFND5HU4CTJDIC4337XX6MK5NJV5XQPOK7AFCPSMQ, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.049754448 (83h17m33.613077451s) [noise=true , address=77.90.31.14:4688 , piece_id=GHO7ZNMRBXPWN34PPDFGKPOOA7L65YVO6AYHFFP5R4XCUGACY3DQ, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.050127175 (83h17m33.613450178s) [noise=true , address=104.36.82.53:4136 , piece_id=QK4OSCIQAD6IB3I4QK4KYU4MTZEOPDJC5GG6HLMCUAXGDNLMSFYA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.091864444 (83h17m33.655187435s) [noise=true , address=79.127.219.48:4076 , piece_id=UQM6DRGET6FQFND5HU4CTJDIC4337XX6MK5NJV5XQPOK7AFCPSMQ, seg_pos={0 0}, upload=66665]: Done uploading piece. err: ecclient: upload failed (node:1QeLZQsubmwYFwTPC9ipEgKJJvFjJ7VrXLC7QncaYoPyyFM4Fn, address:79.127.219.48:4076): protocol: storage node overloaded, request limit: 2000
Jun 22 03:40:58.161661930 (83h17m33.72498492s) [upload=66665]: Next returned nil segment
Jun 22 03:40:58.161681692 (83h17m33.725004695s) [upload=66665]: Waiting for error group managing segments...
Jun 22 03:40:58.190491904 (83h17m33.753814895s) [noise=true , address=104.36.82.11:6125 , piece_id=NP2722BQYMG5H7RQD3TLWYO23TFPT3HSBZFS6JRYYBFYBZWHEB5A, seg_pos={0 0}, upload=66665]: Done uploading piece. err: ecclient: upload failed (node:12vricofwx67FRaxAGLbmgT1TYyKkqfJbxdsyeEPL4R7kg8pKzT, address:104.36.82.11:6125): protocol: storage node overloaded, request limit: 2000
Jun 22 03:40:58.207750455 (83h17m33.77107344s) [noise=true , address=77.90.31.14:4688 , piece_id=GHO7ZNMRBXPWN34PPDFGKPOOA7L65YVO6AYHFFP5R4XCUGACY3DQ, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:40:58.531973385 (83h17m34.095296379s) [noise=true , address=64.49.13.169:5711 , piece_id=ROPI2UEYUINLDKQ4IEVWXTYGSU3PXWGXFI7TC2L675L2KENKWWEQ, seg_pos={0 0}, upload=66665]: Done uploading piece. err: ecclient: upload failed (node:1nxT1Ldr46HPrDRHQ2nrwEBZT5bEVoPAAfBz86ZYwUH13sqwHu, address:64.49.13.169:5711): protocol: storage node overloaded, request limit: 2000
Jun 22 03:40:58.990571568 (83h17m34.553894561s) [noise=true , address=104.36.82.9:6016 , piece_id=MLPYRXQYKDFRHRQ4BKELVLTJD6ZMOHM7YSCS5O5HH6525IQOBPGQ, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:00.325760684 (83h17m35.889083676s) [noise=true , address=104.36.82.14:6227 , piece_id=B4YO53N2MW6VCC5KHU5LWN6PQNGAJ4ULDPFLHL3OFRP3IBDUSVCA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:00.953756837 (83h17m36.517079837s) [noise=true , address=45.91.116.8:5058 , piece_id=FVHGWLW7PVRET7FKWLXJ6V6MUJKWVTFNOKLGIEQQI72ZXZTW74BA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:01.386979509 (83h17m36.950302499s) [noise=true , address=104.36.82.10:6112 , piece_id=56XUDWBP5SSSKOWDY2N2JV6PV63VXTROBOJQLZUUKGLI2ZBE7YMA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:01.689801010 (83h17m37.253124004s) [noise=true , address=104.36.82.3:5792 , piece_id=W2WE2U645R765EZI5CGYAOLJWIS6LQIGEFVNZUK4PS2IC6HEUILA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:01.856352835 (83h17m37.419675823s) [noise=true , address=104.36.82.5:5856 , piece_id=ZATJRQE4NARTU5JDY3XMUDSW5E6DTQZPTD5VQV5OIANWC2CSNARA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:03.717835535 (83h17m39.281158548s) [noise=true , address=104.36.82.42:6832 , piece_id=TRXGPKOEU7HPFSEDKPJN3C6GJOBUZPBM6CNPZWS6ZLOHO2F7SSLA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:03.961658935 (83h17m39.524981945s) [noise=true , address=79.127.220.97:4363 , piece_id=5HMOMDEQKEOHRF67UVYFYSKIMH2IRLMME6S7XKDNVAMAFHE5EO6Q, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:09.244412959 (83h17m44.807735964s) [noise=true , address=104.36.82.56:4252 , piece_id=KHT4ZUAQRO7NJ2IWH6FORUJSH5V2UFKTGYJETRL5SLYRU2D57V4A, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:14.056547620 (83h17m49.619870606s) [noise=true , address=104.36.82.53:4136 , piece_id=QK4OSCIQAD6IB3I4QK4KYU4MTZEOPDJC5GG6HLMCUAXGDNLMSFYA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:19.882456778 (83h17m55.445779771s) [noise=true , address=104.36.82.55:4217 , piece_id=6DKIHWLLTMCGBT63TAG234VGEPOX7QCTXLOBVQSG56LJS5YYSPPA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:28.035171487 (83h18m3.598494478s) [noise=true , address=104.36.82.52:4094 , piece_id=7AXGQ6CXE6H7GTURGOCHG3UBTV3VWLVNIXPNM2NAN7IR2RQCTRIA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:57.717732997 (83h18m33.281055985s) [noise=true , address=104.36.82.12:6158 , piece_id=VWTJS7XRTMBJFUNAYMZCJQ2EEMT5CVILS4JWMPUK5GIYPLQADAJQ, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:57.717762830 (83h18m33.281085829s) [seg_pos={0 0}, upload=66665]: Segment reached optimal threshold of 14 pieces.
Jun 22 03:41:57.718005914 (83h18m33.281328913s) [noise=true , address=104.36.82.54:4167 , piece_id=I4FIVN2M4OF7EARPGIY3X72D2DOARBQGIRU2W7KCCOMUEEMZ776Q, seg_pos={0 0}, upload=66665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hhT46NW5CQ7dWHRxy5MSjyKc3PiQqBifThT7dBmx2sgNvn8GZ): protocol: write tcp 159.223.143.39:56114->104.36.82.54:4167: use of closed network connection; write tcp 159.223.143.39:56114->104.36.82.54:4167: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:56114->104.36.82.54:4167: use of closed network connection
Jun 22 03:41:57.718019794 (83h18m33.281342793s) [seg_pos={0 0}, upload=66665]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jun 22 03:41:57.718095039 (83h18m33.281418038s) [upload=66665]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 22 03:41:57.718100732 (83h18m33.281423731s) [upload=66665]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 22 03:41:57.767433584 (83h18m33.330756651s) [upload=66665]: Done uploading segments...
Jun 22 03:41:57.767452412 (83h18m33.330775402s) [upload=66665]: Upload finished. err: <nil>
Duration: 19.950655809s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......] duration:
12.823135112s
FAIL (2025-06-22 03:45:53 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
4 (83h17m33.572378185s) [upload=66665]: Uploading segments...
Jun 22 03:40:58.009397890 (83h17m33.572720878s) [upload=66665]: Got next segment. Inline: false
Jun 22 03:40:58.009451249 (83h17m33.572774249s) [upload=66665]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 22 03:40:58.009457748 (83h17m33.572780755s) [upload=66665]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 22 03:40:58.044661723 (83h17m33.607984722s) [seg_pos={0 0}, upload=66665]: Begin upload segment...
Jun 22 03:40:58.044722729 (83h17m33.608045718s) [seg_pos={0 0}, upload=66665]: Done begin upload segment.
Jun 22 03:40:58.044746618 (83h17m33.608069618s) [noise=true , address=104.36.82.5:5856 , piece_id=ZATJRQE4NARTU5JDY3XMUDSW5E6DTQZPTD5VQV5OIANWC2CSNARA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.044784017 (83h17m33.608107034s) [noise=true , address=79.127.220.97:4363 , piece_id=5HMOMDEQKEOHRF67UVYFYSKIMH2IRLMME6S7XKDNVAMAFHE5EO6Q, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.045240611 (83h17m33.608563622s) [noise=true , address=104.36.82.52:4094 , piece_id=7AXGQ6CXE6H7GTURGOCHG3UBTV3VWLVNIXPNM2NAN7IR2RQCTRIA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.045699560 (83h17m33.609022565s) [noise=true , address=104.36.82.12:6158 , piece_id=VWTJS7XRTMBJFUNAYMZCJQ2EEMT5CVILS4JWMPUK5GIYPLQADAJQ, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.046153432 (83h17m33.609476443s) [noise=true , address=104.36.82.11:6125 , piece_id=NP2722BQYMG5H7RQD3TLWYO23TFPT3HSBZFS6JRYYBFYBZWHEB5A, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.046516869 (83h17m33.609839878s) [noise=true , address=64.49.13.169:5711 , piece_id=ROPI2UEYUINLDKQ4IEVWXTYGSU3PXWGXFI7TC2L675L2KENKWWEQ, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.046897154 (83h17m33.610220195s) [noise=true , address=104.36.82.54:4167 , piece_id=I4FIVN2M4OF7EARPGIY3X72D2DOARBQGIRU2W7KCCOMUEEMZ776Q, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.047287027 (83h17m33.610610033s) [noise=true , address=104.36.82.3:5792 , piece_id=W2WE2U645R765EZI5CGYAOLJWIS6LQIGEFVNZUK4PS2IC6HEUILA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.047564954 (83h17m33.610887959s) [noise=true , address=45.91.116.8:5058 , piece_id=FVHGWLW7PVRET7FKWLXJ6V6MUJKWVTFNOKLGIEQQI72ZXZTW74BA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.047691900 (83h17m33.611014907s) [noise=true , address=104.36.82.55:4217 , piece_id=6DKIHWLLTMCGBT63TAG234VGEPOX7QCTXLOBVQSG56LJS5YYSPPA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.047925386 (83h17m33.611248395s) [noise=true , address=104.36.82.10:6112 , piece_id=56XUDWBP5SSSKOWDY2N2JV6PV63VXTROBOJQLZUUKGLI2ZBE7YMA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.048123429 (83h17m33.611446434s) [noise=true , address=104.36.82.14:6227 , piece_id=B4YO53N2MW6VCC5KHU5LWN6PQNGAJ4ULDPFLHL3OFRP3IBDUSVCA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.048331201 (83h17m33.61165421s) [noise=true , address=104.36.82.9:6016 , piece_id=MLPYRXQYKDFRHRQ4BKELVLTJD6ZMOHM7YSCS5O5HH6525IQOBPGQ, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.048701328 (83h17m33.612024333s) [noise=true , address=104.36.82.42:6832 , piece_id=TRXGPKOEU7HPFSEDKPJN3C6GJOBUZPBM6CNPZWS6ZLOHO2F7SSLA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.049050929 (83h17m33.612373939s) [noise=true , address=104.36.82.56:4252 , piece_id=KHT4ZUAQRO7NJ2IWH6FORUJSH5V2UFKTGYJETRL5SLYRU2D57V4A, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.049392658 (83h17m33.612715669s) [noise=true , address=79.127.219.48:4076 , piece_id=UQM6DRGET6FQFND5HU4CTJDIC4337XX6MK5NJV5XQPOK7AFCPSMQ, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.049754448 (83h17m33.613077451s) [noise=true , address=77.90.31.14:4688 , piece_id=GHO7ZNMRBXPWN34PPDFGKPOOA7L65YVO6AYHFFP5R4XCUGACY3DQ, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.050127175 (83h17m33.613450178s) [noise=true , address=104.36.82.53:4136 , piece_id=QK4OSCIQAD6IB3I4QK4KYU4MTZEOPDJC5GG6HLMCUAXGDNLMSFYA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.091864444 (83h17m33.655187435s) [noise=true , address=79.127.219.48:4076 , piece_id=UQM6DRGET6FQFND5HU4CTJDIC4337XX6MK5NJV5XQPOK7AFCPSMQ, seg_pos={0 0}, upload=66665]: Done uploading piece. err: ecclient: upload failed (node:1QeLZQsubmwYFwTPC9ipEgKJJvFjJ7VrXLC7QncaYoPyyFM4Fn, address:79.127.219.48:4076): protocol: storage node overloaded, request limit: 2000
Jun 22 03:40:58.161661930 (83h17m33.72498492s) [upload=66665]: Next returned nil segment
Jun 22 03:40:58.161681692 (83h17m33.725004695s) [upload=66665]: Waiting for error group managing segments...
Jun 22 03:40:58.190491904 (83h17m33.753814895s) [noise=true , address=104.36.82.11:6125 , piece_id=NP2722BQYMG5H7RQD3TLWYO23TFPT3HSBZFS6JRYYBFYBZWHEB5A, seg_pos={0 0}, upload=66665]: Done uploading piece. err: ecclient: upload failed (node:12vricofwx67FRaxAGLbmgT1TYyKkqfJbxdsyeEPL4R7kg8pKzT, address:104.36.82.11:6125): protocol: storage node overloaded, request limit: 2000
Jun 22 03:40:58.207750455 (83h17m33.77107344s) [noise=true , address=77.90.31.14:4688 , piece_id=GHO7ZNMRBXPWN34PPDFGKPOOA7L65YVO6AYHFFP5R4XCUGACY3DQ, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:40:58.531973385 (83h17m34.095296379s) [noise=true , address=64.49.13.169:5711 , piece_id=ROPI2UEYUINLDKQ4IEVWXTYGSU3PXWGXFI7TC2L675L2KENKWWEQ, seg_pos={0 0}, upload=66665]: Done uploading piece. err: ecclient: upload failed (node:1nxT1Ldr46HPrDRHQ2nrwEBZT5bEVoPAAfBz86ZYwUH13sqwHu, address:64.49.13.169:5711): protocol: storage node overloaded, request limit: 2000
Jun 22 03:40:58.990571568 (83h17m34.553894561s) [noise=true , address=104.36.82.9:6016 , piece_id=MLPYRXQYKDFRHRQ4BKELVLTJD6ZMOHM7YSCS5O5HH6525IQOBPGQ, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:00.325760684 (83h17m35.889083676s) [noise=true , address=104.36.82.14:6227 , piece_id=B4YO53N2MW6VCC5KHU5LWN6PQNGAJ4ULDPFLHL3OFRP3IBDUSVCA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:00.953756837 (83h17m36.517079837s) [noise=true , address=45.91.116.8:5058 , piece_id=FVHGWLW7PVRET7FKWLXJ6V6MUJKWVTFNOKLGIEQQI72ZXZTW74BA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:01.386979509 (83h17m36.950302499s) [noise=true , address=104.36.82.10:6112 , piece_id=56XUDWBP5SSSKOWDY2N2JV6PV63VXTROBOJQLZUUKGLI2ZBE7YMA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:01.689801010 (83h17m37.253124004s) [noise=true , address=104.36.82.3:5792 , piece_id=W2WE2U645R765EZI5CGYAOLJWIS6LQIGEFVNZUK4PS2IC6HEUILA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:01.856352835 (83h17m37.419675823s) [noise=true , address=104.36.82.5:5856 , piece_id=ZATJRQE4NARTU5JDY3XMUDSW5E6DTQZPTD5VQV5OIANWC2CSNARA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:03.717835535 (83h17m39.281158548s) [noise=true , address=104.36.82.42:6832 , piece_id=TRXGPKOEU7HPFSEDKPJN3C6GJOBUZPBM6CNPZWS6ZLOHO2F7SSLA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:03.961658935 (83h17m39.524981945s) [noise=true , address=79.127.220.97:4363 , piece_id=5HMOMDEQKEOHRF67UVYFYSKIMH2IRLMME6S7XKDNVAMAFHE5EO6Q, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:09.244412959 (83h17m44.807735964s) [noise=true , address=104.36.82.56:4252 , piece_id=KHT4ZUAQRO7NJ2IWH6FORUJSH5V2UFKTGYJETRL5SLYRU2D57V4A, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:14.056547620 (83h17m49.619870606s) [noise=true , address=104.36.82.53:4136 , piece_id=QK4OSCIQAD6IB3I4QK4KYU4MTZEOPDJC5GG6HLMCUAXGDNLMSFYA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:19.882456778 (83h17m55.445779771s) [noise=true , address=104.36.82.55:4217 , piece_id=6DKIHWLLTMCGBT63TAG234VGEPOX7QCTXLOBVQSG56LJS5YYSPPA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:28.035171487 (83h18m3.598494478s) [noise=true , address=104.36.82.52:4094 , piece_id=7AXGQ6CXE6H7GTURGOCHG3UBTV3VWLVNIXPNM2NAN7IR2RQCTRIA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:57.717732997 (83h18m33.281055985s) [noise=true , address=104.36.82.12:6158 , piece_id=VWTJS7XRTMBJFUNAYMZCJQ2EEMT5CVILS4JWMPUK5GIYPLQADAJQ, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:57.717762830 (83h18m33.281085829s) [seg_pos={0 0}, upload=66665]: Segment reached optimal threshold of 14 pieces.
Jun 22 03:41:57.718005914 (83h18m33.281328913s) [noise=true , address=104.36.82.54:4167 , piece_id=I4FIVN2M4OF7EARPGIY3X72D2DOARBQGIRU2W7KCCOMUEEMZ776Q, seg_pos={0 0}, upload=66665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hhT46NW5CQ7dWHRxy5MSjyKc3PiQqBifThT7dBmx2sgNvn8GZ): protocol: write tcp 159.223.143.39:56114->104.36.82.54:4167: use of closed network connection; write tcp 159.223.143.39:56114->104.36.82.54:4167: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:56114->104.36.82.54:4167: use of closed network connection
Jun 22 03:41:57.718019794 (83h18m33.281342793s) [seg_pos={0 0}, upload=66665]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jun 22 03:41:57.718095039 (83h18m33.281418038s) [upload=66665]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 22 03:41:57.718100732 (83h18m33.281423731s) [upload=66665]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 22 03:41:57.767433584 (83h18m33.330756651s) [upload=66665]: Done uploading segments...
Jun 22 03:41:57.767452412 (83h18m33.330775402s) [upload=66665]: Upload finished. err: <nil>
Duration: 12.82316261s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....] duration:
13.456848051s
FAIL (2025-06-22 03:45:00 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
4 (83h17m33.572378185s) [upload=66665]: Uploading segments...
Jun 22 03:40:58.009397890 (83h17m33.572720878s) [upload=66665]: Got next segment. Inline: false
Jun 22 03:40:58.009451249 (83h17m33.572774249s) [upload=66665]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 22 03:40:58.009457748 (83h17m33.572780755s) [upload=66665]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 22 03:40:58.044661723 (83h17m33.607984722s) [seg_pos={0 0}, upload=66665]: Begin upload segment...
Jun 22 03:40:58.044722729 (83h17m33.608045718s) [seg_pos={0 0}, upload=66665]: Done begin upload segment.
Jun 22 03:40:58.044746618 (83h17m33.608069618s) [noise=true , address=104.36.82.5:5856 , piece_id=ZATJRQE4NARTU5JDY3XMUDSW5E6DTQZPTD5VQV5OIANWC2CSNARA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.044784017 (83h17m33.608107034s) [noise=true , address=79.127.220.97:4363 , piece_id=5HMOMDEQKEOHRF67UVYFYSKIMH2IRLMME6S7XKDNVAMAFHE5EO6Q, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.045240611 (83h17m33.608563622s) [noise=true , address=104.36.82.52:4094 , piece_id=7AXGQ6CXE6H7GTURGOCHG3UBTV3VWLVNIXPNM2NAN7IR2RQCTRIA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.045699560 (83h17m33.609022565s) [noise=true , address=104.36.82.12:6158 , piece_id=VWTJS7XRTMBJFUNAYMZCJQ2EEMT5CVILS4JWMPUK5GIYPLQADAJQ, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.046153432 (83h17m33.609476443s) [noise=true , address=104.36.82.11:6125 , piece_id=NP2722BQYMG5H7RQD3TLWYO23TFPT3HSBZFS6JRYYBFYBZWHEB5A, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.046516869 (83h17m33.609839878s) [noise=true , address=64.49.13.169:5711 , piece_id=ROPI2UEYUINLDKQ4IEVWXTYGSU3PXWGXFI7TC2L675L2KENKWWEQ, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.046897154 (83h17m33.610220195s) [noise=true , address=104.36.82.54:4167 , piece_id=I4FIVN2M4OF7EARPGIY3X72D2DOARBQGIRU2W7KCCOMUEEMZ776Q, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.047287027 (83h17m33.610610033s) [noise=true , address=104.36.82.3:5792 , piece_id=W2WE2U645R765EZI5CGYAOLJWIS6LQIGEFVNZUK4PS2IC6HEUILA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.047564954 (83h17m33.610887959s) [noise=true , address=45.91.116.8:5058 , piece_id=FVHGWLW7PVRET7FKWLXJ6V6MUJKWVTFNOKLGIEQQI72ZXZTW74BA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.047691900 (83h17m33.611014907s) [noise=true , address=104.36.82.55:4217 , piece_id=6DKIHWLLTMCGBT63TAG234VGEPOX7QCTXLOBVQSG56LJS5YYSPPA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.047925386 (83h17m33.611248395s) [noise=true , address=104.36.82.10:6112 , piece_id=56XUDWBP5SSSKOWDY2N2JV6PV63VXTROBOJQLZUUKGLI2ZBE7YMA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.048123429 (83h17m33.611446434s) [noise=true , address=104.36.82.14:6227 , piece_id=B4YO53N2MW6VCC5KHU5LWN6PQNGAJ4ULDPFLHL3OFRP3IBDUSVCA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.048331201 (83h17m33.61165421s) [noise=true , address=104.36.82.9:6016 , piece_id=MLPYRXQYKDFRHRQ4BKELVLTJD6ZMOHM7YSCS5O5HH6525IQOBPGQ, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.048701328 (83h17m33.612024333s) [noise=true , address=104.36.82.42:6832 , piece_id=TRXGPKOEU7HPFSEDKPJN3C6GJOBUZPBM6CNPZWS6ZLOHO2F7SSLA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.049050929 (83h17m33.612373939s) [noise=true , address=104.36.82.56:4252 , piece_id=KHT4ZUAQRO7NJ2IWH6FORUJSH5V2UFKTGYJETRL5SLYRU2D57V4A, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.049392658 (83h17m33.612715669s) [noise=true , address=79.127.219.48:4076 , piece_id=UQM6DRGET6FQFND5HU4CTJDIC4337XX6MK5NJV5XQPOK7AFCPSMQ, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.049754448 (83h17m33.613077451s) [noise=true , address=77.90.31.14:4688 , piece_id=GHO7ZNMRBXPWN34PPDFGKPOOA7L65YVO6AYHFFP5R4XCUGACY3DQ, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.050127175 (83h17m33.613450178s) [noise=true , address=104.36.82.53:4136 , piece_id=QK4OSCIQAD6IB3I4QK4KYU4MTZEOPDJC5GG6HLMCUAXGDNLMSFYA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.091864444 (83h17m33.655187435s) [noise=true , address=79.127.219.48:4076 , piece_id=UQM6DRGET6FQFND5HU4CTJDIC4337XX6MK5NJV5XQPOK7AFCPSMQ, seg_pos={0 0}, upload=66665]: Done uploading piece. err: ecclient: upload failed (node:1QeLZQsubmwYFwTPC9ipEgKJJvFjJ7VrXLC7QncaYoPyyFM4Fn, address:79.127.219.48:4076): protocol: storage node overloaded, request limit: 2000
Jun 22 03:40:58.161661930 (83h17m33.72498492s) [upload=66665]: Next returned nil segment
Jun 22 03:40:58.161681692 (83h17m33.725004695s) [upload=66665]: Waiting for error group managing segments...
Jun 22 03:40:58.190491904 (83h17m33.753814895s) [noise=true , address=104.36.82.11:6125 , piece_id=NP2722BQYMG5H7RQD3TLWYO23TFPT3HSBZFS6JRYYBFYBZWHEB5A, seg_pos={0 0}, upload=66665]: Done uploading piece. err: ecclient: upload failed (node:12vricofwx67FRaxAGLbmgT1TYyKkqfJbxdsyeEPL4R7kg8pKzT, address:104.36.82.11:6125): protocol: storage node overloaded, request limit: 2000
Jun 22 03:40:58.207750455 (83h17m33.77107344s) [noise=true , address=77.90.31.14:4688 , piece_id=GHO7ZNMRBXPWN34PPDFGKPOOA7L65YVO6AYHFFP5R4XCUGACY3DQ, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:40:58.531973385 (83h17m34.095296379s) [noise=true , address=64.49.13.169:5711 , piece_id=ROPI2UEYUINLDKQ4IEVWXTYGSU3PXWGXFI7TC2L675L2KENKWWEQ, seg_pos={0 0}, upload=66665]: Done uploading piece. err: ecclient: upload failed (node:1nxT1Ldr46HPrDRHQ2nrwEBZT5bEVoPAAfBz86ZYwUH13sqwHu, address:64.49.13.169:5711): protocol: storage node overloaded, request limit: 2000
Jun 22 03:40:58.990571568 (83h17m34.553894561s) [noise=true , address=104.36.82.9:6016 , piece_id=MLPYRXQYKDFRHRQ4BKELVLTJD6ZMOHM7YSCS5O5HH6525IQOBPGQ, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:00.325760684 (83h17m35.889083676s) [noise=true , address=104.36.82.14:6227 , piece_id=B4YO53N2MW6VCC5KHU5LWN6PQNGAJ4ULDPFLHL3OFRP3IBDUSVCA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:00.953756837 (83h17m36.517079837s) [noise=true , address=45.91.116.8:5058 , piece_id=FVHGWLW7PVRET7FKWLXJ6V6MUJKWVTFNOKLGIEQQI72ZXZTW74BA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:01.386979509 (83h17m36.950302499s) [noise=true , address=104.36.82.10:6112 , piece_id=56XUDWBP5SSSKOWDY2N2JV6PV63VXTROBOJQLZUUKGLI2ZBE7YMA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:01.689801010 (83h17m37.253124004s) [noise=true , address=104.36.82.3:5792 , piece_id=W2WE2U645R765EZI5CGYAOLJWIS6LQIGEFVNZUK4PS2IC6HEUILA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:01.856352835 (83h17m37.419675823s) [noise=true , address=104.36.82.5:5856 , piece_id=ZATJRQE4NARTU5JDY3XMUDSW5E6DTQZPTD5VQV5OIANWC2CSNARA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:03.717835535 (83h17m39.281158548s) [noise=true , address=104.36.82.42:6832 , piece_id=TRXGPKOEU7HPFSEDKPJN3C6GJOBUZPBM6CNPZWS6ZLOHO2F7SSLA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:03.961658935 (83h17m39.524981945s) [noise=true , address=79.127.220.97:4363 , piece_id=5HMOMDEQKEOHRF67UVYFYSKIMH2IRLMME6S7XKDNVAMAFHE5EO6Q, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:09.244412959 (83h17m44.807735964s) [noise=true , address=104.36.82.56:4252 , piece_id=KHT4ZUAQRO7NJ2IWH6FORUJSH5V2UFKTGYJETRL5SLYRU2D57V4A, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:14.056547620 (83h17m49.619870606s) [noise=true , address=104.36.82.53:4136 , piece_id=QK4OSCIQAD6IB3I4QK4KYU4MTZEOPDJC5GG6HLMCUAXGDNLMSFYA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:19.882456778 (83h17m55.445779771s) [noise=true , address=104.36.82.55:4217 , piece_id=6DKIHWLLTMCGBT63TAG234VGEPOX7QCTXLOBVQSG56LJS5YYSPPA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:28.035171487 (83h18m3.598494478s) [noise=true , address=104.36.82.52:4094 , piece_id=7AXGQ6CXE6H7GTURGOCHG3UBTV3VWLVNIXPNM2NAN7IR2RQCTRIA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:57.717732997 (83h18m33.281055985s) [noise=true , address=104.36.82.12:6158 , piece_id=VWTJS7XRTMBJFUNAYMZCJQ2EEMT5CVILS4JWMPUK5GIYPLQADAJQ, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:57.717762830 (83h18m33.281085829s) [seg_pos={0 0}, upload=66665]: Segment reached optimal threshold of 14 pieces.
Jun 22 03:41:57.718005914 (83h18m33.281328913s) [noise=true , address=104.36.82.54:4167 , piece_id=I4FIVN2M4OF7EARPGIY3X72D2DOARBQGIRU2W7KCCOMUEEMZ776Q, seg_pos={0 0}, upload=66665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hhT46NW5CQ7dWHRxy5MSjyKc3PiQqBifThT7dBmx2sgNvn8GZ): protocol: write tcp 159.223.143.39:56114->104.36.82.54:4167: use of closed network connection; write tcp 159.223.143.39:56114->104.36.82.54:4167: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:56114->104.36.82.54:4167: use of closed network connection
Jun 22 03:41:57.718019794 (83h18m33.281342793s) [seg_pos={0 0}, upload=66665]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jun 22 03:41:57.718095039 (83h18m33.281418038s) [upload=66665]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 22 03:41:57.718100732 (83h18m33.281423731s) [upload=66665]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 22 03:41:57.767433584 (83h18m33.330756651s) [upload=66665]: Done uploading segments...
Jun 22 03:41:57.767452412 (83h18m33.330775402s) [upload=66665]: Upload finished. err: <nil>
Duration: 13.456878133s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....] duration:
14.132436721s
FAIL (2025-06-22 03:43:55 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
4 (83h17m33.572378185s) [upload=66665]: Uploading segments...
Jun 22 03:40:58.009397890 (83h17m33.572720878s) [upload=66665]: Got next segment. Inline: false
Jun 22 03:40:58.009451249 (83h17m33.572774249s) [upload=66665]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 22 03:40:58.009457748 (83h17m33.572780755s) [upload=66665]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 22 03:40:58.044661723 (83h17m33.607984722s) [seg_pos={0 0}, upload=66665]: Begin upload segment...
Jun 22 03:40:58.044722729 (83h17m33.608045718s) [seg_pos={0 0}, upload=66665]: Done begin upload segment.
Jun 22 03:40:58.044746618 (83h17m33.608069618s) [noise=true , address=104.36.82.5:5856 , piece_id=ZATJRQE4NARTU5JDY3XMUDSW5E6DTQZPTD5VQV5OIANWC2CSNARA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.044784017 (83h17m33.608107034s) [noise=true , address=79.127.220.97:4363 , piece_id=5HMOMDEQKEOHRF67UVYFYSKIMH2IRLMME6S7XKDNVAMAFHE5EO6Q, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.045240611 (83h17m33.608563622s) [noise=true , address=104.36.82.52:4094 , piece_id=7AXGQ6CXE6H7GTURGOCHG3UBTV3VWLVNIXPNM2NAN7IR2RQCTRIA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.045699560 (83h17m33.609022565s) [noise=true , address=104.36.82.12:6158 , piece_id=VWTJS7XRTMBJFUNAYMZCJQ2EEMT5CVILS4JWMPUK5GIYPLQADAJQ, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.046153432 (83h17m33.609476443s) [noise=true , address=104.36.82.11:6125 , piece_id=NP2722BQYMG5H7RQD3TLWYO23TFPT3HSBZFS6JRYYBFYBZWHEB5A, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.046516869 (83h17m33.609839878s) [noise=true , address=64.49.13.169:5711 , piece_id=ROPI2UEYUINLDKQ4IEVWXTYGSU3PXWGXFI7TC2L675L2KENKWWEQ, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.046897154 (83h17m33.610220195s) [noise=true , address=104.36.82.54:4167 , piece_id=I4FIVN2M4OF7EARPGIY3X72D2DOARBQGIRU2W7KCCOMUEEMZ776Q, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.047287027 (83h17m33.610610033s) [noise=true , address=104.36.82.3:5792 , piece_id=W2WE2U645R765EZI5CGYAOLJWIS6LQIGEFVNZUK4PS2IC6HEUILA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.047564954 (83h17m33.610887959s) [noise=true , address=45.91.116.8:5058 , piece_id=FVHGWLW7PVRET7FKWLXJ6V6MUJKWVTFNOKLGIEQQI72ZXZTW74BA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.047691900 (83h17m33.611014907s) [noise=true , address=104.36.82.55:4217 , piece_id=6DKIHWLLTMCGBT63TAG234VGEPOX7QCTXLOBVQSG56LJS5YYSPPA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.047925386 (83h17m33.611248395s) [noise=true , address=104.36.82.10:6112 , piece_id=56XUDWBP5SSSKOWDY2N2JV6PV63VXTROBOJQLZUUKGLI2ZBE7YMA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.048123429 (83h17m33.611446434s) [noise=true , address=104.36.82.14:6227 , piece_id=B4YO53N2MW6VCC5KHU5LWN6PQNGAJ4ULDPFLHL3OFRP3IBDUSVCA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.048331201 (83h17m33.61165421s) [noise=true , address=104.36.82.9:6016 , piece_id=MLPYRXQYKDFRHRQ4BKELVLTJD6ZMOHM7YSCS5O5HH6525IQOBPGQ, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.048701328 (83h17m33.612024333s) [noise=true , address=104.36.82.42:6832 , piece_id=TRXGPKOEU7HPFSEDKPJN3C6GJOBUZPBM6CNPZWS6ZLOHO2F7SSLA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.049050929 (83h17m33.612373939s) [noise=true , address=104.36.82.56:4252 , piece_id=KHT4ZUAQRO7NJ2IWH6FORUJSH5V2UFKTGYJETRL5SLYRU2D57V4A, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.049392658 (83h17m33.612715669s) [noise=true , address=79.127.219.48:4076 , piece_id=UQM6DRGET6FQFND5HU4CTJDIC4337XX6MK5NJV5XQPOK7AFCPSMQ, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.049754448 (83h17m33.613077451s) [noise=true , address=77.90.31.14:4688 , piece_id=GHO7ZNMRBXPWN34PPDFGKPOOA7L65YVO6AYHFFP5R4XCUGACY3DQ, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.050127175 (83h17m33.613450178s) [noise=true , address=104.36.82.53:4136 , piece_id=QK4OSCIQAD6IB3I4QK4KYU4MTZEOPDJC5GG6HLMCUAXGDNLMSFYA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.091864444 (83h17m33.655187435s) [noise=true , address=79.127.219.48:4076 , piece_id=UQM6DRGET6FQFND5HU4CTJDIC4337XX6MK5NJV5XQPOK7AFCPSMQ, seg_pos={0 0}, upload=66665]: Done uploading piece. err: ecclient: upload failed (node:1QeLZQsubmwYFwTPC9ipEgKJJvFjJ7VrXLC7QncaYoPyyFM4Fn, address:79.127.219.48:4076): protocol: storage node overloaded, request limit: 2000
Jun 22 03:40:58.161661930 (83h17m33.72498492s) [upload=66665]: Next returned nil segment
Jun 22 03:40:58.161681692 (83h17m33.725004695s) [upload=66665]: Waiting for error group managing segments...
Jun 22 03:40:58.190491904 (83h17m33.753814895s) [noise=true , address=104.36.82.11:6125 , piece_id=NP2722BQYMG5H7RQD3TLWYO23TFPT3HSBZFS6JRYYBFYBZWHEB5A, seg_pos={0 0}, upload=66665]: Done uploading piece. err: ecclient: upload failed (node:12vricofwx67FRaxAGLbmgT1TYyKkqfJbxdsyeEPL4R7kg8pKzT, address:104.36.82.11:6125): protocol: storage node overloaded, request limit: 2000
Jun 22 03:40:58.207750455 (83h17m33.77107344s) [noise=true , address=77.90.31.14:4688 , piece_id=GHO7ZNMRBXPWN34PPDFGKPOOA7L65YVO6AYHFFP5R4XCUGACY3DQ, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:40:58.531973385 (83h17m34.095296379s) [noise=true , address=64.49.13.169:5711 , piece_id=ROPI2UEYUINLDKQ4IEVWXTYGSU3PXWGXFI7TC2L675L2KENKWWEQ, seg_pos={0 0}, upload=66665]: Done uploading piece. err: ecclient: upload failed (node:1nxT1Ldr46HPrDRHQ2nrwEBZT5bEVoPAAfBz86ZYwUH13sqwHu, address:64.49.13.169:5711): protocol: storage node overloaded, request limit: 2000
Jun 22 03:40:58.990571568 (83h17m34.553894561s) [noise=true , address=104.36.82.9:6016 , piece_id=MLPYRXQYKDFRHRQ4BKELVLTJD6ZMOHM7YSCS5O5HH6525IQOBPGQ, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:00.325760684 (83h17m35.889083676s) [noise=true , address=104.36.82.14:6227 , piece_id=B4YO53N2MW6VCC5KHU5LWN6PQNGAJ4ULDPFLHL3OFRP3IBDUSVCA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:00.953756837 (83h17m36.517079837s) [noise=true , address=45.91.116.8:5058 , piece_id=FVHGWLW7PVRET7FKWLXJ6V6MUJKWVTFNOKLGIEQQI72ZXZTW74BA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:01.386979509 (83h17m36.950302499s) [noise=true , address=104.36.82.10:6112 , piece_id=56XUDWBP5SSSKOWDY2N2JV6PV63VXTROBOJQLZUUKGLI2ZBE7YMA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:01.689801010 (83h17m37.253124004s) [noise=true , address=104.36.82.3:5792 , piece_id=W2WE2U645R765EZI5CGYAOLJWIS6LQIGEFVNZUK4PS2IC6HEUILA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:01.856352835 (83h17m37.419675823s) [noise=true , address=104.36.82.5:5856 , piece_id=ZATJRQE4NARTU5JDY3XMUDSW5E6DTQZPTD5VQV5OIANWC2CSNARA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:03.717835535 (83h17m39.281158548s) [noise=true , address=104.36.82.42:6832 , piece_id=TRXGPKOEU7HPFSEDKPJN3C6GJOBUZPBM6CNPZWS6ZLOHO2F7SSLA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:03.961658935 (83h17m39.524981945s) [noise=true , address=79.127.220.97:4363 , piece_id=5HMOMDEQKEOHRF67UVYFYSKIMH2IRLMME6S7XKDNVAMAFHE5EO6Q, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:09.244412959 (83h17m44.807735964s) [noise=true , address=104.36.82.56:4252 , piece_id=KHT4ZUAQRO7NJ2IWH6FORUJSH5V2UFKTGYJETRL5SLYRU2D57V4A, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:14.056547620 (83h17m49.619870606s) [noise=true , address=104.36.82.53:4136 , piece_id=QK4OSCIQAD6IB3I4QK4KYU4MTZEOPDJC5GG6HLMCUAXGDNLMSFYA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:19.882456778 (83h17m55.445779771s) [noise=true , address=104.36.82.55:4217 , piece_id=6DKIHWLLTMCGBT63TAG234VGEPOX7QCTXLOBVQSG56LJS5YYSPPA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:28.035171487 (83h18m3.598494478s) [noise=true , address=104.36.82.52:4094 , piece_id=7AXGQ6CXE6H7GTURGOCHG3UBTV3VWLVNIXPNM2NAN7IR2RQCTRIA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:57.717732997 (83h18m33.281055985s) [noise=true , address=104.36.82.12:6158 , piece_id=VWTJS7XRTMBJFUNAYMZCJQ2EEMT5CVILS4JWMPUK5GIYPLQADAJQ, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:57.717762830 (83h18m33.281085829s) [seg_pos={0 0}, upload=66665]: Segment reached optimal threshold of 14 pieces.
Jun 22 03:41:57.718005914 (83h18m33.281328913s) [noise=true , address=104.36.82.54:4167 , piece_id=I4FIVN2M4OF7EARPGIY3X72D2DOARBQGIRU2W7KCCOMUEEMZ776Q, seg_pos={0 0}, upload=66665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hhT46NW5CQ7dWHRxy5MSjyKc3PiQqBifThT7dBmx2sgNvn8GZ): protocol: write tcp 159.223.143.39:56114->104.36.82.54:4167: use of closed network connection; write tcp 159.223.143.39:56114->104.36.82.54:4167: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:56114->104.36.82.54:4167: use of closed network connection
Jun 22 03:41:57.718019794 (83h18m33.281342793s) [seg_pos={0 0}, upload=66665]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jun 22 03:41:57.718095039 (83h18m33.281418038s) [upload=66665]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 22 03:41:57.718100732 (83h18m33.281423731s) [upload=66665]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 22 03:41:57.767433584 (83h18m33.330756651s) [upload=66665]: Done uploading segments...
Jun 22 03:41:57.767452412 (83h18m33.330775402s) [upload=66665]: Upload finished. err: <nil>
Duration: 14.132468424s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...] duration:
13.154995057s
FAIL (2025-06-22 03:43:23 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
4 (83h17m33.572378185s) [upload=66665]: Uploading segments...
Jun 22 03:40:58.009397890 (83h17m33.572720878s) [upload=66665]: Got next segment. Inline: false
Jun 22 03:40:58.009451249 (83h17m33.572774249s) [upload=66665]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 22 03:40:58.009457748 (83h17m33.572780755s) [upload=66665]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 22 03:40:58.044661723 (83h17m33.607984722s) [seg_pos={0 0}, upload=66665]: Begin upload segment...
Jun 22 03:40:58.044722729 (83h17m33.608045718s) [seg_pos={0 0}, upload=66665]: Done begin upload segment.
Jun 22 03:40:58.044746618 (83h17m33.608069618s) [noise=true , address=104.36.82.5:5856 , piece_id=ZATJRQE4NARTU5JDY3XMUDSW5E6DTQZPTD5VQV5OIANWC2CSNARA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.044784017 (83h17m33.608107034s) [noise=true , address=79.127.220.97:4363 , piece_id=5HMOMDEQKEOHRF67UVYFYSKIMH2IRLMME6S7XKDNVAMAFHE5EO6Q, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.045240611 (83h17m33.608563622s) [noise=true , address=104.36.82.52:4094 , piece_id=7AXGQ6CXE6H7GTURGOCHG3UBTV3VWLVNIXPNM2NAN7IR2RQCTRIA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.045699560 (83h17m33.609022565s) [noise=true , address=104.36.82.12:6158 , piece_id=VWTJS7XRTMBJFUNAYMZCJQ2EEMT5CVILS4JWMPUK5GIYPLQADAJQ, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.046153432 (83h17m33.609476443s) [noise=true , address=104.36.82.11:6125 , piece_id=NP2722BQYMG5H7RQD3TLWYO23TFPT3HSBZFS6JRYYBFYBZWHEB5A, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.046516869 (83h17m33.609839878s) [noise=true , address=64.49.13.169:5711 , piece_id=ROPI2UEYUINLDKQ4IEVWXTYGSU3PXWGXFI7TC2L675L2KENKWWEQ, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.046897154 (83h17m33.610220195s) [noise=true , address=104.36.82.54:4167 , piece_id=I4FIVN2M4OF7EARPGIY3X72D2DOARBQGIRU2W7KCCOMUEEMZ776Q, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.047287027 (83h17m33.610610033s) [noise=true , address=104.36.82.3:5792 , piece_id=W2WE2U645R765EZI5CGYAOLJWIS6LQIGEFVNZUK4PS2IC6HEUILA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.047564954 (83h17m33.610887959s) [noise=true , address=45.91.116.8:5058 , piece_id=FVHGWLW7PVRET7FKWLXJ6V6MUJKWVTFNOKLGIEQQI72ZXZTW74BA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.047691900 (83h17m33.611014907s) [noise=true , address=104.36.82.55:4217 , piece_id=6DKIHWLLTMCGBT63TAG234VGEPOX7QCTXLOBVQSG56LJS5YYSPPA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.047925386 (83h17m33.611248395s) [noise=true , address=104.36.82.10:6112 , piece_id=56XUDWBP5SSSKOWDY2N2JV6PV63VXTROBOJQLZUUKGLI2ZBE7YMA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.048123429 (83h17m33.611446434s) [noise=true , address=104.36.82.14:6227 , piece_id=B4YO53N2MW6VCC5KHU5LWN6PQNGAJ4ULDPFLHL3OFRP3IBDUSVCA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.048331201 (83h17m33.61165421s) [noise=true , address=104.36.82.9:6016 , piece_id=MLPYRXQYKDFRHRQ4BKELVLTJD6ZMOHM7YSCS5O5HH6525IQOBPGQ, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.048701328 (83h17m33.612024333s) [noise=true , address=104.36.82.42:6832 , piece_id=TRXGPKOEU7HPFSEDKPJN3C6GJOBUZPBM6CNPZWS6ZLOHO2F7SSLA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.049050929 (83h17m33.612373939s) [noise=true , address=104.36.82.56:4252 , piece_id=KHT4ZUAQRO7NJ2IWH6FORUJSH5V2UFKTGYJETRL5SLYRU2D57V4A, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.049392658 (83h17m33.612715669s) [noise=true , address=79.127.219.48:4076 , piece_id=UQM6DRGET6FQFND5HU4CTJDIC4337XX6MK5NJV5XQPOK7AFCPSMQ, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.049754448 (83h17m33.613077451s) [noise=true , address=77.90.31.14:4688 , piece_id=GHO7ZNMRBXPWN34PPDFGKPOOA7L65YVO6AYHFFP5R4XCUGACY3DQ, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.050127175 (83h17m33.613450178s) [noise=true , address=104.36.82.53:4136 , piece_id=QK4OSCIQAD6IB3I4QK4KYU4MTZEOPDJC5GG6HLMCUAXGDNLMSFYA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.091864444 (83h17m33.655187435s) [noise=true , address=79.127.219.48:4076 , piece_id=UQM6DRGET6FQFND5HU4CTJDIC4337XX6MK5NJV5XQPOK7AFCPSMQ, seg_pos={0 0}, upload=66665]: Done uploading piece. err: ecclient: upload failed (node:1QeLZQsubmwYFwTPC9ipEgKJJvFjJ7VrXLC7QncaYoPyyFM4Fn, address:79.127.219.48:4076): protocol: storage node overloaded, request limit: 2000
Jun 22 03:40:58.161661930 (83h17m33.72498492s) [upload=66665]: Next returned nil segment
Jun 22 03:40:58.161681692 (83h17m33.725004695s) [upload=66665]: Waiting for error group managing segments...
Jun 22 03:40:58.190491904 (83h17m33.753814895s) [noise=true , address=104.36.82.11:6125 , piece_id=NP2722BQYMG5H7RQD3TLWYO23TFPT3HSBZFS6JRYYBFYBZWHEB5A, seg_pos={0 0}, upload=66665]: Done uploading piece. err: ecclient: upload failed (node:12vricofwx67FRaxAGLbmgT1TYyKkqfJbxdsyeEPL4R7kg8pKzT, address:104.36.82.11:6125): protocol: storage node overloaded, request limit: 2000
Jun 22 03:40:58.207750455 (83h17m33.77107344s) [noise=true , address=77.90.31.14:4688 , piece_id=GHO7ZNMRBXPWN34PPDFGKPOOA7L65YVO6AYHFFP5R4XCUGACY3DQ, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:40:58.531973385 (83h17m34.095296379s) [noise=true , address=64.49.13.169:5711 , piece_id=ROPI2UEYUINLDKQ4IEVWXTYGSU3PXWGXFI7TC2L675L2KENKWWEQ, seg_pos={0 0}, upload=66665]: Done uploading piece. err: ecclient: upload failed (node:1nxT1Ldr46HPrDRHQ2nrwEBZT5bEVoPAAfBz86ZYwUH13sqwHu, address:64.49.13.169:5711): protocol: storage node overloaded, request limit: 2000
Jun 22 03:40:58.990571568 (83h17m34.553894561s) [noise=true , address=104.36.82.9:6016 , piece_id=MLPYRXQYKDFRHRQ4BKELVLTJD6ZMOHM7YSCS5O5HH6525IQOBPGQ, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:00.325760684 (83h17m35.889083676s) [noise=true , address=104.36.82.14:6227 , piece_id=B4YO53N2MW6VCC5KHU5LWN6PQNGAJ4ULDPFLHL3OFRP3IBDUSVCA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:00.953756837 (83h17m36.517079837s) [noise=true , address=45.91.116.8:5058 , piece_id=FVHGWLW7PVRET7FKWLXJ6V6MUJKWVTFNOKLGIEQQI72ZXZTW74BA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:01.386979509 (83h17m36.950302499s) [noise=true , address=104.36.82.10:6112 , piece_id=56XUDWBP5SSSKOWDY2N2JV6PV63VXTROBOJQLZUUKGLI2ZBE7YMA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:01.689801010 (83h17m37.253124004s) [noise=true , address=104.36.82.3:5792 , piece_id=W2WE2U645R765EZI5CGYAOLJWIS6LQIGEFVNZUK4PS2IC6HEUILA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:01.856352835 (83h17m37.419675823s) [noise=true , address=104.36.82.5:5856 , piece_id=ZATJRQE4NARTU5JDY3XMUDSW5E6DTQZPTD5VQV5OIANWC2CSNARA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:03.717835535 (83h17m39.281158548s) [noise=true , address=104.36.82.42:6832 , piece_id=TRXGPKOEU7HPFSEDKPJN3C6GJOBUZPBM6CNPZWS6ZLOHO2F7SSLA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:03.961658935 (83h17m39.524981945s) [noise=true , address=79.127.220.97:4363 , piece_id=5HMOMDEQKEOHRF67UVYFYSKIMH2IRLMME6S7XKDNVAMAFHE5EO6Q, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:09.244412959 (83h17m44.807735964s) [noise=true , address=104.36.82.56:4252 , piece_id=KHT4ZUAQRO7NJ2IWH6FORUJSH5V2UFKTGYJETRL5SLYRU2D57V4A, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:14.056547620 (83h17m49.619870606s) [noise=true , address=104.36.82.53:4136 , piece_id=QK4OSCIQAD6IB3I4QK4KYU4MTZEOPDJC5GG6HLMCUAXGDNLMSFYA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:19.882456778 (83h17m55.445779771s) [noise=true , address=104.36.82.55:4217 , piece_id=6DKIHWLLTMCGBT63TAG234VGEPOX7QCTXLOBVQSG56LJS5YYSPPA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:28.035171487 (83h18m3.598494478s) [noise=true , address=104.36.82.52:4094 , piece_id=7AXGQ6CXE6H7GTURGOCHG3UBTV3VWLVNIXPNM2NAN7IR2RQCTRIA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:57.717732997 (83h18m33.281055985s) [noise=true , address=104.36.82.12:6158 , piece_id=VWTJS7XRTMBJFUNAYMZCJQ2EEMT5CVILS4JWMPUK5GIYPLQADAJQ, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:57.717762830 (83h18m33.281085829s) [seg_pos={0 0}, upload=66665]: Segment reached optimal threshold of 14 pieces.
Jun 22 03:41:57.718005914 (83h18m33.281328913s) [noise=true , address=104.36.82.54:4167 , piece_id=I4FIVN2M4OF7EARPGIY3X72D2DOARBQGIRU2W7KCCOMUEEMZ776Q, seg_pos={0 0}, upload=66665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hhT46NW5CQ7dWHRxy5MSjyKc3PiQqBifThT7dBmx2sgNvn8GZ): protocol: write tcp 159.223.143.39:56114->104.36.82.54:4167: use of closed network connection; write tcp 159.223.143.39:56114->104.36.82.54:4167: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:56114->104.36.82.54:4167: use of closed network connection
Jun 22 03:41:57.718019794 (83h18m33.281342793s) [seg_pos={0 0}, upload=66665]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jun 22 03:41:57.718095039 (83h18m33.281418038s) [upload=66665]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 22 03:41:57.718100732 (83h18m33.281423731s) [upload=66665]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 22 03:41:57.767433584 (83h18m33.330756651s) [upload=66665]: Done uploading segments...
Jun 22 03:41:57.767452412 (83h18m33.330775402s) [upload=66665]: Upload finished. err: <nil>
Duration: 13.155022004s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..] duration:
21.809745849s
FAIL (2025-06-22 03:42:51 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
4 (83h17m33.572378185s) [upload=66665]: Uploading segments...
Jun 22 03:40:58.009397890 (83h17m33.572720878s) [upload=66665]: Got next segment. Inline: false
Jun 22 03:40:58.009451249 (83h17m33.572774249s) [upload=66665]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 22 03:40:58.009457748 (83h17m33.572780755s) [upload=66665]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 22 03:40:58.044661723 (83h17m33.607984722s) [seg_pos={0 0}, upload=66665]: Begin upload segment...
Jun 22 03:40:58.044722729 (83h17m33.608045718s) [seg_pos={0 0}, upload=66665]: Done begin upload segment.
Jun 22 03:40:58.044746618 (83h17m33.608069618s) [noise=true , address=104.36.82.5:5856 , piece_id=ZATJRQE4NARTU5JDY3XMUDSW5E6DTQZPTD5VQV5OIANWC2CSNARA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.044784017 (83h17m33.608107034s) [noise=true , address=79.127.220.97:4363 , piece_id=5HMOMDEQKEOHRF67UVYFYSKIMH2IRLMME6S7XKDNVAMAFHE5EO6Q, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.045240611 (83h17m33.608563622s) [noise=true , address=104.36.82.52:4094 , piece_id=7AXGQ6CXE6H7GTURGOCHG3UBTV3VWLVNIXPNM2NAN7IR2RQCTRIA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.045699560 (83h17m33.609022565s) [noise=true , address=104.36.82.12:6158 , piece_id=VWTJS7XRTMBJFUNAYMZCJQ2EEMT5CVILS4JWMPUK5GIYPLQADAJQ, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.046153432 (83h17m33.609476443s) [noise=true , address=104.36.82.11:6125 , piece_id=NP2722BQYMG5H7RQD3TLWYO23TFPT3HSBZFS6JRYYBFYBZWHEB5A, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.046516869 (83h17m33.609839878s) [noise=true , address=64.49.13.169:5711 , piece_id=ROPI2UEYUINLDKQ4IEVWXTYGSU3PXWGXFI7TC2L675L2KENKWWEQ, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.046897154 (83h17m33.610220195s) [noise=true , address=104.36.82.54:4167 , piece_id=I4FIVN2M4OF7EARPGIY3X72D2DOARBQGIRU2W7KCCOMUEEMZ776Q, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.047287027 (83h17m33.610610033s) [noise=true , address=104.36.82.3:5792 , piece_id=W2WE2U645R765EZI5CGYAOLJWIS6LQIGEFVNZUK4PS2IC6HEUILA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.047564954 (83h17m33.610887959s) [noise=true , address=45.91.116.8:5058 , piece_id=FVHGWLW7PVRET7FKWLXJ6V6MUJKWVTFNOKLGIEQQI72ZXZTW74BA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.047691900 (83h17m33.611014907s) [noise=true , address=104.36.82.55:4217 , piece_id=6DKIHWLLTMCGBT63TAG234VGEPOX7QCTXLOBVQSG56LJS5YYSPPA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.047925386 (83h17m33.611248395s) [noise=true , address=104.36.82.10:6112 , piece_id=56XUDWBP5SSSKOWDY2N2JV6PV63VXTROBOJQLZUUKGLI2ZBE7YMA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.048123429 (83h17m33.611446434s) [noise=true , address=104.36.82.14:6227 , piece_id=B4YO53N2MW6VCC5KHU5LWN6PQNGAJ4ULDPFLHL3OFRP3IBDUSVCA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.048331201 (83h17m33.61165421s) [noise=true , address=104.36.82.9:6016 , piece_id=MLPYRXQYKDFRHRQ4BKELVLTJD6ZMOHM7YSCS5O5HH6525IQOBPGQ, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.048701328 (83h17m33.612024333s) [noise=true , address=104.36.82.42:6832 , piece_id=TRXGPKOEU7HPFSEDKPJN3C6GJOBUZPBM6CNPZWS6ZLOHO2F7SSLA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.049050929 (83h17m33.612373939s) [noise=true , address=104.36.82.56:4252 , piece_id=KHT4ZUAQRO7NJ2IWH6FORUJSH5V2UFKTGYJETRL5SLYRU2D57V4A, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.049392658 (83h17m33.612715669s) [noise=true , address=79.127.219.48:4076 , piece_id=UQM6DRGET6FQFND5HU4CTJDIC4337XX6MK5NJV5XQPOK7AFCPSMQ, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.049754448 (83h17m33.613077451s) [noise=true , address=77.90.31.14:4688 , piece_id=GHO7ZNMRBXPWN34PPDFGKPOOA7L65YVO6AYHFFP5R4XCUGACY3DQ, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.050127175 (83h17m33.613450178s) [noise=true , address=104.36.82.53:4136 , piece_id=QK4OSCIQAD6IB3I4QK4KYU4MTZEOPDJC5GG6HLMCUAXGDNLMSFYA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.091864444 (83h17m33.655187435s) [noise=true , address=79.127.219.48:4076 , piece_id=UQM6DRGET6FQFND5HU4CTJDIC4337XX6MK5NJV5XQPOK7AFCPSMQ, seg_pos={0 0}, upload=66665]: Done uploading piece. err: ecclient: upload failed (node:1QeLZQsubmwYFwTPC9ipEgKJJvFjJ7VrXLC7QncaYoPyyFM4Fn, address:79.127.219.48:4076): protocol: storage node overloaded, request limit: 2000
Jun 22 03:40:58.161661930 (83h17m33.72498492s) [upload=66665]: Next returned nil segment
Jun 22 03:40:58.161681692 (83h17m33.725004695s) [upload=66665]: Waiting for error group managing segments...
Jun 22 03:40:58.190491904 (83h17m33.753814895s) [noise=true , address=104.36.82.11:6125 , piece_id=NP2722BQYMG5H7RQD3TLWYO23TFPT3HSBZFS6JRYYBFYBZWHEB5A, seg_pos={0 0}, upload=66665]: Done uploading piece. err: ecclient: upload failed (node:12vricofwx67FRaxAGLbmgT1TYyKkqfJbxdsyeEPL4R7kg8pKzT, address:104.36.82.11:6125): protocol: storage node overloaded, request limit: 2000
Jun 22 03:40:58.207750455 (83h17m33.77107344s) [noise=true , address=77.90.31.14:4688 , piece_id=GHO7ZNMRBXPWN34PPDFGKPOOA7L65YVO6AYHFFP5R4XCUGACY3DQ, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:40:58.531973385 (83h17m34.095296379s) [noise=true , address=64.49.13.169:5711 , piece_id=ROPI2UEYUINLDKQ4IEVWXTYGSU3PXWGXFI7TC2L675L2KENKWWEQ, seg_pos={0 0}, upload=66665]: Done uploading piece. err: ecclient: upload failed (node:1nxT1Ldr46HPrDRHQ2nrwEBZT5bEVoPAAfBz86ZYwUH13sqwHu, address:64.49.13.169:5711): protocol: storage node overloaded, request limit: 2000
Jun 22 03:40:58.990571568 (83h17m34.553894561s) [noise=true , address=104.36.82.9:6016 , piece_id=MLPYRXQYKDFRHRQ4BKELVLTJD6ZMOHM7YSCS5O5HH6525IQOBPGQ, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:00.325760684 (83h17m35.889083676s) [noise=true , address=104.36.82.14:6227 , piece_id=B4YO53N2MW6VCC5KHU5LWN6PQNGAJ4ULDPFLHL3OFRP3IBDUSVCA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:00.953756837 (83h17m36.517079837s) [noise=true , address=45.91.116.8:5058 , piece_id=FVHGWLW7PVRET7FKWLXJ6V6MUJKWVTFNOKLGIEQQI72ZXZTW74BA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:01.386979509 (83h17m36.950302499s) [noise=true , address=104.36.82.10:6112 , piece_id=56XUDWBP5SSSKOWDY2N2JV6PV63VXTROBOJQLZUUKGLI2ZBE7YMA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:01.689801010 (83h17m37.253124004s) [noise=true , address=104.36.82.3:5792 , piece_id=W2WE2U645R765EZI5CGYAOLJWIS6LQIGEFVNZUK4PS2IC6HEUILA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:01.856352835 (83h17m37.419675823s) [noise=true , address=104.36.82.5:5856 , piece_id=ZATJRQE4NARTU5JDY3XMUDSW5E6DTQZPTD5VQV5OIANWC2CSNARA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:03.717835535 (83h17m39.281158548s) [noise=true , address=104.36.82.42:6832 , piece_id=TRXGPKOEU7HPFSEDKPJN3C6GJOBUZPBM6CNPZWS6ZLOHO2F7SSLA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:03.961658935 (83h17m39.524981945s) [noise=true , address=79.127.220.97:4363 , piece_id=5HMOMDEQKEOHRF67UVYFYSKIMH2IRLMME6S7XKDNVAMAFHE5EO6Q, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:09.244412959 (83h17m44.807735964s) [noise=true , address=104.36.82.56:4252 , piece_id=KHT4ZUAQRO7NJ2IWH6FORUJSH5V2UFKTGYJETRL5SLYRU2D57V4A, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:14.056547620 (83h17m49.619870606s) [noise=true , address=104.36.82.53:4136 , piece_id=QK4OSCIQAD6IB3I4QK4KYU4MTZEOPDJC5GG6HLMCUAXGDNLMSFYA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:19.882456778 (83h17m55.445779771s) [noise=true , address=104.36.82.55:4217 , piece_id=6DKIHWLLTMCGBT63TAG234VGEPOX7QCTXLOBVQSG56LJS5YYSPPA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:28.035171487 (83h18m3.598494478s) [noise=true , address=104.36.82.52:4094 , piece_id=7AXGQ6CXE6H7GTURGOCHG3UBTV3VWLVNIXPNM2NAN7IR2RQCTRIA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:57.717732997 (83h18m33.281055985s) [noise=true , address=104.36.82.12:6158 , piece_id=VWTJS7XRTMBJFUNAYMZCJQ2EEMT5CVILS4JWMPUK5GIYPLQADAJQ, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:57.717762830 (83h18m33.281085829s) [seg_pos={0 0}, upload=66665]: Segment reached optimal threshold of 14 pieces.
Jun 22 03:41:57.718005914 (83h18m33.281328913s) [noise=true , address=104.36.82.54:4167 , piece_id=I4FIVN2M4OF7EARPGIY3X72D2DOARBQGIRU2W7KCCOMUEEMZ776Q, seg_pos={0 0}, upload=66665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hhT46NW5CQ7dWHRxy5MSjyKc3PiQqBifThT7dBmx2sgNvn8GZ): protocol: write tcp 159.223.143.39:56114->104.36.82.54:4167: use of closed network connection; write tcp 159.223.143.39:56114->104.36.82.54:4167: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:56114->104.36.82.54:4167: use of closed network connection
Jun 22 03:41:57.718019794 (83h18m33.281342793s) [seg_pos={0 0}, upload=66665]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jun 22 03:41:57.718095039 (83h18m33.281418038s) [upload=66665]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 22 03:41:57.718100732 (83h18m33.281423731s) [upload=66665]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 22 03:41:57.767433584 (83h18m33.330756651s) [upload=66665]: Done uploading segments...
Jun 22 03:41:57.767452412 (83h18m33.330775402s) [upload=66665]: Upload finished. err: <nil>
Duration: 21.809774295s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.] duration:
16.727728404s
FAIL (2025-06-22 03:42:14 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
4 (83h17m33.572378185s) [upload=66665]: Uploading segments...
Jun 22 03:40:58.009397890 (83h17m33.572720878s) [upload=66665]: Got next segment. Inline: false
Jun 22 03:40:58.009451249 (83h17m33.572774249s) [upload=66665]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 22 03:40:58.009457748 (83h17m33.572780755s) [upload=66665]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 22 03:40:58.044661723 (83h17m33.607984722s) [seg_pos={0 0}, upload=66665]: Begin upload segment...
Jun 22 03:40:58.044722729 (83h17m33.608045718s) [seg_pos={0 0}, upload=66665]: Done begin upload segment.
Jun 22 03:40:58.044746618 (83h17m33.608069618s) [noise=true , address=104.36.82.5:5856 , piece_id=ZATJRQE4NARTU5JDY3XMUDSW5E6DTQZPTD5VQV5OIANWC2CSNARA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.044784017 (83h17m33.608107034s) [noise=true , address=79.127.220.97:4363 , piece_id=5HMOMDEQKEOHRF67UVYFYSKIMH2IRLMME6S7XKDNVAMAFHE5EO6Q, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.045240611 (83h17m33.608563622s) [noise=true , address=104.36.82.52:4094 , piece_id=7AXGQ6CXE6H7GTURGOCHG3UBTV3VWLVNIXPNM2NAN7IR2RQCTRIA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.045699560 (83h17m33.609022565s) [noise=true , address=104.36.82.12:6158 , piece_id=VWTJS7XRTMBJFUNAYMZCJQ2EEMT5CVILS4JWMPUK5GIYPLQADAJQ, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.046153432 (83h17m33.609476443s) [noise=true , address=104.36.82.11:6125 , piece_id=NP2722BQYMG5H7RQD3TLWYO23TFPT3HSBZFS6JRYYBFYBZWHEB5A, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.046516869 (83h17m33.609839878s) [noise=true , address=64.49.13.169:5711 , piece_id=ROPI2UEYUINLDKQ4IEVWXTYGSU3PXWGXFI7TC2L675L2KENKWWEQ, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.046897154 (83h17m33.610220195s) [noise=true , address=104.36.82.54:4167 , piece_id=I4FIVN2M4OF7EARPGIY3X72D2DOARBQGIRU2W7KCCOMUEEMZ776Q, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.047287027 (83h17m33.610610033s) [noise=true , address=104.36.82.3:5792 , piece_id=W2WE2U645R765EZI5CGYAOLJWIS6LQIGEFVNZUK4PS2IC6HEUILA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.047564954 (83h17m33.610887959s) [noise=true , address=45.91.116.8:5058 , piece_id=FVHGWLW7PVRET7FKWLXJ6V6MUJKWVTFNOKLGIEQQI72ZXZTW74BA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.047691900 (83h17m33.611014907s) [noise=true , address=104.36.82.55:4217 , piece_id=6DKIHWLLTMCGBT63TAG234VGEPOX7QCTXLOBVQSG56LJS5YYSPPA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.047925386 (83h17m33.611248395s) [noise=true , address=104.36.82.10:6112 , piece_id=56XUDWBP5SSSKOWDY2N2JV6PV63VXTROBOJQLZUUKGLI2ZBE7YMA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.048123429 (83h17m33.611446434s) [noise=true , address=104.36.82.14:6227 , piece_id=B4YO53N2MW6VCC5KHU5LWN6PQNGAJ4ULDPFLHL3OFRP3IBDUSVCA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.048331201 (83h17m33.61165421s) [noise=true , address=104.36.82.9:6016 , piece_id=MLPYRXQYKDFRHRQ4BKELVLTJD6ZMOHM7YSCS5O5HH6525IQOBPGQ, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.048701328 (83h17m33.612024333s) [noise=true , address=104.36.82.42:6832 , piece_id=TRXGPKOEU7HPFSEDKPJN3C6GJOBUZPBM6CNPZWS6ZLOHO2F7SSLA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.049050929 (83h17m33.612373939s) [noise=true , address=104.36.82.56:4252 , piece_id=KHT4ZUAQRO7NJ2IWH6FORUJSH5V2UFKTGYJETRL5SLYRU2D57V4A, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.049392658 (83h17m33.612715669s) [noise=true , address=79.127.219.48:4076 , piece_id=UQM6DRGET6FQFND5HU4CTJDIC4337XX6MK5NJV5XQPOK7AFCPSMQ, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.049754448 (83h17m33.613077451s) [noise=true , address=77.90.31.14:4688 , piece_id=GHO7ZNMRBXPWN34PPDFGKPOOA7L65YVO6AYHFFP5R4XCUGACY3DQ, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.050127175 (83h17m33.613450178s) [noise=true , address=104.36.82.53:4136 , piece_id=QK4OSCIQAD6IB3I4QK4KYU4MTZEOPDJC5GG6HLMCUAXGDNLMSFYA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.091864444 (83h17m33.655187435s) [noise=true , address=79.127.219.48:4076 , piece_id=UQM6DRGET6FQFND5HU4CTJDIC4337XX6MK5NJV5XQPOK7AFCPSMQ, seg_pos={0 0}, upload=66665]: Done uploading piece. err: ecclient: upload failed (node:1QeLZQsubmwYFwTPC9ipEgKJJvFjJ7VrXLC7QncaYoPyyFM4Fn, address:79.127.219.48:4076): protocol: storage node overloaded, request limit: 2000
Jun 22 03:40:58.161661930 (83h17m33.72498492s) [upload=66665]: Next returned nil segment
Jun 22 03:40:58.161681692 (83h17m33.725004695s) [upload=66665]: Waiting for error group managing segments...
Jun 22 03:40:58.190491904 (83h17m33.753814895s) [noise=true , address=104.36.82.11:6125 , piece_id=NP2722BQYMG5H7RQD3TLWYO23TFPT3HSBZFS6JRYYBFYBZWHEB5A, seg_pos={0 0}, upload=66665]: Done uploading piece. err: ecclient: upload failed (node:12vricofwx67FRaxAGLbmgT1TYyKkqfJbxdsyeEPL4R7kg8pKzT, address:104.36.82.11:6125): protocol: storage node overloaded, request limit: 2000
Jun 22 03:40:58.207750455 (83h17m33.77107344s) [noise=true , address=77.90.31.14:4688 , piece_id=GHO7ZNMRBXPWN34PPDFGKPOOA7L65YVO6AYHFFP5R4XCUGACY3DQ, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:40:58.531973385 (83h17m34.095296379s) [noise=true , address=64.49.13.169:5711 , piece_id=ROPI2UEYUINLDKQ4IEVWXTYGSU3PXWGXFI7TC2L675L2KENKWWEQ, seg_pos={0 0}, upload=66665]: Done uploading piece. err: ecclient: upload failed (node:1nxT1Ldr46HPrDRHQ2nrwEBZT5bEVoPAAfBz86ZYwUH13sqwHu, address:64.49.13.169:5711): protocol: storage node overloaded, request limit: 2000
Jun 22 03:40:58.990571568 (83h17m34.553894561s) [noise=true , address=104.36.82.9:6016 , piece_id=MLPYRXQYKDFRHRQ4BKELVLTJD6ZMOHM7YSCS5O5HH6525IQOBPGQ, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:00.325760684 (83h17m35.889083676s) [noise=true , address=104.36.82.14:6227 , piece_id=B4YO53N2MW6VCC5KHU5LWN6PQNGAJ4ULDPFLHL3OFRP3IBDUSVCA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:00.953756837 (83h17m36.517079837s) [noise=true , address=45.91.116.8:5058 , piece_id=FVHGWLW7PVRET7FKWLXJ6V6MUJKWVTFNOKLGIEQQI72ZXZTW74BA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:01.386979509 (83h17m36.950302499s) [noise=true , address=104.36.82.10:6112 , piece_id=56XUDWBP5SSSKOWDY2N2JV6PV63VXTROBOJQLZUUKGLI2ZBE7YMA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:01.689801010 (83h17m37.253124004s) [noise=true , address=104.36.82.3:5792 , piece_id=W2WE2U645R765EZI5CGYAOLJWIS6LQIGEFVNZUK4PS2IC6HEUILA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:01.856352835 (83h17m37.419675823s) [noise=true , address=104.36.82.5:5856 , piece_id=ZATJRQE4NARTU5JDY3XMUDSW5E6DTQZPTD5VQV5OIANWC2CSNARA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:03.717835535 (83h17m39.281158548s) [noise=true , address=104.36.82.42:6832 , piece_id=TRXGPKOEU7HPFSEDKPJN3C6GJOBUZPBM6CNPZWS6ZLOHO2F7SSLA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:03.961658935 (83h17m39.524981945s) [noise=true , address=79.127.220.97:4363 , piece_id=5HMOMDEQKEOHRF67UVYFYSKIMH2IRLMME6S7XKDNVAMAFHE5EO6Q, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:09.244412959 (83h17m44.807735964s) [noise=true , address=104.36.82.56:4252 , piece_id=KHT4ZUAQRO7NJ2IWH6FORUJSH5V2UFKTGYJETRL5SLYRU2D57V4A, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:14.056547620 (83h17m49.619870606s) [noise=true , address=104.36.82.53:4136 , piece_id=QK4OSCIQAD6IB3I4QK4KYU4MTZEOPDJC5GG6HLMCUAXGDNLMSFYA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:19.882456778 (83h17m55.445779771s) [noise=true , address=104.36.82.55:4217 , piece_id=6DKIHWLLTMCGBT63TAG234VGEPOX7QCTXLOBVQSG56LJS5YYSPPA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:28.035171487 (83h18m3.598494478s) [noise=true , address=104.36.82.52:4094 , piece_id=7AXGQ6CXE6H7GTURGOCHG3UBTV3VWLVNIXPNM2NAN7IR2RQCTRIA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:57.717732997 (83h18m33.281055985s) [noise=true , address=104.36.82.12:6158 , piece_id=VWTJS7XRTMBJFUNAYMZCJQ2EEMT5CVILS4JWMPUK5GIYPLQADAJQ, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:57.717762830 (83h18m33.281085829s) [seg_pos={0 0}, upload=66665]: Segment reached optimal threshold of 14 pieces.
Jun 22 03:41:57.718005914 (83h18m33.281328913s) [noise=true , address=104.36.82.54:4167 , piece_id=I4FIVN2M4OF7EARPGIY3X72D2DOARBQGIRU2W7KCCOMUEEMZ776Q, seg_pos={0 0}, upload=66665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hhT46NW5CQ7dWHRxy5MSjyKc3PiQqBifThT7dBmx2sgNvn8GZ): protocol: write tcp 159.223.143.39:56114->104.36.82.54:4167: use of closed network connection; write tcp 159.223.143.39:56114->104.36.82.54:4167: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:56114->104.36.82.54:4167: use of closed network connection
Jun 22 03:41:57.718019794 (83h18m33.281342793s) [seg_pos={0 0}, upload=66665]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jun 22 03:41:57.718095039 (83h18m33.281418038s) [upload=66665]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 22 03:41:57.718100732 (83h18m33.281423731s) [upload=66665]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 22 03:41:57.767433584 (83h18m33.330756651s) [upload=66665]: Done uploading segments...
Jun 22 03:41:57.767452412 (83h18m33.330775402s) [upload=66665]: Upload finished. err: <nil>
Duration: 16.727748379s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
1m0.004687615s
FAIL (2025-06-22 03:41:57 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
4 (83h17m33.572378185s) [upload=66665]: Uploading segments...
Jun 22 03:40:58.009397890 (83h17m33.572720878s) [upload=66665]: Got next segment. Inline: false
Jun 22 03:40:58.009451249 (83h17m33.572774249s) [upload=66665]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 22 03:40:58.009457748 (83h17m33.572780755s) [upload=66665]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 22 03:40:58.044661723 (83h17m33.607984722s) [seg_pos={0 0}, upload=66665]: Begin upload segment...
Jun 22 03:40:58.044722729 (83h17m33.608045718s) [seg_pos={0 0}, upload=66665]: Done begin upload segment.
Jun 22 03:40:58.044746618 (83h17m33.608069618s) [noise=true , address=104.36.82.5:5856 , piece_id=ZATJRQE4NARTU5JDY3XMUDSW5E6DTQZPTD5VQV5OIANWC2CSNARA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.044784017 (83h17m33.608107034s) [noise=true , address=79.127.220.97:4363 , piece_id=5HMOMDEQKEOHRF67UVYFYSKIMH2IRLMME6S7XKDNVAMAFHE5EO6Q, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.045240611 (83h17m33.608563622s) [noise=true , address=104.36.82.52:4094 , piece_id=7AXGQ6CXE6H7GTURGOCHG3UBTV3VWLVNIXPNM2NAN7IR2RQCTRIA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.045699560 (83h17m33.609022565s) [noise=true , address=104.36.82.12:6158 , piece_id=VWTJS7XRTMBJFUNAYMZCJQ2EEMT5CVILS4JWMPUK5GIYPLQADAJQ, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.046153432 (83h17m33.609476443s) [noise=true , address=104.36.82.11:6125 , piece_id=NP2722BQYMG5H7RQD3TLWYO23TFPT3HSBZFS6JRYYBFYBZWHEB5A, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.046516869 (83h17m33.609839878s) [noise=true , address=64.49.13.169:5711 , piece_id=ROPI2UEYUINLDKQ4IEVWXTYGSU3PXWGXFI7TC2L675L2KENKWWEQ, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.046897154 (83h17m33.610220195s) [noise=true , address=104.36.82.54:4167 , piece_id=I4FIVN2M4OF7EARPGIY3X72D2DOARBQGIRU2W7KCCOMUEEMZ776Q, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.047287027 (83h17m33.610610033s) [noise=true , address=104.36.82.3:5792 , piece_id=W2WE2U645R765EZI5CGYAOLJWIS6LQIGEFVNZUK4PS2IC6HEUILA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.047564954 (83h17m33.610887959s) [noise=true , address=45.91.116.8:5058 , piece_id=FVHGWLW7PVRET7FKWLXJ6V6MUJKWVTFNOKLGIEQQI72ZXZTW74BA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.047691900 (83h17m33.611014907s) [noise=true , address=104.36.82.55:4217 , piece_id=6DKIHWLLTMCGBT63TAG234VGEPOX7QCTXLOBVQSG56LJS5YYSPPA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.047925386 (83h17m33.611248395s) [noise=true , address=104.36.82.10:6112 , piece_id=56XUDWBP5SSSKOWDY2N2JV6PV63VXTROBOJQLZUUKGLI2ZBE7YMA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.048123429 (83h17m33.611446434s) [noise=true , address=104.36.82.14:6227 , piece_id=B4YO53N2MW6VCC5KHU5LWN6PQNGAJ4ULDPFLHL3OFRP3IBDUSVCA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.048331201 (83h17m33.61165421s) [noise=true , address=104.36.82.9:6016 , piece_id=MLPYRXQYKDFRHRQ4BKELVLTJD6ZMOHM7YSCS5O5HH6525IQOBPGQ, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.048701328 (83h17m33.612024333s) [noise=true , address=104.36.82.42:6832 , piece_id=TRXGPKOEU7HPFSEDKPJN3C6GJOBUZPBM6CNPZWS6ZLOHO2F7SSLA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.049050929 (83h17m33.612373939s) [noise=true , address=104.36.82.56:4252 , piece_id=KHT4ZUAQRO7NJ2IWH6FORUJSH5V2UFKTGYJETRL5SLYRU2D57V4A, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.049392658 (83h17m33.612715669s) [noise=true , address=79.127.219.48:4076 , piece_id=UQM6DRGET6FQFND5HU4CTJDIC4337XX6MK5NJV5XQPOK7AFCPSMQ, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.049754448 (83h17m33.613077451s) [noise=true , address=77.90.31.14:4688 , piece_id=GHO7ZNMRBXPWN34PPDFGKPOOA7L65YVO6AYHFFP5R4XCUGACY3DQ, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.050127175 (83h17m33.613450178s) [noise=true , address=104.36.82.53:4136 , piece_id=QK4OSCIQAD6IB3I4QK4KYU4MTZEOPDJC5GG6HLMCUAXGDNLMSFYA, seg_pos={0 0}, upload=66665]: Uploading piece...
Jun 22 03:40:58.091864444 (83h17m33.655187435s) [noise=true , address=79.127.219.48:4076 , piece_id=UQM6DRGET6FQFND5HU4CTJDIC4337XX6MK5NJV5XQPOK7AFCPSMQ, seg_pos={0 0}, upload=66665]: Done uploading piece. err: ecclient: upload failed (node:1QeLZQsubmwYFwTPC9ipEgKJJvFjJ7VrXLC7QncaYoPyyFM4Fn, address:79.127.219.48:4076): protocol: storage node overloaded, request limit: 2000
Jun 22 03:40:58.161661930 (83h17m33.72498492s) [upload=66665]: Next returned nil segment
Jun 22 03:40:58.161681692 (83h17m33.725004695s) [upload=66665]: Waiting for error group managing segments...
Jun 22 03:40:58.190491904 (83h17m33.753814895s) [noise=true , address=104.36.82.11:6125 , piece_id=NP2722BQYMG5H7RQD3TLWYO23TFPT3HSBZFS6JRYYBFYBZWHEB5A, seg_pos={0 0}, upload=66665]: Done uploading piece. err: ecclient: upload failed (node:12vricofwx67FRaxAGLbmgT1TYyKkqfJbxdsyeEPL4R7kg8pKzT, address:104.36.82.11:6125): protocol: storage node overloaded, request limit: 2000
Jun 22 03:40:58.207750455 (83h17m33.77107344s) [noise=true , address=77.90.31.14:4688 , piece_id=GHO7ZNMRBXPWN34PPDFGKPOOA7L65YVO6AYHFFP5R4XCUGACY3DQ, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:40:58.531973385 (83h17m34.095296379s) [noise=true , address=64.49.13.169:5711 , piece_id=ROPI2UEYUINLDKQ4IEVWXTYGSU3PXWGXFI7TC2L675L2KENKWWEQ, seg_pos={0 0}, upload=66665]: Done uploading piece. err: ecclient: upload failed (node:1nxT1Ldr46HPrDRHQ2nrwEBZT5bEVoPAAfBz86ZYwUH13sqwHu, address:64.49.13.169:5711): protocol: storage node overloaded, request limit: 2000
Jun 22 03:40:58.990571568 (83h17m34.553894561s) [noise=true , address=104.36.82.9:6016 , piece_id=MLPYRXQYKDFRHRQ4BKELVLTJD6ZMOHM7YSCS5O5HH6525IQOBPGQ, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:00.325760684 (83h17m35.889083676s) [noise=true , address=104.36.82.14:6227 , piece_id=B4YO53N2MW6VCC5KHU5LWN6PQNGAJ4ULDPFLHL3OFRP3IBDUSVCA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:00.953756837 (83h17m36.517079837s) [noise=true , address=45.91.116.8:5058 , piece_id=FVHGWLW7PVRET7FKWLXJ6V6MUJKWVTFNOKLGIEQQI72ZXZTW74BA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:01.386979509 (83h17m36.950302499s) [noise=true , address=104.36.82.10:6112 , piece_id=56XUDWBP5SSSKOWDY2N2JV6PV63VXTROBOJQLZUUKGLI2ZBE7YMA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:01.689801010 (83h17m37.253124004s) [noise=true , address=104.36.82.3:5792 , piece_id=W2WE2U645R765EZI5CGYAOLJWIS6LQIGEFVNZUK4PS2IC6HEUILA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:01.856352835 (83h17m37.419675823s) [noise=true , address=104.36.82.5:5856 , piece_id=ZATJRQE4NARTU5JDY3XMUDSW5E6DTQZPTD5VQV5OIANWC2CSNARA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:03.717835535 (83h17m39.281158548s) [noise=true , address=104.36.82.42:6832 , piece_id=TRXGPKOEU7HPFSEDKPJN3C6GJOBUZPBM6CNPZWS6ZLOHO2F7SSLA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:03.961658935 (83h17m39.524981945s) [noise=true , address=79.127.220.97:4363 , piece_id=5HMOMDEQKEOHRF67UVYFYSKIMH2IRLMME6S7XKDNVAMAFHE5EO6Q, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:09.244412959 (83h17m44.807735964s) [noise=true , address=104.36.82.56:4252 , piece_id=KHT4ZUAQRO7NJ2IWH6FORUJSH5V2UFKTGYJETRL5SLYRU2D57V4A, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:14.056547620 (83h17m49.619870606s) [noise=true , address=104.36.82.53:4136 , piece_id=QK4OSCIQAD6IB3I4QK4KYU4MTZEOPDJC5GG6HLMCUAXGDNLMSFYA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:19.882456778 (83h17m55.445779771s) [noise=true , address=104.36.82.55:4217 , piece_id=6DKIHWLLTMCGBT63TAG234VGEPOX7QCTXLOBVQSG56LJS5YYSPPA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:28.035171487 (83h18m3.598494478s) [noise=true , address=104.36.82.52:4094 , piece_id=7AXGQ6CXE6H7GTURGOCHG3UBTV3VWLVNIXPNM2NAN7IR2RQCTRIA, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:57.717732997 (83h18m33.281055985s) [noise=true , address=104.36.82.12:6158 , piece_id=VWTJS7XRTMBJFUNAYMZCJQ2EEMT5CVILS4JWMPUK5GIYPLQADAJQ, seg_pos={0 0}, upload=66665]: Done uploading piece. err: <nil>
Jun 22 03:41:57.717762830 (83h18m33.281085829s) [seg_pos={0 0}, upload=66665]: Segment reached optimal threshold of 14 pieces.
Jun 22 03:41:57.718005914 (83h18m33.281328913s) [noise=true , address=104.36.82.54:4167 , piece_id=I4FIVN2M4OF7EARPGIY3X72D2DOARBQGIRU2W7KCCOMUEEMZ776Q, seg_pos={0 0}, upload=66665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hhT46NW5CQ7dWHRxy5MSjyKc3PiQqBifThT7dBmx2sgNvn8GZ): protocol: write tcp 159.223.143.39:56114->104.36.82.54:4167: use of closed network connection; write tcp 159.223.143.39:56114->104.36.82.54:4167: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:56114->104.36.82.54:4167: use of closed network connection
Jun 22 03:41:57.718019794 (83h18m33.281342793s) [seg_pos={0 0}, upload=66665]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jun 22 03:41:57.718095039 (83h18m33.281418038s) [upload=66665]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 22 03:41:57.718100732 (83h18m33.281423731s) [upload=66665]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 22 03:41:57.767433584 (83h18m33.330756651s) [upload=66665]: Done uploading segments...
Jun 22 03:41:57.767452412 (83h18m33.330775402s) [upload=66665]: Upload finished. err: <nil>
Duration: 1m0.004703627s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
7.206741378s
FAIL (2025-06-22 03:18:59 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 7.206816319s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
12.143284434s
FAIL (2025-06-22 03:17:54 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 12.143305602s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
13.716983804s
FAIL (2025-06-22 03:16:58 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 13.717015409s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................] duration:
16.78432061s
FAIL (2025-06-22 03:16:31 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 16.784354653s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
48.158329981s
FAIL (2025-06-22 03:15:53 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 48.158729599s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................] duration:
21.405270971s
FAIL (2025-06-22 03:14:14 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 21.405289065s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [......X.....................................................] duration:
26.583659882s
FAIL (2025-06-22 03:13:52 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 26.583707282s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.......X....................................................] duration:
20.038390732s
FAIL (2025-06-22 03:13:04 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 20.038412748s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [........X...................................................] duration:
18.51041159s
FAIL (2025-06-22 03:12:27 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 18.510435369s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.........X..................................................] duration:
32.730542631s
FAIL (2025-06-22 03:11:54 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 32.730584453s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [..........X.................................................] duration:
21.075848249s
FAIL (2025-06-22 03:10:29 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 21.075873371s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [...........X................................................] duration:
15.440282275s
FAIL (2025-06-22 03:09:17 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 15.440307547s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [............X...............................................] duration:
12.742721674s
FAIL (2025-06-22 03:08:22 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 12.742746828s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.............X..............................................] duration:
24.99335575s
FAIL (2025-06-22 03:07:31 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 24.993380563s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [..............X.............................................] duration:
10.032267455s
FAIL (2025-06-22 03:06:49 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 10.032283351s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [...............X............................................] duration:
22.137176577s
FAIL (2025-06-22 03:06:39 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 22.13719902s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [................X...........................................] duration:
22.154187064s
FAIL (2025-06-22 03:05:35 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 22.154365368s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [X................X..........................................] duration:
9.903442632s
FAIL (2025-06-22 03:04:34 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 9.903469707s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [.X................X.........................................] duration:
7.478559618s
FAIL (2025-06-22 03:04:00 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 7.478679232s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [..X................X........................................] duration:
11.883551441s
FAIL (2025-06-22 03:03:18 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 11.883572583s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [...X................X.......................................] duration:
18.742620299s
FAIL (2025-06-22 03:02:53 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 18.742732155s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [....X................X......................................] duration:
12.707100569s
FAIL (2025-06-22 03:02:16 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 12.707121056s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [.....X................X.....................................] duration:
14.280679457s
FAIL (2025-06-22 03:01:16 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 14.280704609s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [......X................X....................................] duration:
31.800966559s
FAIL (2025-06-22 03:00:40 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 31.80098908s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [.......X................X...................................] duration:
41.174063632s
FAIL (2025-06-22 02:59:46 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 41.174110841s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [........X................X..................................] duration:
10.342894106s
FAIL (2025-06-22 02:58:11 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 10.342930279s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [.........X................X.................................] duration:
22.415780986s
FAIL (2025-06-22 02:57:34 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 22.415852882s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [..........X................X................................] duration:
51.2142647s
FAIL (2025-06-22 02:56:54 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 51.21429732s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [...........X................X...............................] duration:
20.588237312s
FAIL (2025-06-22 02:55:55 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 20.588258504s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [............X................X..............................] duration:
19.209871724s
FAIL (2025-06-22 02:55:26 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 19.209955816s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [.............X................X.............................] duration:
7.576856851s
FAIL (2025-06-22 02:54:22 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 7.576879272s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [..............X................X............................] duration:
16.663604676s
FAIL (2025-06-22 02:54:04 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 16.663627537s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [...............X................X...........................] duration:
19.541760324s
FAIL (2025-06-22 02:53:47 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 19.541805628s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [................X................X..........................] duration:
9.797938591s
FAIL (2025-06-22 02:53:11 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 9.797973189s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [.................X................X.........................] duration:
22.618900226s
FAIL (2025-06-22 02:52:53 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 22.619434522s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [..................X................X........................] duration:
25.056774435s
FAIL (2025-06-22 02:51:41 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 25.056798823s
core
uplink
us-select-1
us1-placement-20
|