Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [..............................................X.............] duration:
2.533102364s trace:
https://jaeger.dev.storj.io/trace/43abdc33eb6dc242
FAIL (2026-01-31 21:42:34 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: manager closed: closed: read tcp 159.223.143.39:47078->35.207.121.91:7777: read: connection reset by peer
======LOGS======
.149.201.147:4005 , piece_id=65O63ETGC35NG5C3OWPZGQKTYRWJQ7O73P26ZC3ZFKYECETCQ3YA, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.683232901 (389h38m43.166690107s) [noise=true , address=185.167.204.206:4072 , piece_id=4VVQE6RWCILAA4JTRKSFXKN677CXXE2JFVL3NYHC7K6R54237PXQ, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.683709179 (389h38m43.167166383s) [noise=true , address=89.149.201.147:4004 , piece_id=METSYHH76QGQFFGUDDPIYTEGFZX5DRBY2LB3FTULP5PQ26ZJA7AQ, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.683724493 (389h38m43.167181697s) [noise=true , address=89.149.201.147:4010 , piece_id=RG7F5CWJFUAASYE7QS6Y4PGPRT45SADGUOW3W7M5I5E5G7L34FDA, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.684173473 (389h38m43.16763068s) [noise=true , address=185.167.204.206:4037 , piece_id=QSQ4ULBDAHRSXUJAWD4PTNVRYXO4F2MY626IKB2FDHKPCCGULXWA, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.684694161 (389h38m43.168151369s) [noise=true , address=185.167.204.205:4017 , piece_id=3LM3MACH5OSCSALKAUIOVRSBCX5HHX2VG7RTQZ6BJ4ALHRQOL7KQ, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.686216206 (389h38m43.169673413s) [noise=true , address=81.171.14.70:4034 , piece_id=QQWI3YYGGFGITTEPDDHEYGDLAGPOMHQ5NYG2MLWP6VC5ZJI77BUQ, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.688413710 (389h38m43.171870916s) [noise=true , address=89.149.201.147:4009 , piece_id=ET7TF7LSLL6NAIYI7DTWXOCWSL2OSQ46DAFGQPM4D7MZC4KL3MTA, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.690659778 (389h38m43.174116984s) [noise=true , address=185.167.204.205:4009 , piece_id=L6JZ2XWT2MLGXCWIA55LFTZ2WE5TGBCFTJBWRTVPUTHCEYCOX24A, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.692330711 (389h38m43.175787902s) [noise=true , address=89.149.201.147:4019 , piece_id=SE62AS4I6LN4AA7CSBUK7FWZJTDHUIN56FNTASDN3Y64PVNFL5VA, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:50.141023513 (389h38m43.624480718s) [upload=405070]: Next returned nil segment
Jan 31 21:33:50.141047430 (389h38m43.624504638s) [upload=405070]: Waiting for error group managing segments...
Jan 31 21:33:50.324609708 (389h38m43.808066905s) [noise=true , address=185.167.204.205:4017 , piece_id=3LM3MACH5OSCSALKAUIOVRSBCX5HHX2VG7RTQZ6BJ4ALHRQOL7KQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.325388947 (389h38m43.808846162s) [noise=true , address=81.171.14.70:4034 , piece_id=QQWI3YYGGFGITTEPDDHEYGDLAGPOMHQ5NYG2MLWP6VC5ZJI77BUQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.341012709 (389h38m43.824469907s) [noise=true , address=185.167.204.205:4023 , piece_id=WHKCMG2FOY5XSLLHJN5EWKFKSOYXFS3OBYFO4IGN26Y4DYE3H42Q, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.344607851 (389h38m43.828065048s) [noise=true , address=89.149.201.147:4019 , piece_id=SE62AS4I6LN4AA7CSBUK7FWZJTDHUIN56FNTASDN3Y64PVNFL5VA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.345220277 (389h38m43.828677475s) [noise=true , address=185.167.204.206:4061 , piece_id=OIFH6PI53RPJJ44BDBRYI4GYQYLLHLV4633VJSDZB4U2LRUMEATA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.345872439 (389h38m43.829329637s) [noise=true , address=89.149.201.147:4014 , piece_id=FDEIWM2YPK2NHAOHY5NE74Q6XB7YJLGTINJ63KADKRLYPBY2CYYQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.347218937 (389h38m43.830676136s) [noise=true , address=185.167.204.205:4027 , piece_id=JMAGQ7KEDUDN7MBXQEGQGVB5NNWUO5HU7BZATHBORQ7EJN46UF6A, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.352289470 (389h38m43.835746665s) [noise=true , address=185.167.204.206:4066 , piece_id=HCYOMLOPP6PLWUD3H5VQ3PGUJUIG7RQAS2QKEPUOCRGWPCYSVUJQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.352722623 (389h38m43.836179824s) [noise=true , address=185.167.204.206:4072 , piece_id=4VVQE6RWCILAA4JTRKSFXKN677CXXE2JFVL3NYHC7K6R54237PXQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.353039283 (389h38m43.83649647s) [noise=true , address=185.167.204.206:4070 , piece_id=4OFRPTUQ34OGUGGBT6A4J64BWZP4PFKKAJOPMODPLPI4QMNLCLAA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.353362981 (389h38m43.836820185s) [noise=true , address=185.167.204.206:4044 , piece_id=QB46W43MJBO2JI2HMWH3RIFC7YKAPTI7HJUM46YF5CTZYJC2T4NQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.354258977 (389h38m43.837716169s) [noise=true , address=185.167.204.205:4019 , piece_id=5AC555XIK5W5UPETOAQEXYBE2YWT3WTCN762EQSR2IYSU54PXDCQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.365410692 (389h38m43.848867902s) [noise=true , address=185.167.204.206:4045 , piece_id=ZZKJVEV43JFLX7L6XRKIOC2JRRC7P4B4XBZXGWLL4SSJW4YFLCZQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.368609194 (389h38m43.852066395s) [noise=true , address=89.149.201.147:4005 , piece_id=65O63ETGC35NG5C3OWPZGQKTYRWJQ7O73P26ZC3ZFKYECETCQ3YA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.371782410 (389h38m43.855239598s) [noise=true , address=185.167.204.205:4032 , piece_id=J5NVOENBLOIS5JTQ62TI62IUR4Y2P52IYLLZDK2OGGMA4UHNHRDQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.373618919 (389h38m43.857076116s) [noise=true , address=185.167.204.205:4010 , piece_id=J7G2ETYDMHDBVZKESZ22AUBEZIC52FGZU7CROLSWTOGF2SGBWQRQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.379704851 (389h38m43.863162051s) [noise=true , address=185.167.204.206:4037 , piece_id=QSQ4ULBDAHRSXUJAWD4PTNVRYXO4F2MY626IKB2FDHKPCCGULXWA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.394563019 (389h38m43.878020215s) [noise=true , address=89.149.201.147:4009 , piece_id=ET7TF7LSLL6NAIYI7DTWXOCWSL2OSQ46DAFGQPM4D7MZC4KL3MTA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.397572775 (389h38m43.881029977s) [noise=true , address=185.167.204.205:4009 , piece_id=L6JZ2XWT2MLGXCWIA55LFTZ2WE5TGBCFTJBWRTVPUTHCEYCOX24A, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.526690895 (389h38m44.010148083s) [noise=true , address=81.171.14.70:4033 , piece_id=AXEWZ3F53TIT6N7P25DHNBCDLHTYGWGLZQRICQP52GA4VTMX6IOA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.807303763 (389h38m44.290760961s) [noise=true , address=89.149.201.147:4010 , piece_id=RG7F5CWJFUAASYE7QS6Y4PGPRT45SADGUOW3W7M5I5E5G7L34FDA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.811867760 (389h38m44.295324957s) [noise=true , address=89.149.201.147:4015 , piece_id=PV7RO2L2JTENDLOVF5NGMANGIVJKKHSAJBW5N4YX5VZMXNGD5XWA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.817491869 (389h38m44.300949066s) [noise=true , address=89.149.201.147:4023 , piece_id=G4IHTFGTLH2FJCMEXJKGAUD2ZD3NSNTBK3ZWPYIKTFRVV46XQTNQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.826882849 (389h38m44.310340046s) [noise=true , address=81.171.14.70:4030 , piece_id=26XAQQBYQGZ2UXPROFLMX5ATVFOUMZ5LEXC4FZ2MGREVROH57PUA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.829999912 (389h38m44.313457109s) [noise=true , address=89.149.201.147:4002 , piece_id=J6DZWEXEPL53SI65XV7S32QNFGPHIORND42UPPQEBEFGDY3QCHJQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.833608524 (389h38m44.31706572s) [noise=true , address=89.149.201.147:4004 , piece_id=METSYHH76QGQFFGUDDPIYTEGFZX5DRBY2LB3FTULP5PQ26ZJA7AQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.833631335 (389h38m44.317088532s) [seg_pos={0 0}, upload=405070]: Satisfied cohort requirements after 26 pieces
Jan 31 21:33:50.833925231 (389h38m44.317382437s) [noise=true , address=89.149.201.147:4013 , piece_id=YA2Q5VWLCWM3HOSVFKB3VJL2JDSX6PUQMNYKOM6RQHJUXQV743JQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ZKP9g2cY7B3b8eyfHqoChUG2nYtz3nbFRGA6szgtACzT4M6J): protocol: expected piece hash; context canceled; EOF
Jan 31 21:33:50.833933563 (389h38m44.31739076s) [noise=true , address=89.149.201.147:4013 , piece_id=YA2Q5VWLCWM3HOSVFKB3VJL2JDSX6PUQMNYKOM6RQHJUXQV743JQ, seg_pos={0 0}, upload=405070]: Long tail cancellation due to optimal threshold
Jan 31 21:33:50.834025355 (389h38m44.317482562s) [noise=true , address=81.171.14.70:4039 , piece_id=TKSY6DCTWWY7UEFMTIWIP6GRG236CLB6CM34JGIGN3FN56KR3TQA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC): protocol: expected piece hash; context canceled; EOF
Jan 31 21:33:50.834046459 (389h38m44.317503672s) [noise=true , address=81.171.14.70:4039 , piece_id=TKSY6DCTWWY7UEFMTIWIP6GRG236CLB6CM34JGIGN3FN56KR3TQA, seg_pos={0 0}, upload=405070]: Long tail cancellation due to optimal threshold
Jan 31 21:33:50.834078259 (389h38m44.317535447s) [seg_pos={0 0}, upload=405070]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Jan 31 21:33:50.834187246 (389h38m44.31764446s) [upload=405070]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jan 31 21:33:50.834196158 (389h38m44.317653366s) [upload=405070]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jan 31 21:33:51.191301790 (389h38m44.67475899s) [upload=405070]: Done uploading segments...
Jan 31 21:33:51.191411925 (389h38m44.674869122s) [upload=405070]: Upload finished. err: metaclient: manager closed: closed: read tcp 159.223.143.39:47078->35.207.121.91:7777: read: connection reset by peer
Duration: 2.533127977s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............] duration:
2.174575576s trace:
https://jaeger.dev.storj.io/trace/43abdc33eb6dc242
FAIL (2026-01-31 21:41:58 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: manager closed: closed: read tcp 159.223.143.39:47078->35.207.121.91:7777: read: connection reset by peer
======LOGS======
.149.201.147:4005 , piece_id=65O63ETGC35NG5C3OWPZGQKTYRWJQ7O73P26ZC3ZFKYECETCQ3YA, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.683232901 (389h38m43.166690107s) [noise=true , address=185.167.204.206:4072 , piece_id=4VVQE6RWCILAA4JTRKSFXKN677CXXE2JFVL3NYHC7K6R54237PXQ, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.683709179 (389h38m43.167166383s) [noise=true , address=89.149.201.147:4004 , piece_id=METSYHH76QGQFFGUDDPIYTEGFZX5DRBY2LB3FTULP5PQ26ZJA7AQ, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.683724493 (389h38m43.167181697s) [noise=true , address=89.149.201.147:4010 , piece_id=RG7F5CWJFUAASYE7QS6Y4PGPRT45SADGUOW3W7M5I5E5G7L34FDA, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.684173473 (389h38m43.16763068s) [noise=true , address=185.167.204.206:4037 , piece_id=QSQ4ULBDAHRSXUJAWD4PTNVRYXO4F2MY626IKB2FDHKPCCGULXWA, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.684694161 (389h38m43.168151369s) [noise=true , address=185.167.204.205:4017 , piece_id=3LM3MACH5OSCSALKAUIOVRSBCX5HHX2VG7RTQZ6BJ4ALHRQOL7KQ, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.686216206 (389h38m43.169673413s) [noise=true , address=81.171.14.70:4034 , piece_id=QQWI3YYGGFGITTEPDDHEYGDLAGPOMHQ5NYG2MLWP6VC5ZJI77BUQ, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.688413710 (389h38m43.171870916s) [noise=true , address=89.149.201.147:4009 , piece_id=ET7TF7LSLL6NAIYI7DTWXOCWSL2OSQ46DAFGQPM4D7MZC4KL3MTA, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.690659778 (389h38m43.174116984s) [noise=true , address=185.167.204.205:4009 , piece_id=L6JZ2XWT2MLGXCWIA55LFTZ2WE5TGBCFTJBWRTVPUTHCEYCOX24A, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.692330711 (389h38m43.175787902s) [noise=true , address=89.149.201.147:4019 , piece_id=SE62AS4I6LN4AA7CSBUK7FWZJTDHUIN56FNTASDN3Y64PVNFL5VA, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:50.141023513 (389h38m43.624480718s) [upload=405070]: Next returned nil segment
Jan 31 21:33:50.141047430 (389h38m43.624504638s) [upload=405070]: Waiting for error group managing segments...
Jan 31 21:33:50.324609708 (389h38m43.808066905s) [noise=true , address=185.167.204.205:4017 , piece_id=3LM3MACH5OSCSALKAUIOVRSBCX5HHX2VG7RTQZ6BJ4ALHRQOL7KQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.325388947 (389h38m43.808846162s) [noise=true , address=81.171.14.70:4034 , piece_id=QQWI3YYGGFGITTEPDDHEYGDLAGPOMHQ5NYG2MLWP6VC5ZJI77BUQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.341012709 (389h38m43.824469907s) [noise=true , address=185.167.204.205:4023 , piece_id=WHKCMG2FOY5XSLLHJN5EWKFKSOYXFS3OBYFO4IGN26Y4DYE3H42Q, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.344607851 (389h38m43.828065048s) [noise=true , address=89.149.201.147:4019 , piece_id=SE62AS4I6LN4AA7CSBUK7FWZJTDHUIN56FNTASDN3Y64PVNFL5VA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.345220277 (389h38m43.828677475s) [noise=true , address=185.167.204.206:4061 , piece_id=OIFH6PI53RPJJ44BDBRYI4GYQYLLHLV4633VJSDZB4U2LRUMEATA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.345872439 (389h38m43.829329637s) [noise=true , address=89.149.201.147:4014 , piece_id=FDEIWM2YPK2NHAOHY5NE74Q6XB7YJLGTINJ63KADKRLYPBY2CYYQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.347218937 (389h38m43.830676136s) [noise=true , address=185.167.204.205:4027 , piece_id=JMAGQ7KEDUDN7MBXQEGQGVB5NNWUO5HU7BZATHBORQ7EJN46UF6A, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.352289470 (389h38m43.835746665s) [noise=true , address=185.167.204.206:4066 , piece_id=HCYOMLOPP6PLWUD3H5VQ3PGUJUIG7RQAS2QKEPUOCRGWPCYSVUJQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.352722623 (389h38m43.836179824s) [noise=true , address=185.167.204.206:4072 , piece_id=4VVQE6RWCILAA4JTRKSFXKN677CXXE2JFVL3NYHC7K6R54237PXQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.353039283 (389h38m43.83649647s) [noise=true , address=185.167.204.206:4070 , piece_id=4OFRPTUQ34OGUGGBT6A4J64BWZP4PFKKAJOPMODPLPI4QMNLCLAA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.353362981 (389h38m43.836820185s) [noise=true , address=185.167.204.206:4044 , piece_id=QB46W43MJBO2JI2HMWH3RIFC7YKAPTI7HJUM46YF5CTZYJC2T4NQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.354258977 (389h38m43.837716169s) [noise=true , address=185.167.204.205:4019 , piece_id=5AC555XIK5W5UPETOAQEXYBE2YWT3WTCN762EQSR2IYSU54PXDCQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.365410692 (389h38m43.848867902s) [noise=true , address=185.167.204.206:4045 , piece_id=ZZKJVEV43JFLX7L6XRKIOC2JRRC7P4B4XBZXGWLL4SSJW4YFLCZQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.368609194 (389h38m43.852066395s) [noise=true , address=89.149.201.147:4005 , piece_id=65O63ETGC35NG5C3OWPZGQKTYRWJQ7O73P26ZC3ZFKYECETCQ3YA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.371782410 (389h38m43.855239598s) [noise=true , address=185.167.204.205:4032 , piece_id=J5NVOENBLOIS5JTQ62TI62IUR4Y2P52IYLLZDK2OGGMA4UHNHRDQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.373618919 (389h38m43.857076116s) [noise=true , address=185.167.204.205:4010 , piece_id=J7G2ETYDMHDBVZKESZ22AUBEZIC52FGZU7CROLSWTOGF2SGBWQRQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.379704851 (389h38m43.863162051s) [noise=true , address=185.167.204.206:4037 , piece_id=QSQ4ULBDAHRSXUJAWD4PTNVRYXO4F2MY626IKB2FDHKPCCGULXWA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.394563019 (389h38m43.878020215s) [noise=true , address=89.149.201.147:4009 , piece_id=ET7TF7LSLL6NAIYI7DTWXOCWSL2OSQ46DAFGQPM4D7MZC4KL3MTA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.397572775 (389h38m43.881029977s) [noise=true , address=185.167.204.205:4009 , piece_id=L6JZ2XWT2MLGXCWIA55LFTZ2WE5TGBCFTJBWRTVPUTHCEYCOX24A, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.526690895 (389h38m44.010148083s) [noise=true , address=81.171.14.70:4033 , piece_id=AXEWZ3F53TIT6N7P25DHNBCDLHTYGWGLZQRICQP52GA4VTMX6IOA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.807303763 (389h38m44.290760961s) [noise=true , address=89.149.201.147:4010 , piece_id=RG7F5CWJFUAASYE7QS6Y4PGPRT45SADGUOW3W7M5I5E5G7L34FDA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.811867760 (389h38m44.295324957s) [noise=true , address=89.149.201.147:4015 , piece_id=PV7RO2L2JTENDLOVF5NGMANGIVJKKHSAJBW5N4YX5VZMXNGD5XWA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.817491869 (389h38m44.300949066s) [noise=true , address=89.149.201.147:4023 , piece_id=G4IHTFGTLH2FJCMEXJKGAUD2ZD3NSNTBK3ZWPYIKTFRVV46XQTNQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.826882849 (389h38m44.310340046s) [noise=true , address=81.171.14.70:4030 , piece_id=26XAQQBYQGZ2UXPROFLMX5ATVFOUMZ5LEXC4FZ2MGREVROH57PUA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.829999912 (389h38m44.313457109s) [noise=true , address=89.149.201.147:4002 , piece_id=J6DZWEXEPL53SI65XV7S32QNFGPHIORND42UPPQEBEFGDY3QCHJQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.833608524 (389h38m44.31706572s) [noise=true , address=89.149.201.147:4004 , piece_id=METSYHH76QGQFFGUDDPIYTEGFZX5DRBY2LB3FTULP5PQ26ZJA7AQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.833631335 (389h38m44.317088532s) [seg_pos={0 0}, upload=405070]: Satisfied cohort requirements after 26 pieces
Jan 31 21:33:50.833925231 (389h38m44.317382437s) [noise=true , address=89.149.201.147:4013 , piece_id=YA2Q5VWLCWM3HOSVFKB3VJL2JDSX6PUQMNYKOM6RQHJUXQV743JQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ZKP9g2cY7B3b8eyfHqoChUG2nYtz3nbFRGA6szgtACzT4M6J): protocol: expected piece hash; context canceled; EOF
Jan 31 21:33:50.833933563 (389h38m44.31739076s) [noise=true , address=89.149.201.147:4013 , piece_id=YA2Q5VWLCWM3HOSVFKB3VJL2JDSX6PUQMNYKOM6RQHJUXQV743JQ, seg_pos={0 0}, upload=405070]: Long tail cancellation due to optimal threshold
Jan 31 21:33:50.834025355 (389h38m44.317482562s) [noise=true , address=81.171.14.70:4039 , piece_id=TKSY6DCTWWY7UEFMTIWIP6GRG236CLB6CM34JGIGN3FN56KR3TQA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC): protocol: expected piece hash; context canceled; EOF
Jan 31 21:33:50.834046459 (389h38m44.317503672s) [noise=true , address=81.171.14.70:4039 , piece_id=TKSY6DCTWWY7UEFMTIWIP6GRG236CLB6CM34JGIGN3FN56KR3TQA, seg_pos={0 0}, upload=405070]: Long tail cancellation due to optimal threshold
Jan 31 21:33:50.834078259 (389h38m44.317535447s) [seg_pos={0 0}, upload=405070]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Jan 31 21:33:50.834187246 (389h38m44.31764446s) [upload=405070]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jan 31 21:33:50.834196158 (389h38m44.317653366s) [upload=405070]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jan 31 21:33:51.191301790 (389h38m44.67475899s) [upload=405070]: Done uploading segments...
Jan 31 21:33:51.191411925 (389h38m44.674869122s) [upload=405070]: Upload finished. err: metaclient: manager closed: closed: read tcp 159.223.143.39:47078->35.207.121.91:7777: read: connection reset by peer
Duration: 2.174600328s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........] duration:
2.557829126s trace:
https://jaeger.dev.storj.io/trace/43abdc33eb6dc242
FAIL (2026-01-31 21:41:32 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: manager closed: closed: read tcp 159.223.143.39:47078->35.207.121.91:7777: read: connection reset by peer
======LOGS======
.149.201.147:4005 , piece_id=65O63ETGC35NG5C3OWPZGQKTYRWJQ7O73P26ZC3ZFKYECETCQ3YA, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.683232901 (389h38m43.166690107s) [noise=true , address=185.167.204.206:4072 , piece_id=4VVQE6RWCILAA4JTRKSFXKN677CXXE2JFVL3NYHC7K6R54237PXQ, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.683709179 (389h38m43.167166383s) [noise=true , address=89.149.201.147:4004 , piece_id=METSYHH76QGQFFGUDDPIYTEGFZX5DRBY2LB3FTULP5PQ26ZJA7AQ, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.683724493 (389h38m43.167181697s) [noise=true , address=89.149.201.147:4010 , piece_id=RG7F5CWJFUAASYE7QS6Y4PGPRT45SADGUOW3W7M5I5E5G7L34FDA, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.684173473 (389h38m43.16763068s) [noise=true , address=185.167.204.206:4037 , piece_id=QSQ4ULBDAHRSXUJAWD4PTNVRYXO4F2MY626IKB2FDHKPCCGULXWA, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.684694161 (389h38m43.168151369s) [noise=true , address=185.167.204.205:4017 , piece_id=3LM3MACH5OSCSALKAUIOVRSBCX5HHX2VG7RTQZ6BJ4ALHRQOL7KQ, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.686216206 (389h38m43.169673413s) [noise=true , address=81.171.14.70:4034 , piece_id=QQWI3YYGGFGITTEPDDHEYGDLAGPOMHQ5NYG2MLWP6VC5ZJI77BUQ, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.688413710 (389h38m43.171870916s) [noise=true , address=89.149.201.147:4009 , piece_id=ET7TF7LSLL6NAIYI7DTWXOCWSL2OSQ46DAFGQPM4D7MZC4KL3MTA, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.690659778 (389h38m43.174116984s) [noise=true , address=185.167.204.205:4009 , piece_id=L6JZ2XWT2MLGXCWIA55LFTZ2WE5TGBCFTJBWRTVPUTHCEYCOX24A, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.692330711 (389h38m43.175787902s) [noise=true , address=89.149.201.147:4019 , piece_id=SE62AS4I6LN4AA7CSBUK7FWZJTDHUIN56FNTASDN3Y64PVNFL5VA, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:50.141023513 (389h38m43.624480718s) [upload=405070]: Next returned nil segment
Jan 31 21:33:50.141047430 (389h38m43.624504638s) [upload=405070]: Waiting for error group managing segments...
Jan 31 21:33:50.324609708 (389h38m43.808066905s) [noise=true , address=185.167.204.205:4017 , piece_id=3LM3MACH5OSCSALKAUIOVRSBCX5HHX2VG7RTQZ6BJ4ALHRQOL7KQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.325388947 (389h38m43.808846162s) [noise=true , address=81.171.14.70:4034 , piece_id=QQWI3YYGGFGITTEPDDHEYGDLAGPOMHQ5NYG2MLWP6VC5ZJI77BUQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.341012709 (389h38m43.824469907s) [noise=true , address=185.167.204.205:4023 , piece_id=WHKCMG2FOY5XSLLHJN5EWKFKSOYXFS3OBYFO4IGN26Y4DYE3H42Q, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.344607851 (389h38m43.828065048s) [noise=true , address=89.149.201.147:4019 , piece_id=SE62AS4I6LN4AA7CSBUK7FWZJTDHUIN56FNTASDN3Y64PVNFL5VA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.345220277 (389h38m43.828677475s) [noise=true , address=185.167.204.206:4061 , piece_id=OIFH6PI53RPJJ44BDBRYI4GYQYLLHLV4633VJSDZB4U2LRUMEATA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.345872439 (389h38m43.829329637s) [noise=true , address=89.149.201.147:4014 , piece_id=FDEIWM2YPK2NHAOHY5NE74Q6XB7YJLGTINJ63KADKRLYPBY2CYYQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.347218937 (389h38m43.830676136s) [noise=true , address=185.167.204.205:4027 , piece_id=JMAGQ7KEDUDN7MBXQEGQGVB5NNWUO5HU7BZATHBORQ7EJN46UF6A, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.352289470 (389h38m43.835746665s) [noise=true , address=185.167.204.206:4066 , piece_id=HCYOMLOPP6PLWUD3H5VQ3PGUJUIG7RQAS2QKEPUOCRGWPCYSVUJQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.352722623 (389h38m43.836179824s) [noise=true , address=185.167.204.206:4072 , piece_id=4VVQE6RWCILAA4JTRKSFXKN677CXXE2JFVL3NYHC7K6R54237PXQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.353039283 (389h38m43.83649647s) [noise=true , address=185.167.204.206:4070 , piece_id=4OFRPTUQ34OGUGGBT6A4J64BWZP4PFKKAJOPMODPLPI4QMNLCLAA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.353362981 (389h38m43.836820185s) [noise=true , address=185.167.204.206:4044 , piece_id=QB46W43MJBO2JI2HMWH3RIFC7YKAPTI7HJUM46YF5CTZYJC2T4NQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.354258977 (389h38m43.837716169s) [noise=true , address=185.167.204.205:4019 , piece_id=5AC555XIK5W5UPETOAQEXYBE2YWT3WTCN762EQSR2IYSU54PXDCQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.365410692 (389h38m43.848867902s) [noise=true , address=185.167.204.206:4045 , piece_id=ZZKJVEV43JFLX7L6XRKIOC2JRRC7P4B4XBZXGWLL4SSJW4YFLCZQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.368609194 (389h38m43.852066395s) [noise=true , address=89.149.201.147:4005 , piece_id=65O63ETGC35NG5C3OWPZGQKTYRWJQ7O73P26ZC3ZFKYECETCQ3YA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.371782410 (389h38m43.855239598s) [noise=true , address=185.167.204.205:4032 , piece_id=J5NVOENBLOIS5JTQ62TI62IUR4Y2P52IYLLZDK2OGGMA4UHNHRDQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.373618919 (389h38m43.857076116s) [noise=true , address=185.167.204.205:4010 , piece_id=J7G2ETYDMHDBVZKESZ22AUBEZIC52FGZU7CROLSWTOGF2SGBWQRQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.379704851 (389h38m43.863162051s) [noise=true , address=185.167.204.206:4037 , piece_id=QSQ4ULBDAHRSXUJAWD4PTNVRYXO4F2MY626IKB2FDHKPCCGULXWA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.394563019 (389h38m43.878020215s) [noise=true , address=89.149.201.147:4009 , piece_id=ET7TF7LSLL6NAIYI7DTWXOCWSL2OSQ46DAFGQPM4D7MZC4KL3MTA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.397572775 (389h38m43.881029977s) [noise=true , address=185.167.204.205:4009 , piece_id=L6JZ2XWT2MLGXCWIA55LFTZ2WE5TGBCFTJBWRTVPUTHCEYCOX24A, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.526690895 (389h38m44.010148083s) [noise=true , address=81.171.14.70:4033 , piece_id=AXEWZ3F53TIT6N7P25DHNBCDLHTYGWGLZQRICQP52GA4VTMX6IOA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.807303763 (389h38m44.290760961s) [noise=true , address=89.149.201.147:4010 , piece_id=RG7F5CWJFUAASYE7QS6Y4PGPRT45SADGUOW3W7M5I5E5G7L34FDA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.811867760 (389h38m44.295324957s) [noise=true , address=89.149.201.147:4015 , piece_id=PV7RO2L2JTENDLOVF5NGMANGIVJKKHSAJBW5N4YX5VZMXNGD5XWA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.817491869 (389h38m44.300949066s) [noise=true , address=89.149.201.147:4023 , piece_id=G4IHTFGTLH2FJCMEXJKGAUD2ZD3NSNTBK3ZWPYIKTFRVV46XQTNQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.826882849 (389h38m44.310340046s) [noise=true , address=81.171.14.70:4030 , piece_id=26XAQQBYQGZ2UXPROFLMX5ATVFOUMZ5LEXC4FZ2MGREVROH57PUA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.829999912 (389h38m44.313457109s) [noise=true , address=89.149.201.147:4002 , piece_id=J6DZWEXEPL53SI65XV7S32QNFGPHIORND42UPPQEBEFGDY3QCHJQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.833608524 (389h38m44.31706572s) [noise=true , address=89.149.201.147:4004 , piece_id=METSYHH76QGQFFGUDDPIYTEGFZX5DRBY2LB3FTULP5PQ26ZJA7AQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.833631335 (389h38m44.317088532s) [seg_pos={0 0}, upload=405070]: Satisfied cohort requirements after 26 pieces
Jan 31 21:33:50.833925231 (389h38m44.317382437s) [noise=true , address=89.149.201.147:4013 , piece_id=YA2Q5VWLCWM3HOSVFKB3VJL2JDSX6PUQMNYKOM6RQHJUXQV743JQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ZKP9g2cY7B3b8eyfHqoChUG2nYtz3nbFRGA6szgtACzT4M6J): protocol: expected piece hash; context canceled; EOF
Jan 31 21:33:50.833933563 (389h38m44.31739076s) [noise=true , address=89.149.201.147:4013 , piece_id=YA2Q5VWLCWM3HOSVFKB3VJL2JDSX6PUQMNYKOM6RQHJUXQV743JQ, seg_pos={0 0}, upload=405070]: Long tail cancellation due to optimal threshold
Jan 31 21:33:50.834025355 (389h38m44.317482562s) [noise=true , address=81.171.14.70:4039 , piece_id=TKSY6DCTWWY7UEFMTIWIP6GRG236CLB6CM34JGIGN3FN56KR3TQA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC): protocol: expected piece hash; context canceled; EOF
Jan 31 21:33:50.834046459 (389h38m44.317503672s) [noise=true , address=81.171.14.70:4039 , piece_id=TKSY6DCTWWY7UEFMTIWIP6GRG236CLB6CM34JGIGN3FN56KR3TQA, seg_pos={0 0}, upload=405070]: Long tail cancellation due to optimal threshold
Jan 31 21:33:50.834078259 (389h38m44.317535447s) [seg_pos={0 0}, upload=405070]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Jan 31 21:33:50.834187246 (389h38m44.31764446s) [upload=405070]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jan 31 21:33:50.834196158 (389h38m44.317653366s) [upload=405070]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jan 31 21:33:51.191301790 (389h38m44.67475899s) [upload=405070]: Done uploading segments...
Jan 31 21:33:51.191411925 (389h38m44.674869122s) [upload=405070]: Upload finished. err: metaclient: manager closed: closed: read tcp 159.223.143.39:47078->35.207.121.91:7777: read: connection reset by peer
Duration: 2.557856745s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........] duration:
2.625444025s trace:
https://jaeger.dev.storj.io/trace/43abdc33eb6dc242
FAIL (2026-01-31 21:40:53 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: manager closed: closed: read tcp 159.223.143.39:47078->35.207.121.91:7777: read: connection reset by peer
======LOGS======
.149.201.147:4005 , piece_id=65O63ETGC35NG5C3OWPZGQKTYRWJQ7O73P26ZC3ZFKYECETCQ3YA, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.683232901 (389h38m43.166690107s) [noise=true , address=185.167.204.206:4072 , piece_id=4VVQE6RWCILAA4JTRKSFXKN677CXXE2JFVL3NYHC7K6R54237PXQ, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.683709179 (389h38m43.167166383s) [noise=true , address=89.149.201.147:4004 , piece_id=METSYHH76QGQFFGUDDPIYTEGFZX5DRBY2LB3FTULP5PQ26ZJA7AQ, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.683724493 (389h38m43.167181697s) [noise=true , address=89.149.201.147:4010 , piece_id=RG7F5CWJFUAASYE7QS6Y4PGPRT45SADGUOW3W7M5I5E5G7L34FDA, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.684173473 (389h38m43.16763068s) [noise=true , address=185.167.204.206:4037 , piece_id=QSQ4ULBDAHRSXUJAWD4PTNVRYXO4F2MY626IKB2FDHKPCCGULXWA, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.684694161 (389h38m43.168151369s) [noise=true , address=185.167.204.205:4017 , piece_id=3LM3MACH5OSCSALKAUIOVRSBCX5HHX2VG7RTQZ6BJ4ALHRQOL7KQ, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.686216206 (389h38m43.169673413s) [noise=true , address=81.171.14.70:4034 , piece_id=QQWI3YYGGFGITTEPDDHEYGDLAGPOMHQ5NYG2MLWP6VC5ZJI77BUQ, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.688413710 (389h38m43.171870916s) [noise=true , address=89.149.201.147:4009 , piece_id=ET7TF7LSLL6NAIYI7DTWXOCWSL2OSQ46DAFGQPM4D7MZC4KL3MTA, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.690659778 (389h38m43.174116984s) [noise=true , address=185.167.204.205:4009 , piece_id=L6JZ2XWT2MLGXCWIA55LFTZ2WE5TGBCFTJBWRTVPUTHCEYCOX24A, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.692330711 (389h38m43.175787902s) [noise=true , address=89.149.201.147:4019 , piece_id=SE62AS4I6LN4AA7CSBUK7FWZJTDHUIN56FNTASDN3Y64PVNFL5VA, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:50.141023513 (389h38m43.624480718s) [upload=405070]: Next returned nil segment
Jan 31 21:33:50.141047430 (389h38m43.624504638s) [upload=405070]: Waiting for error group managing segments...
Jan 31 21:33:50.324609708 (389h38m43.808066905s) [noise=true , address=185.167.204.205:4017 , piece_id=3LM3MACH5OSCSALKAUIOVRSBCX5HHX2VG7RTQZ6BJ4ALHRQOL7KQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.325388947 (389h38m43.808846162s) [noise=true , address=81.171.14.70:4034 , piece_id=QQWI3YYGGFGITTEPDDHEYGDLAGPOMHQ5NYG2MLWP6VC5ZJI77BUQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.341012709 (389h38m43.824469907s) [noise=true , address=185.167.204.205:4023 , piece_id=WHKCMG2FOY5XSLLHJN5EWKFKSOYXFS3OBYFO4IGN26Y4DYE3H42Q, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.344607851 (389h38m43.828065048s) [noise=true , address=89.149.201.147:4019 , piece_id=SE62AS4I6LN4AA7CSBUK7FWZJTDHUIN56FNTASDN3Y64PVNFL5VA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.345220277 (389h38m43.828677475s) [noise=true , address=185.167.204.206:4061 , piece_id=OIFH6PI53RPJJ44BDBRYI4GYQYLLHLV4633VJSDZB4U2LRUMEATA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.345872439 (389h38m43.829329637s) [noise=true , address=89.149.201.147:4014 , piece_id=FDEIWM2YPK2NHAOHY5NE74Q6XB7YJLGTINJ63KADKRLYPBY2CYYQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.347218937 (389h38m43.830676136s) [noise=true , address=185.167.204.205:4027 , piece_id=JMAGQ7KEDUDN7MBXQEGQGVB5NNWUO5HU7BZATHBORQ7EJN46UF6A, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.352289470 (389h38m43.835746665s) [noise=true , address=185.167.204.206:4066 , piece_id=HCYOMLOPP6PLWUD3H5VQ3PGUJUIG7RQAS2QKEPUOCRGWPCYSVUJQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.352722623 (389h38m43.836179824s) [noise=true , address=185.167.204.206:4072 , piece_id=4VVQE6RWCILAA4JTRKSFXKN677CXXE2JFVL3NYHC7K6R54237PXQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.353039283 (389h38m43.83649647s) [noise=true , address=185.167.204.206:4070 , piece_id=4OFRPTUQ34OGUGGBT6A4J64BWZP4PFKKAJOPMODPLPI4QMNLCLAA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.353362981 (389h38m43.836820185s) [noise=true , address=185.167.204.206:4044 , piece_id=QB46W43MJBO2JI2HMWH3RIFC7YKAPTI7HJUM46YF5CTZYJC2T4NQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.354258977 (389h38m43.837716169s) [noise=true , address=185.167.204.205:4019 , piece_id=5AC555XIK5W5UPETOAQEXYBE2YWT3WTCN762EQSR2IYSU54PXDCQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.365410692 (389h38m43.848867902s) [noise=true , address=185.167.204.206:4045 , piece_id=ZZKJVEV43JFLX7L6XRKIOC2JRRC7P4B4XBZXGWLL4SSJW4YFLCZQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.368609194 (389h38m43.852066395s) [noise=true , address=89.149.201.147:4005 , piece_id=65O63ETGC35NG5C3OWPZGQKTYRWJQ7O73P26ZC3ZFKYECETCQ3YA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.371782410 (389h38m43.855239598s) [noise=true , address=185.167.204.205:4032 , piece_id=J5NVOENBLOIS5JTQ62TI62IUR4Y2P52IYLLZDK2OGGMA4UHNHRDQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.373618919 (389h38m43.857076116s) [noise=true , address=185.167.204.205:4010 , piece_id=J7G2ETYDMHDBVZKESZ22AUBEZIC52FGZU7CROLSWTOGF2SGBWQRQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.379704851 (389h38m43.863162051s) [noise=true , address=185.167.204.206:4037 , piece_id=QSQ4ULBDAHRSXUJAWD4PTNVRYXO4F2MY626IKB2FDHKPCCGULXWA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.394563019 (389h38m43.878020215s) [noise=true , address=89.149.201.147:4009 , piece_id=ET7TF7LSLL6NAIYI7DTWXOCWSL2OSQ46DAFGQPM4D7MZC4KL3MTA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.397572775 (389h38m43.881029977s) [noise=true , address=185.167.204.205:4009 , piece_id=L6JZ2XWT2MLGXCWIA55LFTZ2WE5TGBCFTJBWRTVPUTHCEYCOX24A, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.526690895 (389h38m44.010148083s) [noise=true , address=81.171.14.70:4033 , piece_id=AXEWZ3F53TIT6N7P25DHNBCDLHTYGWGLZQRICQP52GA4VTMX6IOA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.807303763 (389h38m44.290760961s) [noise=true , address=89.149.201.147:4010 , piece_id=RG7F5CWJFUAASYE7QS6Y4PGPRT45SADGUOW3W7M5I5E5G7L34FDA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.811867760 (389h38m44.295324957s) [noise=true , address=89.149.201.147:4015 , piece_id=PV7RO2L2JTENDLOVF5NGMANGIVJKKHSAJBW5N4YX5VZMXNGD5XWA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.817491869 (389h38m44.300949066s) [noise=true , address=89.149.201.147:4023 , piece_id=G4IHTFGTLH2FJCMEXJKGAUD2ZD3NSNTBK3ZWPYIKTFRVV46XQTNQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.826882849 (389h38m44.310340046s) [noise=true , address=81.171.14.70:4030 , piece_id=26XAQQBYQGZ2UXPROFLMX5ATVFOUMZ5LEXC4FZ2MGREVROH57PUA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.829999912 (389h38m44.313457109s) [noise=true , address=89.149.201.147:4002 , piece_id=J6DZWEXEPL53SI65XV7S32QNFGPHIORND42UPPQEBEFGDY3QCHJQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.833608524 (389h38m44.31706572s) [noise=true , address=89.149.201.147:4004 , piece_id=METSYHH76QGQFFGUDDPIYTEGFZX5DRBY2LB3FTULP5PQ26ZJA7AQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.833631335 (389h38m44.317088532s) [seg_pos={0 0}, upload=405070]: Satisfied cohort requirements after 26 pieces
Jan 31 21:33:50.833925231 (389h38m44.317382437s) [noise=true , address=89.149.201.147:4013 , piece_id=YA2Q5VWLCWM3HOSVFKB3VJL2JDSX6PUQMNYKOM6RQHJUXQV743JQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ZKP9g2cY7B3b8eyfHqoChUG2nYtz3nbFRGA6szgtACzT4M6J): protocol: expected piece hash; context canceled; EOF
Jan 31 21:33:50.833933563 (389h38m44.31739076s) [noise=true , address=89.149.201.147:4013 , piece_id=YA2Q5VWLCWM3HOSVFKB3VJL2JDSX6PUQMNYKOM6RQHJUXQV743JQ, seg_pos={0 0}, upload=405070]: Long tail cancellation due to optimal threshold
Jan 31 21:33:50.834025355 (389h38m44.317482562s) [noise=true , address=81.171.14.70:4039 , piece_id=TKSY6DCTWWY7UEFMTIWIP6GRG236CLB6CM34JGIGN3FN56KR3TQA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC): protocol: expected piece hash; context canceled; EOF
Jan 31 21:33:50.834046459 (389h38m44.317503672s) [noise=true , address=81.171.14.70:4039 , piece_id=TKSY6DCTWWY7UEFMTIWIP6GRG236CLB6CM34JGIGN3FN56KR3TQA, seg_pos={0 0}, upload=405070]: Long tail cancellation due to optimal threshold
Jan 31 21:33:50.834078259 (389h38m44.317535447s) [seg_pos={0 0}, upload=405070]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Jan 31 21:33:50.834187246 (389h38m44.31764446s) [upload=405070]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jan 31 21:33:50.834196158 (389h38m44.317653366s) [upload=405070]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jan 31 21:33:51.191301790 (389h38m44.67475899s) [upload=405070]: Done uploading segments...
Jan 31 21:33:51.191411925 (389h38m44.674869122s) [upload=405070]: Upload finished. err: metaclient: manager closed: closed: read tcp 159.223.143.39:47078->35.207.121.91:7777: read: connection reset by peer
Duration: 2.625475774s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
2.591147828s trace:
https://jaeger.dev.storj.io/trace/43abdc33eb6dc242
FAIL (2026-01-31 21:39:56 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: manager closed: closed: read tcp 159.223.143.39:47078->35.207.121.91:7777: read: connection reset by peer
======LOGS======
.149.201.147:4005 , piece_id=65O63ETGC35NG5C3OWPZGQKTYRWJQ7O73P26ZC3ZFKYECETCQ3YA, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.683232901 (389h38m43.166690107s) [noise=true , address=185.167.204.206:4072 , piece_id=4VVQE6RWCILAA4JTRKSFXKN677CXXE2JFVL3NYHC7K6R54237PXQ, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.683709179 (389h38m43.167166383s) [noise=true , address=89.149.201.147:4004 , piece_id=METSYHH76QGQFFGUDDPIYTEGFZX5DRBY2LB3FTULP5PQ26ZJA7AQ, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.683724493 (389h38m43.167181697s) [noise=true , address=89.149.201.147:4010 , piece_id=RG7F5CWJFUAASYE7QS6Y4PGPRT45SADGUOW3W7M5I5E5G7L34FDA, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.684173473 (389h38m43.16763068s) [noise=true , address=185.167.204.206:4037 , piece_id=QSQ4ULBDAHRSXUJAWD4PTNVRYXO4F2MY626IKB2FDHKPCCGULXWA, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.684694161 (389h38m43.168151369s) [noise=true , address=185.167.204.205:4017 , piece_id=3LM3MACH5OSCSALKAUIOVRSBCX5HHX2VG7RTQZ6BJ4ALHRQOL7KQ, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.686216206 (389h38m43.169673413s) [noise=true , address=81.171.14.70:4034 , piece_id=QQWI3YYGGFGITTEPDDHEYGDLAGPOMHQ5NYG2MLWP6VC5ZJI77BUQ, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.688413710 (389h38m43.171870916s) [noise=true , address=89.149.201.147:4009 , piece_id=ET7TF7LSLL6NAIYI7DTWXOCWSL2OSQ46DAFGQPM4D7MZC4KL3MTA, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.690659778 (389h38m43.174116984s) [noise=true , address=185.167.204.205:4009 , piece_id=L6JZ2XWT2MLGXCWIA55LFTZ2WE5TGBCFTJBWRTVPUTHCEYCOX24A, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.692330711 (389h38m43.175787902s) [noise=true , address=89.149.201.147:4019 , piece_id=SE62AS4I6LN4AA7CSBUK7FWZJTDHUIN56FNTASDN3Y64PVNFL5VA, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:50.141023513 (389h38m43.624480718s) [upload=405070]: Next returned nil segment
Jan 31 21:33:50.141047430 (389h38m43.624504638s) [upload=405070]: Waiting for error group managing segments...
Jan 31 21:33:50.324609708 (389h38m43.808066905s) [noise=true , address=185.167.204.205:4017 , piece_id=3LM3MACH5OSCSALKAUIOVRSBCX5HHX2VG7RTQZ6BJ4ALHRQOL7KQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.325388947 (389h38m43.808846162s) [noise=true , address=81.171.14.70:4034 , piece_id=QQWI3YYGGFGITTEPDDHEYGDLAGPOMHQ5NYG2MLWP6VC5ZJI77BUQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.341012709 (389h38m43.824469907s) [noise=true , address=185.167.204.205:4023 , piece_id=WHKCMG2FOY5XSLLHJN5EWKFKSOYXFS3OBYFO4IGN26Y4DYE3H42Q, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.344607851 (389h38m43.828065048s) [noise=true , address=89.149.201.147:4019 , piece_id=SE62AS4I6LN4AA7CSBUK7FWZJTDHUIN56FNTASDN3Y64PVNFL5VA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.345220277 (389h38m43.828677475s) [noise=true , address=185.167.204.206:4061 , piece_id=OIFH6PI53RPJJ44BDBRYI4GYQYLLHLV4633VJSDZB4U2LRUMEATA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.345872439 (389h38m43.829329637s) [noise=true , address=89.149.201.147:4014 , piece_id=FDEIWM2YPK2NHAOHY5NE74Q6XB7YJLGTINJ63KADKRLYPBY2CYYQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.347218937 (389h38m43.830676136s) [noise=true , address=185.167.204.205:4027 , piece_id=JMAGQ7KEDUDN7MBXQEGQGVB5NNWUO5HU7BZATHBORQ7EJN46UF6A, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.352289470 (389h38m43.835746665s) [noise=true , address=185.167.204.206:4066 , piece_id=HCYOMLOPP6PLWUD3H5VQ3PGUJUIG7RQAS2QKEPUOCRGWPCYSVUJQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.352722623 (389h38m43.836179824s) [noise=true , address=185.167.204.206:4072 , piece_id=4VVQE6RWCILAA4JTRKSFXKN677CXXE2JFVL3NYHC7K6R54237PXQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.353039283 (389h38m43.83649647s) [noise=true , address=185.167.204.206:4070 , piece_id=4OFRPTUQ34OGUGGBT6A4J64BWZP4PFKKAJOPMODPLPI4QMNLCLAA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.353362981 (389h38m43.836820185s) [noise=true , address=185.167.204.206:4044 , piece_id=QB46W43MJBO2JI2HMWH3RIFC7YKAPTI7HJUM46YF5CTZYJC2T4NQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.354258977 (389h38m43.837716169s) [noise=true , address=185.167.204.205:4019 , piece_id=5AC555XIK5W5UPETOAQEXYBE2YWT3WTCN762EQSR2IYSU54PXDCQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.365410692 (389h38m43.848867902s) [noise=true , address=185.167.204.206:4045 , piece_id=ZZKJVEV43JFLX7L6XRKIOC2JRRC7P4B4XBZXGWLL4SSJW4YFLCZQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.368609194 (389h38m43.852066395s) [noise=true , address=89.149.201.147:4005 , piece_id=65O63ETGC35NG5C3OWPZGQKTYRWJQ7O73P26ZC3ZFKYECETCQ3YA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.371782410 (389h38m43.855239598s) [noise=true , address=185.167.204.205:4032 , piece_id=J5NVOENBLOIS5JTQ62TI62IUR4Y2P52IYLLZDK2OGGMA4UHNHRDQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.373618919 (389h38m43.857076116s) [noise=true , address=185.167.204.205:4010 , piece_id=J7G2ETYDMHDBVZKESZ22AUBEZIC52FGZU7CROLSWTOGF2SGBWQRQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.379704851 (389h38m43.863162051s) [noise=true , address=185.167.204.206:4037 , piece_id=QSQ4ULBDAHRSXUJAWD4PTNVRYXO4F2MY626IKB2FDHKPCCGULXWA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.394563019 (389h38m43.878020215s) [noise=true , address=89.149.201.147:4009 , piece_id=ET7TF7LSLL6NAIYI7DTWXOCWSL2OSQ46DAFGQPM4D7MZC4KL3MTA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.397572775 (389h38m43.881029977s) [noise=true , address=185.167.204.205:4009 , piece_id=L6JZ2XWT2MLGXCWIA55LFTZ2WE5TGBCFTJBWRTVPUTHCEYCOX24A, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.526690895 (389h38m44.010148083s) [noise=true , address=81.171.14.70:4033 , piece_id=AXEWZ3F53TIT6N7P25DHNBCDLHTYGWGLZQRICQP52GA4VTMX6IOA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.807303763 (389h38m44.290760961s) [noise=true , address=89.149.201.147:4010 , piece_id=RG7F5CWJFUAASYE7QS6Y4PGPRT45SADGUOW3W7M5I5E5G7L34FDA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.811867760 (389h38m44.295324957s) [noise=true , address=89.149.201.147:4015 , piece_id=PV7RO2L2JTENDLOVF5NGMANGIVJKKHSAJBW5N4YX5VZMXNGD5XWA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.817491869 (389h38m44.300949066s) [noise=true , address=89.149.201.147:4023 , piece_id=G4IHTFGTLH2FJCMEXJKGAUD2ZD3NSNTBK3ZWPYIKTFRVV46XQTNQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.826882849 (389h38m44.310340046s) [noise=true , address=81.171.14.70:4030 , piece_id=26XAQQBYQGZ2UXPROFLMX5ATVFOUMZ5LEXC4FZ2MGREVROH57PUA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.829999912 (389h38m44.313457109s) [noise=true , address=89.149.201.147:4002 , piece_id=J6DZWEXEPL53SI65XV7S32QNFGPHIORND42UPPQEBEFGDY3QCHJQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.833608524 (389h38m44.31706572s) [noise=true , address=89.149.201.147:4004 , piece_id=METSYHH76QGQFFGUDDPIYTEGFZX5DRBY2LB3FTULP5PQ26ZJA7AQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.833631335 (389h38m44.317088532s) [seg_pos={0 0}, upload=405070]: Satisfied cohort requirements after 26 pieces
Jan 31 21:33:50.833925231 (389h38m44.317382437s) [noise=true , address=89.149.201.147:4013 , piece_id=YA2Q5VWLCWM3HOSVFKB3VJL2JDSX6PUQMNYKOM6RQHJUXQV743JQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ZKP9g2cY7B3b8eyfHqoChUG2nYtz3nbFRGA6szgtACzT4M6J): protocol: expected piece hash; context canceled; EOF
Jan 31 21:33:50.833933563 (389h38m44.31739076s) [noise=true , address=89.149.201.147:4013 , piece_id=YA2Q5VWLCWM3HOSVFKB3VJL2JDSX6PUQMNYKOM6RQHJUXQV743JQ, seg_pos={0 0}, upload=405070]: Long tail cancellation due to optimal threshold
Jan 31 21:33:50.834025355 (389h38m44.317482562s) [noise=true , address=81.171.14.70:4039 , piece_id=TKSY6DCTWWY7UEFMTIWIP6GRG236CLB6CM34JGIGN3FN56KR3TQA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC): protocol: expected piece hash; context canceled; EOF
Jan 31 21:33:50.834046459 (389h38m44.317503672s) [noise=true , address=81.171.14.70:4039 , piece_id=TKSY6DCTWWY7UEFMTIWIP6GRG236CLB6CM34JGIGN3FN56KR3TQA, seg_pos={0 0}, upload=405070]: Long tail cancellation due to optimal threshold
Jan 31 21:33:50.834078259 (389h38m44.317535447s) [seg_pos={0 0}, upload=405070]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Jan 31 21:33:50.834187246 (389h38m44.31764446s) [upload=405070]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jan 31 21:33:50.834196158 (389h38m44.317653366s) [upload=405070]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jan 31 21:33:51.191301790 (389h38m44.67475899s) [upload=405070]: Done uploading segments...
Jan 31 21:33:51.191411925 (389h38m44.674869122s) [upload=405070]: Upload finished. err: metaclient: manager closed: closed: read tcp 159.223.143.39:47078->35.207.121.91:7777: read: connection reset by peer
Duration: 2.591180211s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........] duration:
2.132624794s trace:
https://jaeger.dev.storj.io/trace/43abdc33eb6dc242
FAIL (2026-01-31 21:39:30 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: manager closed: closed: read tcp 159.223.143.39:47078->35.207.121.91:7777: read: connection reset by peer
======LOGS======
.149.201.147:4005 , piece_id=65O63ETGC35NG5C3OWPZGQKTYRWJQ7O73P26ZC3ZFKYECETCQ3YA, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.683232901 (389h38m43.166690107s) [noise=true , address=185.167.204.206:4072 , piece_id=4VVQE6RWCILAA4JTRKSFXKN677CXXE2JFVL3NYHC7K6R54237PXQ, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.683709179 (389h38m43.167166383s) [noise=true , address=89.149.201.147:4004 , piece_id=METSYHH76QGQFFGUDDPIYTEGFZX5DRBY2LB3FTULP5PQ26ZJA7AQ, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.683724493 (389h38m43.167181697s) [noise=true , address=89.149.201.147:4010 , piece_id=RG7F5CWJFUAASYE7QS6Y4PGPRT45SADGUOW3W7M5I5E5G7L34FDA, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.684173473 (389h38m43.16763068s) [noise=true , address=185.167.204.206:4037 , piece_id=QSQ4ULBDAHRSXUJAWD4PTNVRYXO4F2MY626IKB2FDHKPCCGULXWA, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.684694161 (389h38m43.168151369s) [noise=true , address=185.167.204.205:4017 , piece_id=3LM3MACH5OSCSALKAUIOVRSBCX5HHX2VG7RTQZ6BJ4ALHRQOL7KQ, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.686216206 (389h38m43.169673413s) [noise=true , address=81.171.14.70:4034 , piece_id=QQWI3YYGGFGITTEPDDHEYGDLAGPOMHQ5NYG2MLWP6VC5ZJI77BUQ, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.688413710 (389h38m43.171870916s) [noise=true , address=89.149.201.147:4009 , piece_id=ET7TF7LSLL6NAIYI7DTWXOCWSL2OSQ46DAFGQPM4D7MZC4KL3MTA, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.690659778 (389h38m43.174116984s) [noise=true , address=185.167.204.205:4009 , piece_id=L6JZ2XWT2MLGXCWIA55LFTZ2WE5TGBCFTJBWRTVPUTHCEYCOX24A, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.692330711 (389h38m43.175787902s) [noise=true , address=89.149.201.147:4019 , piece_id=SE62AS4I6LN4AA7CSBUK7FWZJTDHUIN56FNTASDN3Y64PVNFL5VA, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:50.141023513 (389h38m43.624480718s) [upload=405070]: Next returned nil segment
Jan 31 21:33:50.141047430 (389h38m43.624504638s) [upload=405070]: Waiting for error group managing segments...
Jan 31 21:33:50.324609708 (389h38m43.808066905s) [noise=true , address=185.167.204.205:4017 , piece_id=3LM3MACH5OSCSALKAUIOVRSBCX5HHX2VG7RTQZ6BJ4ALHRQOL7KQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.325388947 (389h38m43.808846162s) [noise=true , address=81.171.14.70:4034 , piece_id=QQWI3YYGGFGITTEPDDHEYGDLAGPOMHQ5NYG2MLWP6VC5ZJI77BUQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.341012709 (389h38m43.824469907s) [noise=true , address=185.167.204.205:4023 , piece_id=WHKCMG2FOY5XSLLHJN5EWKFKSOYXFS3OBYFO4IGN26Y4DYE3H42Q, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.344607851 (389h38m43.828065048s) [noise=true , address=89.149.201.147:4019 , piece_id=SE62AS4I6LN4AA7CSBUK7FWZJTDHUIN56FNTASDN3Y64PVNFL5VA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.345220277 (389h38m43.828677475s) [noise=true , address=185.167.204.206:4061 , piece_id=OIFH6PI53RPJJ44BDBRYI4GYQYLLHLV4633VJSDZB4U2LRUMEATA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.345872439 (389h38m43.829329637s) [noise=true , address=89.149.201.147:4014 , piece_id=FDEIWM2YPK2NHAOHY5NE74Q6XB7YJLGTINJ63KADKRLYPBY2CYYQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.347218937 (389h38m43.830676136s) [noise=true , address=185.167.204.205:4027 , piece_id=JMAGQ7KEDUDN7MBXQEGQGVB5NNWUO5HU7BZATHBORQ7EJN46UF6A, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.352289470 (389h38m43.835746665s) [noise=true , address=185.167.204.206:4066 , piece_id=HCYOMLOPP6PLWUD3H5VQ3PGUJUIG7RQAS2QKEPUOCRGWPCYSVUJQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.352722623 (389h38m43.836179824s) [noise=true , address=185.167.204.206:4072 , piece_id=4VVQE6RWCILAA4JTRKSFXKN677CXXE2JFVL3NYHC7K6R54237PXQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.353039283 (389h38m43.83649647s) [noise=true , address=185.167.204.206:4070 , piece_id=4OFRPTUQ34OGUGGBT6A4J64BWZP4PFKKAJOPMODPLPI4QMNLCLAA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.353362981 (389h38m43.836820185s) [noise=true , address=185.167.204.206:4044 , piece_id=QB46W43MJBO2JI2HMWH3RIFC7YKAPTI7HJUM46YF5CTZYJC2T4NQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.354258977 (389h38m43.837716169s) [noise=true , address=185.167.204.205:4019 , piece_id=5AC555XIK5W5UPETOAQEXYBE2YWT3WTCN762EQSR2IYSU54PXDCQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.365410692 (389h38m43.848867902s) [noise=true , address=185.167.204.206:4045 , piece_id=ZZKJVEV43JFLX7L6XRKIOC2JRRC7P4B4XBZXGWLL4SSJW4YFLCZQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.368609194 (389h38m43.852066395s) [noise=true , address=89.149.201.147:4005 , piece_id=65O63ETGC35NG5C3OWPZGQKTYRWJQ7O73P26ZC3ZFKYECETCQ3YA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.371782410 (389h38m43.855239598s) [noise=true , address=185.167.204.205:4032 , piece_id=J5NVOENBLOIS5JTQ62TI62IUR4Y2P52IYLLZDK2OGGMA4UHNHRDQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.373618919 (389h38m43.857076116s) [noise=true , address=185.167.204.205:4010 , piece_id=J7G2ETYDMHDBVZKESZ22AUBEZIC52FGZU7CROLSWTOGF2SGBWQRQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.379704851 (389h38m43.863162051s) [noise=true , address=185.167.204.206:4037 , piece_id=QSQ4ULBDAHRSXUJAWD4PTNVRYXO4F2MY626IKB2FDHKPCCGULXWA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.394563019 (389h38m43.878020215s) [noise=true , address=89.149.201.147:4009 , piece_id=ET7TF7LSLL6NAIYI7DTWXOCWSL2OSQ46DAFGQPM4D7MZC4KL3MTA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.397572775 (389h38m43.881029977s) [noise=true , address=185.167.204.205:4009 , piece_id=L6JZ2XWT2MLGXCWIA55LFTZ2WE5TGBCFTJBWRTVPUTHCEYCOX24A, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.526690895 (389h38m44.010148083s) [noise=true , address=81.171.14.70:4033 , piece_id=AXEWZ3F53TIT6N7P25DHNBCDLHTYGWGLZQRICQP52GA4VTMX6IOA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.807303763 (389h38m44.290760961s) [noise=true , address=89.149.201.147:4010 , piece_id=RG7F5CWJFUAASYE7QS6Y4PGPRT45SADGUOW3W7M5I5E5G7L34FDA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.811867760 (389h38m44.295324957s) [noise=true , address=89.149.201.147:4015 , piece_id=PV7RO2L2JTENDLOVF5NGMANGIVJKKHSAJBW5N4YX5VZMXNGD5XWA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.817491869 (389h38m44.300949066s) [noise=true , address=89.149.201.147:4023 , piece_id=G4IHTFGTLH2FJCMEXJKGAUD2ZD3NSNTBK3ZWPYIKTFRVV46XQTNQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.826882849 (389h38m44.310340046s) [noise=true , address=81.171.14.70:4030 , piece_id=26XAQQBYQGZ2UXPROFLMX5ATVFOUMZ5LEXC4FZ2MGREVROH57PUA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.829999912 (389h38m44.313457109s) [noise=true , address=89.149.201.147:4002 , piece_id=J6DZWEXEPL53SI65XV7S32QNFGPHIORND42UPPQEBEFGDY3QCHJQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.833608524 (389h38m44.31706572s) [noise=true , address=89.149.201.147:4004 , piece_id=METSYHH76QGQFFGUDDPIYTEGFZX5DRBY2LB3FTULP5PQ26ZJA7AQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.833631335 (389h38m44.317088532s) [seg_pos={0 0}, upload=405070]: Satisfied cohort requirements after 26 pieces
Jan 31 21:33:50.833925231 (389h38m44.317382437s) [noise=true , address=89.149.201.147:4013 , piece_id=YA2Q5VWLCWM3HOSVFKB3VJL2JDSX6PUQMNYKOM6RQHJUXQV743JQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ZKP9g2cY7B3b8eyfHqoChUG2nYtz3nbFRGA6szgtACzT4M6J): protocol: expected piece hash; context canceled; EOF
Jan 31 21:33:50.833933563 (389h38m44.31739076s) [noise=true , address=89.149.201.147:4013 , piece_id=YA2Q5VWLCWM3HOSVFKB3VJL2JDSX6PUQMNYKOM6RQHJUXQV743JQ, seg_pos={0 0}, upload=405070]: Long tail cancellation due to optimal threshold
Jan 31 21:33:50.834025355 (389h38m44.317482562s) [noise=true , address=81.171.14.70:4039 , piece_id=TKSY6DCTWWY7UEFMTIWIP6GRG236CLB6CM34JGIGN3FN56KR3TQA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC): protocol: expected piece hash; context canceled; EOF
Jan 31 21:33:50.834046459 (389h38m44.317503672s) [noise=true , address=81.171.14.70:4039 , piece_id=TKSY6DCTWWY7UEFMTIWIP6GRG236CLB6CM34JGIGN3FN56KR3TQA, seg_pos={0 0}, upload=405070]: Long tail cancellation due to optimal threshold
Jan 31 21:33:50.834078259 (389h38m44.317535447s) [seg_pos={0 0}, upload=405070]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Jan 31 21:33:50.834187246 (389h38m44.31764446s) [upload=405070]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jan 31 21:33:50.834196158 (389h38m44.317653366s) [upload=405070]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jan 31 21:33:51.191301790 (389h38m44.67475899s) [upload=405070]: Done uploading segments...
Jan 31 21:33:51.191411925 (389h38m44.674869122s) [upload=405070]: Upload finished. err: metaclient: manager closed: closed: read tcp 159.223.143.39:47078->35.207.121.91:7777: read: connection reset by peer
Duration: 2.132657922s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......] duration:
2.116578044s trace:
https://jaeger.dev.storj.io/trace/43abdc33eb6dc242
FAIL (2026-01-31 21:38:51 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: manager closed: closed: read tcp 159.223.143.39:47078->35.207.121.91:7777: read: connection reset by peer
======LOGS======
.149.201.147:4005 , piece_id=65O63ETGC35NG5C3OWPZGQKTYRWJQ7O73P26ZC3ZFKYECETCQ3YA, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.683232901 (389h38m43.166690107s) [noise=true , address=185.167.204.206:4072 , piece_id=4VVQE6RWCILAA4JTRKSFXKN677CXXE2JFVL3NYHC7K6R54237PXQ, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.683709179 (389h38m43.167166383s) [noise=true , address=89.149.201.147:4004 , piece_id=METSYHH76QGQFFGUDDPIYTEGFZX5DRBY2LB3FTULP5PQ26ZJA7AQ, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.683724493 (389h38m43.167181697s) [noise=true , address=89.149.201.147:4010 , piece_id=RG7F5CWJFUAASYE7QS6Y4PGPRT45SADGUOW3W7M5I5E5G7L34FDA, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.684173473 (389h38m43.16763068s) [noise=true , address=185.167.204.206:4037 , piece_id=QSQ4ULBDAHRSXUJAWD4PTNVRYXO4F2MY626IKB2FDHKPCCGULXWA, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.684694161 (389h38m43.168151369s) [noise=true , address=185.167.204.205:4017 , piece_id=3LM3MACH5OSCSALKAUIOVRSBCX5HHX2VG7RTQZ6BJ4ALHRQOL7KQ, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.686216206 (389h38m43.169673413s) [noise=true , address=81.171.14.70:4034 , piece_id=QQWI3YYGGFGITTEPDDHEYGDLAGPOMHQ5NYG2MLWP6VC5ZJI77BUQ, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.688413710 (389h38m43.171870916s) [noise=true , address=89.149.201.147:4009 , piece_id=ET7TF7LSLL6NAIYI7DTWXOCWSL2OSQ46DAFGQPM4D7MZC4KL3MTA, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.690659778 (389h38m43.174116984s) [noise=true , address=185.167.204.205:4009 , piece_id=L6JZ2XWT2MLGXCWIA55LFTZ2WE5TGBCFTJBWRTVPUTHCEYCOX24A, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.692330711 (389h38m43.175787902s) [noise=true , address=89.149.201.147:4019 , piece_id=SE62AS4I6LN4AA7CSBUK7FWZJTDHUIN56FNTASDN3Y64PVNFL5VA, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:50.141023513 (389h38m43.624480718s) [upload=405070]: Next returned nil segment
Jan 31 21:33:50.141047430 (389h38m43.624504638s) [upload=405070]: Waiting for error group managing segments...
Jan 31 21:33:50.324609708 (389h38m43.808066905s) [noise=true , address=185.167.204.205:4017 , piece_id=3LM3MACH5OSCSALKAUIOVRSBCX5HHX2VG7RTQZ6BJ4ALHRQOL7KQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.325388947 (389h38m43.808846162s) [noise=true , address=81.171.14.70:4034 , piece_id=QQWI3YYGGFGITTEPDDHEYGDLAGPOMHQ5NYG2MLWP6VC5ZJI77BUQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.341012709 (389h38m43.824469907s) [noise=true , address=185.167.204.205:4023 , piece_id=WHKCMG2FOY5XSLLHJN5EWKFKSOYXFS3OBYFO4IGN26Y4DYE3H42Q, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.344607851 (389h38m43.828065048s) [noise=true , address=89.149.201.147:4019 , piece_id=SE62AS4I6LN4AA7CSBUK7FWZJTDHUIN56FNTASDN3Y64PVNFL5VA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.345220277 (389h38m43.828677475s) [noise=true , address=185.167.204.206:4061 , piece_id=OIFH6PI53RPJJ44BDBRYI4GYQYLLHLV4633VJSDZB4U2LRUMEATA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.345872439 (389h38m43.829329637s) [noise=true , address=89.149.201.147:4014 , piece_id=FDEIWM2YPK2NHAOHY5NE74Q6XB7YJLGTINJ63KADKRLYPBY2CYYQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.347218937 (389h38m43.830676136s) [noise=true , address=185.167.204.205:4027 , piece_id=JMAGQ7KEDUDN7MBXQEGQGVB5NNWUO5HU7BZATHBORQ7EJN46UF6A, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.352289470 (389h38m43.835746665s) [noise=true , address=185.167.204.206:4066 , piece_id=HCYOMLOPP6PLWUD3H5VQ3PGUJUIG7RQAS2QKEPUOCRGWPCYSVUJQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.352722623 (389h38m43.836179824s) [noise=true , address=185.167.204.206:4072 , piece_id=4VVQE6RWCILAA4JTRKSFXKN677CXXE2JFVL3NYHC7K6R54237PXQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.353039283 (389h38m43.83649647s) [noise=true , address=185.167.204.206:4070 , piece_id=4OFRPTUQ34OGUGGBT6A4J64BWZP4PFKKAJOPMODPLPI4QMNLCLAA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.353362981 (389h38m43.836820185s) [noise=true , address=185.167.204.206:4044 , piece_id=QB46W43MJBO2JI2HMWH3RIFC7YKAPTI7HJUM46YF5CTZYJC2T4NQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.354258977 (389h38m43.837716169s) [noise=true , address=185.167.204.205:4019 , piece_id=5AC555XIK5W5UPETOAQEXYBE2YWT3WTCN762EQSR2IYSU54PXDCQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.365410692 (389h38m43.848867902s) [noise=true , address=185.167.204.206:4045 , piece_id=ZZKJVEV43JFLX7L6XRKIOC2JRRC7P4B4XBZXGWLL4SSJW4YFLCZQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.368609194 (389h38m43.852066395s) [noise=true , address=89.149.201.147:4005 , piece_id=65O63ETGC35NG5C3OWPZGQKTYRWJQ7O73P26ZC3ZFKYECETCQ3YA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.371782410 (389h38m43.855239598s) [noise=true , address=185.167.204.205:4032 , piece_id=J5NVOENBLOIS5JTQ62TI62IUR4Y2P52IYLLZDK2OGGMA4UHNHRDQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.373618919 (389h38m43.857076116s) [noise=true , address=185.167.204.205:4010 , piece_id=J7G2ETYDMHDBVZKESZ22AUBEZIC52FGZU7CROLSWTOGF2SGBWQRQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.379704851 (389h38m43.863162051s) [noise=true , address=185.167.204.206:4037 , piece_id=QSQ4ULBDAHRSXUJAWD4PTNVRYXO4F2MY626IKB2FDHKPCCGULXWA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.394563019 (389h38m43.878020215s) [noise=true , address=89.149.201.147:4009 , piece_id=ET7TF7LSLL6NAIYI7DTWXOCWSL2OSQ46DAFGQPM4D7MZC4KL3MTA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.397572775 (389h38m43.881029977s) [noise=true , address=185.167.204.205:4009 , piece_id=L6JZ2XWT2MLGXCWIA55LFTZ2WE5TGBCFTJBWRTVPUTHCEYCOX24A, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.526690895 (389h38m44.010148083s) [noise=true , address=81.171.14.70:4033 , piece_id=AXEWZ3F53TIT6N7P25DHNBCDLHTYGWGLZQRICQP52GA4VTMX6IOA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.807303763 (389h38m44.290760961s) [noise=true , address=89.149.201.147:4010 , piece_id=RG7F5CWJFUAASYE7QS6Y4PGPRT45SADGUOW3W7M5I5E5G7L34FDA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.811867760 (389h38m44.295324957s) [noise=true , address=89.149.201.147:4015 , piece_id=PV7RO2L2JTENDLOVF5NGMANGIVJKKHSAJBW5N4YX5VZMXNGD5XWA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.817491869 (389h38m44.300949066s) [noise=true , address=89.149.201.147:4023 , piece_id=G4IHTFGTLH2FJCMEXJKGAUD2ZD3NSNTBK3ZWPYIKTFRVV46XQTNQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.826882849 (389h38m44.310340046s) [noise=true , address=81.171.14.70:4030 , piece_id=26XAQQBYQGZ2UXPROFLMX5ATVFOUMZ5LEXC4FZ2MGREVROH57PUA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.829999912 (389h38m44.313457109s) [noise=true , address=89.149.201.147:4002 , piece_id=J6DZWEXEPL53SI65XV7S32QNFGPHIORND42UPPQEBEFGDY3QCHJQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.833608524 (389h38m44.31706572s) [noise=true , address=89.149.201.147:4004 , piece_id=METSYHH76QGQFFGUDDPIYTEGFZX5DRBY2LB3FTULP5PQ26ZJA7AQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.833631335 (389h38m44.317088532s) [seg_pos={0 0}, upload=405070]: Satisfied cohort requirements after 26 pieces
Jan 31 21:33:50.833925231 (389h38m44.317382437s) [noise=true , address=89.149.201.147:4013 , piece_id=YA2Q5VWLCWM3HOSVFKB3VJL2JDSX6PUQMNYKOM6RQHJUXQV743JQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ZKP9g2cY7B3b8eyfHqoChUG2nYtz3nbFRGA6szgtACzT4M6J): protocol: expected piece hash; context canceled; EOF
Jan 31 21:33:50.833933563 (389h38m44.31739076s) [noise=true , address=89.149.201.147:4013 , piece_id=YA2Q5VWLCWM3HOSVFKB3VJL2JDSX6PUQMNYKOM6RQHJUXQV743JQ, seg_pos={0 0}, upload=405070]: Long tail cancellation due to optimal threshold
Jan 31 21:33:50.834025355 (389h38m44.317482562s) [noise=true , address=81.171.14.70:4039 , piece_id=TKSY6DCTWWY7UEFMTIWIP6GRG236CLB6CM34JGIGN3FN56KR3TQA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC): protocol: expected piece hash; context canceled; EOF
Jan 31 21:33:50.834046459 (389h38m44.317503672s) [noise=true , address=81.171.14.70:4039 , piece_id=TKSY6DCTWWY7UEFMTIWIP6GRG236CLB6CM34JGIGN3FN56KR3TQA, seg_pos={0 0}, upload=405070]: Long tail cancellation due to optimal threshold
Jan 31 21:33:50.834078259 (389h38m44.317535447s) [seg_pos={0 0}, upload=405070]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Jan 31 21:33:50.834187246 (389h38m44.31764446s) [upload=405070]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jan 31 21:33:50.834196158 (389h38m44.317653366s) [upload=405070]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jan 31 21:33:51.191301790 (389h38m44.67475899s) [upload=405070]: Done uploading segments...
Jan 31 21:33:51.191411925 (389h38m44.674869122s) [upload=405070]: Upload finished. err: metaclient: manager closed: closed: read tcp 159.223.143.39:47078->35.207.121.91:7777: read: connection reset by peer
Duration: 2.116606519s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......] duration:
2.139987421s trace:
https://jaeger.dev.storj.io/trace/43abdc33eb6dc242
FAIL (2026-01-31 21:38:18 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: manager closed: closed: read tcp 159.223.143.39:47078->35.207.121.91:7777: read: connection reset by peer
======LOGS======
.149.201.147:4005 , piece_id=65O63ETGC35NG5C3OWPZGQKTYRWJQ7O73P26ZC3ZFKYECETCQ3YA, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.683232901 (389h38m43.166690107s) [noise=true , address=185.167.204.206:4072 , piece_id=4VVQE6RWCILAA4JTRKSFXKN677CXXE2JFVL3NYHC7K6R54237PXQ, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.683709179 (389h38m43.167166383s) [noise=true , address=89.149.201.147:4004 , piece_id=METSYHH76QGQFFGUDDPIYTEGFZX5DRBY2LB3FTULP5PQ26ZJA7AQ, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.683724493 (389h38m43.167181697s) [noise=true , address=89.149.201.147:4010 , piece_id=RG7F5CWJFUAASYE7QS6Y4PGPRT45SADGUOW3W7M5I5E5G7L34FDA, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.684173473 (389h38m43.16763068s) [noise=true , address=185.167.204.206:4037 , piece_id=QSQ4ULBDAHRSXUJAWD4PTNVRYXO4F2MY626IKB2FDHKPCCGULXWA, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.684694161 (389h38m43.168151369s) [noise=true , address=185.167.204.205:4017 , piece_id=3LM3MACH5OSCSALKAUIOVRSBCX5HHX2VG7RTQZ6BJ4ALHRQOL7KQ, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.686216206 (389h38m43.169673413s) [noise=true , address=81.171.14.70:4034 , piece_id=QQWI3YYGGFGITTEPDDHEYGDLAGPOMHQ5NYG2MLWP6VC5ZJI77BUQ, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.688413710 (389h38m43.171870916s) [noise=true , address=89.149.201.147:4009 , piece_id=ET7TF7LSLL6NAIYI7DTWXOCWSL2OSQ46DAFGQPM4D7MZC4KL3MTA, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.690659778 (389h38m43.174116984s) [noise=true , address=185.167.204.205:4009 , piece_id=L6JZ2XWT2MLGXCWIA55LFTZ2WE5TGBCFTJBWRTVPUTHCEYCOX24A, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.692330711 (389h38m43.175787902s) [noise=true , address=89.149.201.147:4019 , piece_id=SE62AS4I6LN4AA7CSBUK7FWZJTDHUIN56FNTASDN3Y64PVNFL5VA, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:50.141023513 (389h38m43.624480718s) [upload=405070]: Next returned nil segment
Jan 31 21:33:50.141047430 (389h38m43.624504638s) [upload=405070]: Waiting for error group managing segments...
Jan 31 21:33:50.324609708 (389h38m43.808066905s) [noise=true , address=185.167.204.205:4017 , piece_id=3LM3MACH5OSCSALKAUIOVRSBCX5HHX2VG7RTQZ6BJ4ALHRQOL7KQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.325388947 (389h38m43.808846162s) [noise=true , address=81.171.14.70:4034 , piece_id=QQWI3YYGGFGITTEPDDHEYGDLAGPOMHQ5NYG2MLWP6VC5ZJI77BUQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.341012709 (389h38m43.824469907s) [noise=true , address=185.167.204.205:4023 , piece_id=WHKCMG2FOY5XSLLHJN5EWKFKSOYXFS3OBYFO4IGN26Y4DYE3H42Q, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.344607851 (389h38m43.828065048s) [noise=true , address=89.149.201.147:4019 , piece_id=SE62AS4I6LN4AA7CSBUK7FWZJTDHUIN56FNTASDN3Y64PVNFL5VA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.345220277 (389h38m43.828677475s) [noise=true , address=185.167.204.206:4061 , piece_id=OIFH6PI53RPJJ44BDBRYI4GYQYLLHLV4633VJSDZB4U2LRUMEATA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.345872439 (389h38m43.829329637s) [noise=true , address=89.149.201.147:4014 , piece_id=FDEIWM2YPK2NHAOHY5NE74Q6XB7YJLGTINJ63KADKRLYPBY2CYYQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.347218937 (389h38m43.830676136s) [noise=true , address=185.167.204.205:4027 , piece_id=JMAGQ7KEDUDN7MBXQEGQGVB5NNWUO5HU7BZATHBORQ7EJN46UF6A, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.352289470 (389h38m43.835746665s) [noise=true , address=185.167.204.206:4066 , piece_id=HCYOMLOPP6PLWUD3H5VQ3PGUJUIG7RQAS2QKEPUOCRGWPCYSVUJQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.352722623 (389h38m43.836179824s) [noise=true , address=185.167.204.206:4072 , piece_id=4VVQE6RWCILAA4JTRKSFXKN677CXXE2JFVL3NYHC7K6R54237PXQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.353039283 (389h38m43.83649647s) [noise=true , address=185.167.204.206:4070 , piece_id=4OFRPTUQ34OGUGGBT6A4J64BWZP4PFKKAJOPMODPLPI4QMNLCLAA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.353362981 (389h38m43.836820185s) [noise=true , address=185.167.204.206:4044 , piece_id=QB46W43MJBO2JI2HMWH3RIFC7YKAPTI7HJUM46YF5CTZYJC2T4NQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.354258977 (389h38m43.837716169s) [noise=true , address=185.167.204.205:4019 , piece_id=5AC555XIK5W5UPETOAQEXYBE2YWT3WTCN762EQSR2IYSU54PXDCQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.365410692 (389h38m43.848867902s) [noise=true , address=185.167.204.206:4045 , piece_id=ZZKJVEV43JFLX7L6XRKIOC2JRRC7P4B4XBZXGWLL4SSJW4YFLCZQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.368609194 (389h38m43.852066395s) [noise=true , address=89.149.201.147:4005 , piece_id=65O63ETGC35NG5C3OWPZGQKTYRWJQ7O73P26ZC3ZFKYECETCQ3YA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.371782410 (389h38m43.855239598s) [noise=true , address=185.167.204.205:4032 , piece_id=J5NVOENBLOIS5JTQ62TI62IUR4Y2P52IYLLZDK2OGGMA4UHNHRDQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.373618919 (389h38m43.857076116s) [noise=true , address=185.167.204.205:4010 , piece_id=J7G2ETYDMHDBVZKESZ22AUBEZIC52FGZU7CROLSWTOGF2SGBWQRQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.379704851 (389h38m43.863162051s) [noise=true , address=185.167.204.206:4037 , piece_id=QSQ4ULBDAHRSXUJAWD4PTNVRYXO4F2MY626IKB2FDHKPCCGULXWA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.394563019 (389h38m43.878020215s) [noise=true , address=89.149.201.147:4009 , piece_id=ET7TF7LSLL6NAIYI7DTWXOCWSL2OSQ46DAFGQPM4D7MZC4KL3MTA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.397572775 (389h38m43.881029977s) [noise=true , address=185.167.204.205:4009 , piece_id=L6JZ2XWT2MLGXCWIA55LFTZ2WE5TGBCFTJBWRTVPUTHCEYCOX24A, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.526690895 (389h38m44.010148083s) [noise=true , address=81.171.14.70:4033 , piece_id=AXEWZ3F53TIT6N7P25DHNBCDLHTYGWGLZQRICQP52GA4VTMX6IOA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.807303763 (389h38m44.290760961s) [noise=true , address=89.149.201.147:4010 , piece_id=RG7F5CWJFUAASYE7QS6Y4PGPRT45SADGUOW3W7M5I5E5G7L34FDA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.811867760 (389h38m44.295324957s) [noise=true , address=89.149.201.147:4015 , piece_id=PV7RO2L2JTENDLOVF5NGMANGIVJKKHSAJBW5N4YX5VZMXNGD5XWA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.817491869 (389h38m44.300949066s) [noise=true , address=89.149.201.147:4023 , piece_id=G4IHTFGTLH2FJCMEXJKGAUD2ZD3NSNTBK3ZWPYIKTFRVV46XQTNQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.826882849 (389h38m44.310340046s) [noise=true , address=81.171.14.70:4030 , piece_id=26XAQQBYQGZ2UXPROFLMX5ATVFOUMZ5LEXC4FZ2MGREVROH57PUA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.829999912 (389h38m44.313457109s) [noise=true , address=89.149.201.147:4002 , piece_id=J6DZWEXEPL53SI65XV7S32QNFGPHIORND42UPPQEBEFGDY3QCHJQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.833608524 (389h38m44.31706572s) [noise=true , address=89.149.201.147:4004 , piece_id=METSYHH76QGQFFGUDDPIYTEGFZX5DRBY2LB3FTULP5PQ26ZJA7AQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.833631335 (389h38m44.317088532s) [seg_pos={0 0}, upload=405070]: Satisfied cohort requirements after 26 pieces
Jan 31 21:33:50.833925231 (389h38m44.317382437s) [noise=true , address=89.149.201.147:4013 , piece_id=YA2Q5VWLCWM3HOSVFKB3VJL2JDSX6PUQMNYKOM6RQHJUXQV743JQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ZKP9g2cY7B3b8eyfHqoChUG2nYtz3nbFRGA6szgtACzT4M6J): protocol: expected piece hash; context canceled; EOF
Jan 31 21:33:50.833933563 (389h38m44.31739076s) [noise=true , address=89.149.201.147:4013 , piece_id=YA2Q5VWLCWM3HOSVFKB3VJL2JDSX6PUQMNYKOM6RQHJUXQV743JQ, seg_pos={0 0}, upload=405070]: Long tail cancellation due to optimal threshold
Jan 31 21:33:50.834025355 (389h38m44.317482562s) [noise=true , address=81.171.14.70:4039 , piece_id=TKSY6DCTWWY7UEFMTIWIP6GRG236CLB6CM34JGIGN3FN56KR3TQA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC): protocol: expected piece hash; context canceled; EOF
Jan 31 21:33:50.834046459 (389h38m44.317503672s) [noise=true , address=81.171.14.70:4039 , piece_id=TKSY6DCTWWY7UEFMTIWIP6GRG236CLB6CM34JGIGN3FN56KR3TQA, seg_pos={0 0}, upload=405070]: Long tail cancellation due to optimal threshold
Jan 31 21:33:50.834078259 (389h38m44.317535447s) [seg_pos={0 0}, upload=405070]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Jan 31 21:33:50.834187246 (389h38m44.31764446s) [upload=405070]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jan 31 21:33:50.834196158 (389h38m44.317653366s) [upload=405070]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jan 31 21:33:51.191301790 (389h38m44.67475899s) [upload=405070]: Done uploading segments...
Jan 31 21:33:51.191411925 (389h38m44.674869122s) [upload=405070]: Upload finished. err: metaclient: manager closed: closed: read tcp 159.223.143.39:47078->35.207.121.91:7777: read: connection reset by peer
Duration: 2.140018477s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....] duration:
2.615525099s trace:
https://jaeger.dev.storj.io/trace/43abdc33eb6dc242
FAIL (2026-01-31 21:37:07 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: manager closed: closed: read tcp 159.223.143.39:47078->35.207.121.91:7777: read: connection reset by peer
======LOGS======
.149.201.147:4005 , piece_id=65O63ETGC35NG5C3OWPZGQKTYRWJQ7O73P26ZC3ZFKYECETCQ3YA, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.683232901 (389h38m43.166690107s) [noise=true , address=185.167.204.206:4072 , piece_id=4VVQE6RWCILAA4JTRKSFXKN677CXXE2JFVL3NYHC7K6R54237PXQ, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.683709179 (389h38m43.167166383s) [noise=true , address=89.149.201.147:4004 , piece_id=METSYHH76QGQFFGUDDPIYTEGFZX5DRBY2LB3FTULP5PQ26ZJA7AQ, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.683724493 (389h38m43.167181697s) [noise=true , address=89.149.201.147:4010 , piece_id=RG7F5CWJFUAASYE7QS6Y4PGPRT45SADGUOW3W7M5I5E5G7L34FDA, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.684173473 (389h38m43.16763068s) [noise=true , address=185.167.204.206:4037 , piece_id=QSQ4ULBDAHRSXUJAWD4PTNVRYXO4F2MY626IKB2FDHKPCCGULXWA, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.684694161 (389h38m43.168151369s) [noise=true , address=185.167.204.205:4017 , piece_id=3LM3MACH5OSCSALKAUIOVRSBCX5HHX2VG7RTQZ6BJ4ALHRQOL7KQ, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.686216206 (389h38m43.169673413s) [noise=true , address=81.171.14.70:4034 , piece_id=QQWI3YYGGFGITTEPDDHEYGDLAGPOMHQ5NYG2MLWP6VC5ZJI77BUQ, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.688413710 (389h38m43.171870916s) [noise=true , address=89.149.201.147:4009 , piece_id=ET7TF7LSLL6NAIYI7DTWXOCWSL2OSQ46DAFGQPM4D7MZC4KL3MTA, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.690659778 (389h38m43.174116984s) [noise=true , address=185.167.204.205:4009 , piece_id=L6JZ2XWT2MLGXCWIA55LFTZ2WE5TGBCFTJBWRTVPUTHCEYCOX24A, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.692330711 (389h38m43.175787902s) [noise=true , address=89.149.201.147:4019 , piece_id=SE62AS4I6LN4AA7CSBUK7FWZJTDHUIN56FNTASDN3Y64PVNFL5VA, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:50.141023513 (389h38m43.624480718s) [upload=405070]: Next returned nil segment
Jan 31 21:33:50.141047430 (389h38m43.624504638s) [upload=405070]: Waiting for error group managing segments...
Jan 31 21:33:50.324609708 (389h38m43.808066905s) [noise=true , address=185.167.204.205:4017 , piece_id=3LM3MACH5OSCSALKAUIOVRSBCX5HHX2VG7RTQZ6BJ4ALHRQOL7KQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.325388947 (389h38m43.808846162s) [noise=true , address=81.171.14.70:4034 , piece_id=QQWI3YYGGFGITTEPDDHEYGDLAGPOMHQ5NYG2MLWP6VC5ZJI77BUQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.341012709 (389h38m43.824469907s) [noise=true , address=185.167.204.205:4023 , piece_id=WHKCMG2FOY5XSLLHJN5EWKFKSOYXFS3OBYFO4IGN26Y4DYE3H42Q, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.344607851 (389h38m43.828065048s) [noise=true , address=89.149.201.147:4019 , piece_id=SE62AS4I6LN4AA7CSBUK7FWZJTDHUIN56FNTASDN3Y64PVNFL5VA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.345220277 (389h38m43.828677475s) [noise=true , address=185.167.204.206:4061 , piece_id=OIFH6PI53RPJJ44BDBRYI4GYQYLLHLV4633VJSDZB4U2LRUMEATA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.345872439 (389h38m43.829329637s) [noise=true , address=89.149.201.147:4014 , piece_id=FDEIWM2YPK2NHAOHY5NE74Q6XB7YJLGTINJ63KADKRLYPBY2CYYQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.347218937 (389h38m43.830676136s) [noise=true , address=185.167.204.205:4027 , piece_id=JMAGQ7KEDUDN7MBXQEGQGVB5NNWUO5HU7BZATHBORQ7EJN46UF6A, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.352289470 (389h38m43.835746665s) [noise=true , address=185.167.204.206:4066 , piece_id=HCYOMLOPP6PLWUD3H5VQ3PGUJUIG7RQAS2QKEPUOCRGWPCYSVUJQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.352722623 (389h38m43.836179824s) [noise=true , address=185.167.204.206:4072 , piece_id=4VVQE6RWCILAA4JTRKSFXKN677CXXE2JFVL3NYHC7K6R54237PXQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.353039283 (389h38m43.83649647s) [noise=true , address=185.167.204.206:4070 , piece_id=4OFRPTUQ34OGUGGBT6A4J64BWZP4PFKKAJOPMODPLPI4QMNLCLAA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.353362981 (389h38m43.836820185s) [noise=true , address=185.167.204.206:4044 , piece_id=QB46W43MJBO2JI2HMWH3RIFC7YKAPTI7HJUM46YF5CTZYJC2T4NQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.354258977 (389h38m43.837716169s) [noise=true , address=185.167.204.205:4019 , piece_id=5AC555XIK5W5UPETOAQEXYBE2YWT3WTCN762EQSR2IYSU54PXDCQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.365410692 (389h38m43.848867902s) [noise=true , address=185.167.204.206:4045 , piece_id=ZZKJVEV43JFLX7L6XRKIOC2JRRC7P4B4XBZXGWLL4SSJW4YFLCZQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.368609194 (389h38m43.852066395s) [noise=true , address=89.149.201.147:4005 , piece_id=65O63ETGC35NG5C3OWPZGQKTYRWJQ7O73P26ZC3ZFKYECETCQ3YA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.371782410 (389h38m43.855239598s) [noise=true , address=185.167.204.205:4032 , piece_id=J5NVOENBLOIS5JTQ62TI62IUR4Y2P52IYLLZDK2OGGMA4UHNHRDQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.373618919 (389h38m43.857076116s) [noise=true , address=185.167.204.205:4010 , piece_id=J7G2ETYDMHDBVZKESZ22AUBEZIC52FGZU7CROLSWTOGF2SGBWQRQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.379704851 (389h38m43.863162051s) [noise=true , address=185.167.204.206:4037 , piece_id=QSQ4ULBDAHRSXUJAWD4PTNVRYXO4F2MY626IKB2FDHKPCCGULXWA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.394563019 (389h38m43.878020215s) [noise=true , address=89.149.201.147:4009 , piece_id=ET7TF7LSLL6NAIYI7DTWXOCWSL2OSQ46DAFGQPM4D7MZC4KL3MTA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.397572775 (389h38m43.881029977s) [noise=true , address=185.167.204.205:4009 , piece_id=L6JZ2XWT2MLGXCWIA55LFTZ2WE5TGBCFTJBWRTVPUTHCEYCOX24A, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.526690895 (389h38m44.010148083s) [noise=true , address=81.171.14.70:4033 , piece_id=AXEWZ3F53TIT6N7P25DHNBCDLHTYGWGLZQRICQP52GA4VTMX6IOA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.807303763 (389h38m44.290760961s) [noise=true , address=89.149.201.147:4010 , piece_id=RG7F5CWJFUAASYE7QS6Y4PGPRT45SADGUOW3W7M5I5E5G7L34FDA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.811867760 (389h38m44.295324957s) [noise=true , address=89.149.201.147:4015 , piece_id=PV7RO2L2JTENDLOVF5NGMANGIVJKKHSAJBW5N4YX5VZMXNGD5XWA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.817491869 (389h38m44.300949066s) [noise=true , address=89.149.201.147:4023 , piece_id=G4IHTFGTLH2FJCMEXJKGAUD2ZD3NSNTBK3ZWPYIKTFRVV46XQTNQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.826882849 (389h38m44.310340046s) [noise=true , address=81.171.14.70:4030 , piece_id=26XAQQBYQGZ2UXPROFLMX5ATVFOUMZ5LEXC4FZ2MGREVROH57PUA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.829999912 (389h38m44.313457109s) [noise=true , address=89.149.201.147:4002 , piece_id=J6DZWEXEPL53SI65XV7S32QNFGPHIORND42UPPQEBEFGDY3QCHJQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.833608524 (389h38m44.31706572s) [noise=true , address=89.149.201.147:4004 , piece_id=METSYHH76QGQFFGUDDPIYTEGFZX5DRBY2LB3FTULP5PQ26ZJA7AQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.833631335 (389h38m44.317088532s) [seg_pos={0 0}, upload=405070]: Satisfied cohort requirements after 26 pieces
Jan 31 21:33:50.833925231 (389h38m44.317382437s) [noise=true , address=89.149.201.147:4013 , piece_id=YA2Q5VWLCWM3HOSVFKB3VJL2JDSX6PUQMNYKOM6RQHJUXQV743JQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ZKP9g2cY7B3b8eyfHqoChUG2nYtz3nbFRGA6szgtACzT4M6J): protocol: expected piece hash; context canceled; EOF
Jan 31 21:33:50.833933563 (389h38m44.31739076s) [noise=true , address=89.149.201.147:4013 , piece_id=YA2Q5VWLCWM3HOSVFKB3VJL2JDSX6PUQMNYKOM6RQHJUXQV743JQ, seg_pos={0 0}, upload=405070]: Long tail cancellation due to optimal threshold
Jan 31 21:33:50.834025355 (389h38m44.317482562s) [noise=true , address=81.171.14.70:4039 , piece_id=TKSY6DCTWWY7UEFMTIWIP6GRG236CLB6CM34JGIGN3FN56KR3TQA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC): protocol: expected piece hash; context canceled; EOF
Jan 31 21:33:50.834046459 (389h38m44.317503672s) [noise=true , address=81.171.14.70:4039 , piece_id=TKSY6DCTWWY7UEFMTIWIP6GRG236CLB6CM34JGIGN3FN56KR3TQA, seg_pos={0 0}, upload=405070]: Long tail cancellation due to optimal threshold
Jan 31 21:33:50.834078259 (389h38m44.317535447s) [seg_pos={0 0}, upload=405070]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Jan 31 21:33:50.834187246 (389h38m44.31764446s) [upload=405070]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jan 31 21:33:50.834196158 (389h38m44.317653366s) [upload=405070]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jan 31 21:33:51.191301790 (389h38m44.67475899s) [upload=405070]: Done uploading segments...
Jan 31 21:33:51.191411925 (389h38m44.674869122s) [upload=405070]: Upload finished. err: metaclient: manager closed: closed: read tcp 159.223.143.39:47078->35.207.121.91:7777: read: connection reset by peer
Duration: 2.615554738s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....] duration:
2.557713213s trace:
https://jaeger.dev.storj.io/trace/43abdc33eb6dc242
FAIL (2026-01-31 21:36:31 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: manager closed: closed: read tcp 159.223.143.39:47078->35.207.121.91:7777: read: connection reset by peer
======LOGS======
.149.201.147:4005 , piece_id=65O63ETGC35NG5C3OWPZGQKTYRWJQ7O73P26ZC3ZFKYECETCQ3YA, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.683232901 (389h38m43.166690107s) [noise=true , address=185.167.204.206:4072 , piece_id=4VVQE6RWCILAA4JTRKSFXKN677CXXE2JFVL3NYHC7K6R54237PXQ, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.683709179 (389h38m43.167166383s) [noise=true , address=89.149.201.147:4004 , piece_id=METSYHH76QGQFFGUDDPIYTEGFZX5DRBY2LB3FTULP5PQ26ZJA7AQ, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.683724493 (389h38m43.167181697s) [noise=true , address=89.149.201.147:4010 , piece_id=RG7F5CWJFUAASYE7QS6Y4PGPRT45SADGUOW3W7M5I5E5G7L34FDA, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.684173473 (389h38m43.16763068s) [noise=true , address=185.167.204.206:4037 , piece_id=QSQ4ULBDAHRSXUJAWD4PTNVRYXO4F2MY626IKB2FDHKPCCGULXWA, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.684694161 (389h38m43.168151369s) [noise=true , address=185.167.204.205:4017 , piece_id=3LM3MACH5OSCSALKAUIOVRSBCX5HHX2VG7RTQZ6BJ4ALHRQOL7KQ, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.686216206 (389h38m43.169673413s) [noise=true , address=81.171.14.70:4034 , piece_id=QQWI3YYGGFGITTEPDDHEYGDLAGPOMHQ5NYG2MLWP6VC5ZJI77BUQ, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.688413710 (389h38m43.171870916s) [noise=true , address=89.149.201.147:4009 , piece_id=ET7TF7LSLL6NAIYI7DTWXOCWSL2OSQ46DAFGQPM4D7MZC4KL3MTA, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.690659778 (389h38m43.174116984s) [noise=true , address=185.167.204.205:4009 , piece_id=L6JZ2XWT2MLGXCWIA55LFTZ2WE5TGBCFTJBWRTVPUTHCEYCOX24A, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.692330711 (389h38m43.175787902s) [noise=true , address=89.149.201.147:4019 , piece_id=SE62AS4I6LN4AA7CSBUK7FWZJTDHUIN56FNTASDN3Y64PVNFL5VA, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:50.141023513 (389h38m43.624480718s) [upload=405070]: Next returned nil segment
Jan 31 21:33:50.141047430 (389h38m43.624504638s) [upload=405070]: Waiting for error group managing segments...
Jan 31 21:33:50.324609708 (389h38m43.808066905s) [noise=true , address=185.167.204.205:4017 , piece_id=3LM3MACH5OSCSALKAUIOVRSBCX5HHX2VG7RTQZ6BJ4ALHRQOL7KQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.325388947 (389h38m43.808846162s) [noise=true , address=81.171.14.70:4034 , piece_id=QQWI3YYGGFGITTEPDDHEYGDLAGPOMHQ5NYG2MLWP6VC5ZJI77BUQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.341012709 (389h38m43.824469907s) [noise=true , address=185.167.204.205:4023 , piece_id=WHKCMG2FOY5XSLLHJN5EWKFKSOYXFS3OBYFO4IGN26Y4DYE3H42Q, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.344607851 (389h38m43.828065048s) [noise=true , address=89.149.201.147:4019 , piece_id=SE62AS4I6LN4AA7CSBUK7FWZJTDHUIN56FNTASDN3Y64PVNFL5VA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.345220277 (389h38m43.828677475s) [noise=true , address=185.167.204.206:4061 , piece_id=OIFH6PI53RPJJ44BDBRYI4GYQYLLHLV4633VJSDZB4U2LRUMEATA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.345872439 (389h38m43.829329637s) [noise=true , address=89.149.201.147:4014 , piece_id=FDEIWM2YPK2NHAOHY5NE74Q6XB7YJLGTINJ63KADKRLYPBY2CYYQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.347218937 (389h38m43.830676136s) [noise=true , address=185.167.204.205:4027 , piece_id=JMAGQ7KEDUDN7MBXQEGQGVB5NNWUO5HU7BZATHBORQ7EJN46UF6A, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.352289470 (389h38m43.835746665s) [noise=true , address=185.167.204.206:4066 , piece_id=HCYOMLOPP6PLWUD3H5VQ3PGUJUIG7RQAS2QKEPUOCRGWPCYSVUJQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.352722623 (389h38m43.836179824s) [noise=true , address=185.167.204.206:4072 , piece_id=4VVQE6RWCILAA4JTRKSFXKN677CXXE2JFVL3NYHC7K6R54237PXQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.353039283 (389h38m43.83649647s) [noise=true , address=185.167.204.206:4070 , piece_id=4OFRPTUQ34OGUGGBT6A4J64BWZP4PFKKAJOPMODPLPI4QMNLCLAA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.353362981 (389h38m43.836820185s) [noise=true , address=185.167.204.206:4044 , piece_id=QB46W43MJBO2JI2HMWH3RIFC7YKAPTI7HJUM46YF5CTZYJC2T4NQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.354258977 (389h38m43.837716169s) [noise=true , address=185.167.204.205:4019 , piece_id=5AC555XIK5W5UPETOAQEXYBE2YWT3WTCN762EQSR2IYSU54PXDCQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.365410692 (389h38m43.848867902s) [noise=true , address=185.167.204.206:4045 , piece_id=ZZKJVEV43JFLX7L6XRKIOC2JRRC7P4B4XBZXGWLL4SSJW4YFLCZQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.368609194 (389h38m43.852066395s) [noise=true , address=89.149.201.147:4005 , piece_id=65O63ETGC35NG5C3OWPZGQKTYRWJQ7O73P26ZC3ZFKYECETCQ3YA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.371782410 (389h38m43.855239598s) [noise=true , address=185.167.204.205:4032 , piece_id=J5NVOENBLOIS5JTQ62TI62IUR4Y2P52IYLLZDK2OGGMA4UHNHRDQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.373618919 (389h38m43.857076116s) [noise=true , address=185.167.204.205:4010 , piece_id=J7G2ETYDMHDBVZKESZ22AUBEZIC52FGZU7CROLSWTOGF2SGBWQRQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.379704851 (389h38m43.863162051s) [noise=true , address=185.167.204.206:4037 , piece_id=QSQ4ULBDAHRSXUJAWD4PTNVRYXO4F2MY626IKB2FDHKPCCGULXWA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.394563019 (389h38m43.878020215s) [noise=true , address=89.149.201.147:4009 , piece_id=ET7TF7LSLL6NAIYI7DTWXOCWSL2OSQ46DAFGQPM4D7MZC4KL3MTA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.397572775 (389h38m43.881029977s) [noise=true , address=185.167.204.205:4009 , piece_id=L6JZ2XWT2MLGXCWIA55LFTZ2WE5TGBCFTJBWRTVPUTHCEYCOX24A, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.526690895 (389h38m44.010148083s) [noise=true , address=81.171.14.70:4033 , piece_id=AXEWZ3F53TIT6N7P25DHNBCDLHTYGWGLZQRICQP52GA4VTMX6IOA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.807303763 (389h38m44.290760961s) [noise=true , address=89.149.201.147:4010 , piece_id=RG7F5CWJFUAASYE7QS6Y4PGPRT45SADGUOW3W7M5I5E5G7L34FDA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.811867760 (389h38m44.295324957s) [noise=true , address=89.149.201.147:4015 , piece_id=PV7RO2L2JTENDLOVF5NGMANGIVJKKHSAJBW5N4YX5VZMXNGD5XWA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.817491869 (389h38m44.300949066s) [noise=true , address=89.149.201.147:4023 , piece_id=G4IHTFGTLH2FJCMEXJKGAUD2ZD3NSNTBK3ZWPYIKTFRVV46XQTNQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.826882849 (389h38m44.310340046s) [noise=true , address=81.171.14.70:4030 , piece_id=26XAQQBYQGZ2UXPROFLMX5ATVFOUMZ5LEXC4FZ2MGREVROH57PUA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.829999912 (389h38m44.313457109s) [noise=true , address=89.149.201.147:4002 , piece_id=J6DZWEXEPL53SI65XV7S32QNFGPHIORND42UPPQEBEFGDY3QCHJQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.833608524 (389h38m44.31706572s) [noise=true , address=89.149.201.147:4004 , piece_id=METSYHH76QGQFFGUDDPIYTEGFZX5DRBY2LB3FTULP5PQ26ZJA7AQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.833631335 (389h38m44.317088532s) [seg_pos={0 0}, upload=405070]: Satisfied cohort requirements after 26 pieces
Jan 31 21:33:50.833925231 (389h38m44.317382437s) [noise=true , address=89.149.201.147:4013 , piece_id=YA2Q5VWLCWM3HOSVFKB3VJL2JDSX6PUQMNYKOM6RQHJUXQV743JQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ZKP9g2cY7B3b8eyfHqoChUG2nYtz3nbFRGA6szgtACzT4M6J): protocol: expected piece hash; context canceled; EOF
Jan 31 21:33:50.833933563 (389h38m44.31739076s) [noise=true , address=89.149.201.147:4013 , piece_id=YA2Q5VWLCWM3HOSVFKB3VJL2JDSX6PUQMNYKOM6RQHJUXQV743JQ, seg_pos={0 0}, upload=405070]: Long tail cancellation due to optimal threshold
Jan 31 21:33:50.834025355 (389h38m44.317482562s) [noise=true , address=81.171.14.70:4039 , piece_id=TKSY6DCTWWY7UEFMTIWIP6GRG236CLB6CM34JGIGN3FN56KR3TQA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC): protocol: expected piece hash; context canceled; EOF
Jan 31 21:33:50.834046459 (389h38m44.317503672s) [noise=true , address=81.171.14.70:4039 , piece_id=TKSY6DCTWWY7UEFMTIWIP6GRG236CLB6CM34JGIGN3FN56KR3TQA, seg_pos={0 0}, upload=405070]: Long tail cancellation due to optimal threshold
Jan 31 21:33:50.834078259 (389h38m44.317535447s) [seg_pos={0 0}, upload=405070]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Jan 31 21:33:50.834187246 (389h38m44.31764446s) [upload=405070]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jan 31 21:33:50.834196158 (389h38m44.317653366s) [upload=405070]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jan 31 21:33:51.191301790 (389h38m44.67475899s) [upload=405070]: Done uploading segments...
Jan 31 21:33:51.191411925 (389h38m44.674869122s) [upload=405070]: Upload finished. err: metaclient: manager closed: closed: read tcp 159.223.143.39:47078->35.207.121.91:7777: read: connection reset by peer
Duration: 2.557739113s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...] duration:
2.457035005s trace:
https://jaeger.dev.storj.io/trace/43abdc33eb6dc242
FAIL (2026-01-31 21:35:58 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: manager closed: closed: read tcp 159.223.143.39:47078->35.207.121.91:7777: read: connection reset by peer
======LOGS======
.149.201.147:4005 , piece_id=65O63ETGC35NG5C3OWPZGQKTYRWJQ7O73P26ZC3ZFKYECETCQ3YA, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.683232901 (389h38m43.166690107s) [noise=true , address=185.167.204.206:4072 , piece_id=4VVQE6RWCILAA4JTRKSFXKN677CXXE2JFVL3NYHC7K6R54237PXQ, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.683709179 (389h38m43.167166383s) [noise=true , address=89.149.201.147:4004 , piece_id=METSYHH76QGQFFGUDDPIYTEGFZX5DRBY2LB3FTULP5PQ26ZJA7AQ, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.683724493 (389h38m43.167181697s) [noise=true , address=89.149.201.147:4010 , piece_id=RG7F5CWJFUAASYE7QS6Y4PGPRT45SADGUOW3W7M5I5E5G7L34FDA, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.684173473 (389h38m43.16763068s) [noise=true , address=185.167.204.206:4037 , piece_id=QSQ4ULBDAHRSXUJAWD4PTNVRYXO4F2MY626IKB2FDHKPCCGULXWA, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.684694161 (389h38m43.168151369s) [noise=true , address=185.167.204.205:4017 , piece_id=3LM3MACH5OSCSALKAUIOVRSBCX5HHX2VG7RTQZ6BJ4ALHRQOL7KQ, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.686216206 (389h38m43.169673413s) [noise=true , address=81.171.14.70:4034 , piece_id=QQWI3YYGGFGITTEPDDHEYGDLAGPOMHQ5NYG2MLWP6VC5ZJI77BUQ, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.688413710 (389h38m43.171870916s) [noise=true , address=89.149.201.147:4009 , piece_id=ET7TF7LSLL6NAIYI7DTWXOCWSL2OSQ46DAFGQPM4D7MZC4KL3MTA, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.690659778 (389h38m43.174116984s) [noise=true , address=185.167.204.205:4009 , piece_id=L6JZ2XWT2MLGXCWIA55LFTZ2WE5TGBCFTJBWRTVPUTHCEYCOX24A, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.692330711 (389h38m43.175787902s) [noise=true , address=89.149.201.147:4019 , piece_id=SE62AS4I6LN4AA7CSBUK7FWZJTDHUIN56FNTASDN3Y64PVNFL5VA, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:50.141023513 (389h38m43.624480718s) [upload=405070]: Next returned nil segment
Jan 31 21:33:50.141047430 (389h38m43.624504638s) [upload=405070]: Waiting for error group managing segments...
Jan 31 21:33:50.324609708 (389h38m43.808066905s) [noise=true , address=185.167.204.205:4017 , piece_id=3LM3MACH5OSCSALKAUIOVRSBCX5HHX2VG7RTQZ6BJ4ALHRQOL7KQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.325388947 (389h38m43.808846162s) [noise=true , address=81.171.14.70:4034 , piece_id=QQWI3YYGGFGITTEPDDHEYGDLAGPOMHQ5NYG2MLWP6VC5ZJI77BUQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.341012709 (389h38m43.824469907s) [noise=true , address=185.167.204.205:4023 , piece_id=WHKCMG2FOY5XSLLHJN5EWKFKSOYXFS3OBYFO4IGN26Y4DYE3H42Q, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.344607851 (389h38m43.828065048s) [noise=true , address=89.149.201.147:4019 , piece_id=SE62AS4I6LN4AA7CSBUK7FWZJTDHUIN56FNTASDN3Y64PVNFL5VA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.345220277 (389h38m43.828677475s) [noise=true , address=185.167.204.206:4061 , piece_id=OIFH6PI53RPJJ44BDBRYI4GYQYLLHLV4633VJSDZB4U2LRUMEATA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.345872439 (389h38m43.829329637s) [noise=true , address=89.149.201.147:4014 , piece_id=FDEIWM2YPK2NHAOHY5NE74Q6XB7YJLGTINJ63KADKRLYPBY2CYYQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.347218937 (389h38m43.830676136s) [noise=true , address=185.167.204.205:4027 , piece_id=JMAGQ7KEDUDN7MBXQEGQGVB5NNWUO5HU7BZATHBORQ7EJN46UF6A, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.352289470 (389h38m43.835746665s) [noise=true , address=185.167.204.206:4066 , piece_id=HCYOMLOPP6PLWUD3H5VQ3PGUJUIG7RQAS2QKEPUOCRGWPCYSVUJQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.352722623 (389h38m43.836179824s) [noise=true , address=185.167.204.206:4072 , piece_id=4VVQE6RWCILAA4JTRKSFXKN677CXXE2JFVL3NYHC7K6R54237PXQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.353039283 (389h38m43.83649647s) [noise=true , address=185.167.204.206:4070 , piece_id=4OFRPTUQ34OGUGGBT6A4J64BWZP4PFKKAJOPMODPLPI4QMNLCLAA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.353362981 (389h38m43.836820185s) [noise=true , address=185.167.204.206:4044 , piece_id=QB46W43MJBO2JI2HMWH3RIFC7YKAPTI7HJUM46YF5CTZYJC2T4NQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.354258977 (389h38m43.837716169s) [noise=true , address=185.167.204.205:4019 , piece_id=5AC555XIK5W5UPETOAQEXYBE2YWT3WTCN762EQSR2IYSU54PXDCQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.365410692 (389h38m43.848867902s) [noise=true , address=185.167.204.206:4045 , piece_id=ZZKJVEV43JFLX7L6XRKIOC2JRRC7P4B4XBZXGWLL4SSJW4YFLCZQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.368609194 (389h38m43.852066395s) [noise=true , address=89.149.201.147:4005 , piece_id=65O63ETGC35NG5C3OWPZGQKTYRWJQ7O73P26ZC3ZFKYECETCQ3YA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.371782410 (389h38m43.855239598s) [noise=true , address=185.167.204.205:4032 , piece_id=J5NVOENBLOIS5JTQ62TI62IUR4Y2P52IYLLZDK2OGGMA4UHNHRDQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.373618919 (389h38m43.857076116s) [noise=true , address=185.167.204.205:4010 , piece_id=J7G2ETYDMHDBVZKESZ22AUBEZIC52FGZU7CROLSWTOGF2SGBWQRQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.379704851 (389h38m43.863162051s) [noise=true , address=185.167.204.206:4037 , piece_id=QSQ4ULBDAHRSXUJAWD4PTNVRYXO4F2MY626IKB2FDHKPCCGULXWA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.394563019 (389h38m43.878020215s) [noise=true , address=89.149.201.147:4009 , piece_id=ET7TF7LSLL6NAIYI7DTWXOCWSL2OSQ46DAFGQPM4D7MZC4KL3MTA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.397572775 (389h38m43.881029977s) [noise=true , address=185.167.204.205:4009 , piece_id=L6JZ2XWT2MLGXCWIA55LFTZ2WE5TGBCFTJBWRTVPUTHCEYCOX24A, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.526690895 (389h38m44.010148083s) [noise=true , address=81.171.14.70:4033 , piece_id=AXEWZ3F53TIT6N7P25DHNBCDLHTYGWGLZQRICQP52GA4VTMX6IOA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.807303763 (389h38m44.290760961s) [noise=true , address=89.149.201.147:4010 , piece_id=RG7F5CWJFUAASYE7QS6Y4PGPRT45SADGUOW3W7M5I5E5G7L34FDA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.811867760 (389h38m44.295324957s) [noise=true , address=89.149.201.147:4015 , piece_id=PV7RO2L2JTENDLOVF5NGMANGIVJKKHSAJBW5N4YX5VZMXNGD5XWA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.817491869 (389h38m44.300949066s) [noise=true , address=89.149.201.147:4023 , piece_id=G4IHTFGTLH2FJCMEXJKGAUD2ZD3NSNTBK3ZWPYIKTFRVV46XQTNQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.826882849 (389h38m44.310340046s) [noise=true , address=81.171.14.70:4030 , piece_id=26XAQQBYQGZ2UXPROFLMX5ATVFOUMZ5LEXC4FZ2MGREVROH57PUA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.829999912 (389h38m44.313457109s) [noise=true , address=89.149.201.147:4002 , piece_id=J6DZWEXEPL53SI65XV7S32QNFGPHIORND42UPPQEBEFGDY3QCHJQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.833608524 (389h38m44.31706572s) [noise=true , address=89.149.201.147:4004 , piece_id=METSYHH76QGQFFGUDDPIYTEGFZX5DRBY2LB3FTULP5PQ26ZJA7AQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.833631335 (389h38m44.317088532s) [seg_pos={0 0}, upload=405070]: Satisfied cohort requirements after 26 pieces
Jan 31 21:33:50.833925231 (389h38m44.317382437s) [noise=true , address=89.149.201.147:4013 , piece_id=YA2Q5VWLCWM3HOSVFKB3VJL2JDSX6PUQMNYKOM6RQHJUXQV743JQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ZKP9g2cY7B3b8eyfHqoChUG2nYtz3nbFRGA6szgtACzT4M6J): protocol: expected piece hash; context canceled; EOF
Jan 31 21:33:50.833933563 (389h38m44.31739076s) [noise=true , address=89.149.201.147:4013 , piece_id=YA2Q5VWLCWM3HOSVFKB3VJL2JDSX6PUQMNYKOM6RQHJUXQV743JQ, seg_pos={0 0}, upload=405070]: Long tail cancellation due to optimal threshold
Jan 31 21:33:50.834025355 (389h38m44.317482562s) [noise=true , address=81.171.14.70:4039 , piece_id=TKSY6DCTWWY7UEFMTIWIP6GRG236CLB6CM34JGIGN3FN56KR3TQA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC): protocol: expected piece hash; context canceled; EOF
Jan 31 21:33:50.834046459 (389h38m44.317503672s) [noise=true , address=81.171.14.70:4039 , piece_id=TKSY6DCTWWY7UEFMTIWIP6GRG236CLB6CM34JGIGN3FN56KR3TQA, seg_pos={0 0}, upload=405070]: Long tail cancellation due to optimal threshold
Jan 31 21:33:50.834078259 (389h38m44.317535447s) [seg_pos={0 0}, upload=405070]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Jan 31 21:33:50.834187246 (389h38m44.31764446s) [upload=405070]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jan 31 21:33:50.834196158 (389h38m44.317653366s) [upload=405070]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jan 31 21:33:51.191301790 (389h38m44.67475899s) [upload=405070]: Done uploading segments...
Jan 31 21:33:51.191411925 (389h38m44.674869122s) [upload=405070]: Upload finished. err: metaclient: manager closed: closed: read tcp 159.223.143.39:47078->35.207.121.91:7777: read: connection reset by peer
Duration: 2.457062452s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..] duration:
2.592286642s trace:
https://jaeger.dev.storj.io/trace/43abdc33eb6dc242
FAIL (2026-01-31 21:35:26 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: manager closed: closed: read tcp 159.223.143.39:47078->35.207.121.91:7777: read: connection reset by peer
======LOGS======
.149.201.147:4005 , piece_id=65O63ETGC35NG5C3OWPZGQKTYRWJQ7O73P26ZC3ZFKYECETCQ3YA, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.683232901 (389h38m43.166690107s) [noise=true , address=185.167.204.206:4072 , piece_id=4VVQE6RWCILAA4JTRKSFXKN677CXXE2JFVL3NYHC7K6R54237PXQ, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.683709179 (389h38m43.167166383s) [noise=true , address=89.149.201.147:4004 , piece_id=METSYHH76QGQFFGUDDPIYTEGFZX5DRBY2LB3FTULP5PQ26ZJA7AQ, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.683724493 (389h38m43.167181697s) [noise=true , address=89.149.201.147:4010 , piece_id=RG7F5CWJFUAASYE7QS6Y4PGPRT45SADGUOW3W7M5I5E5G7L34FDA, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.684173473 (389h38m43.16763068s) [noise=true , address=185.167.204.206:4037 , piece_id=QSQ4ULBDAHRSXUJAWD4PTNVRYXO4F2MY626IKB2FDHKPCCGULXWA, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.684694161 (389h38m43.168151369s) [noise=true , address=185.167.204.205:4017 , piece_id=3LM3MACH5OSCSALKAUIOVRSBCX5HHX2VG7RTQZ6BJ4ALHRQOL7KQ, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.686216206 (389h38m43.169673413s) [noise=true , address=81.171.14.70:4034 , piece_id=QQWI3YYGGFGITTEPDDHEYGDLAGPOMHQ5NYG2MLWP6VC5ZJI77BUQ, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.688413710 (389h38m43.171870916s) [noise=true , address=89.149.201.147:4009 , piece_id=ET7TF7LSLL6NAIYI7DTWXOCWSL2OSQ46DAFGQPM4D7MZC4KL3MTA, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.690659778 (389h38m43.174116984s) [noise=true , address=185.167.204.205:4009 , piece_id=L6JZ2XWT2MLGXCWIA55LFTZ2WE5TGBCFTJBWRTVPUTHCEYCOX24A, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.692330711 (389h38m43.175787902s) [noise=true , address=89.149.201.147:4019 , piece_id=SE62AS4I6LN4AA7CSBUK7FWZJTDHUIN56FNTASDN3Y64PVNFL5VA, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:50.141023513 (389h38m43.624480718s) [upload=405070]: Next returned nil segment
Jan 31 21:33:50.141047430 (389h38m43.624504638s) [upload=405070]: Waiting for error group managing segments...
Jan 31 21:33:50.324609708 (389h38m43.808066905s) [noise=true , address=185.167.204.205:4017 , piece_id=3LM3MACH5OSCSALKAUIOVRSBCX5HHX2VG7RTQZ6BJ4ALHRQOL7KQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.325388947 (389h38m43.808846162s) [noise=true , address=81.171.14.70:4034 , piece_id=QQWI3YYGGFGITTEPDDHEYGDLAGPOMHQ5NYG2MLWP6VC5ZJI77BUQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.341012709 (389h38m43.824469907s) [noise=true , address=185.167.204.205:4023 , piece_id=WHKCMG2FOY5XSLLHJN5EWKFKSOYXFS3OBYFO4IGN26Y4DYE3H42Q, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.344607851 (389h38m43.828065048s) [noise=true , address=89.149.201.147:4019 , piece_id=SE62AS4I6LN4AA7CSBUK7FWZJTDHUIN56FNTASDN3Y64PVNFL5VA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.345220277 (389h38m43.828677475s) [noise=true , address=185.167.204.206:4061 , piece_id=OIFH6PI53RPJJ44BDBRYI4GYQYLLHLV4633VJSDZB4U2LRUMEATA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.345872439 (389h38m43.829329637s) [noise=true , address=89.149.201.147:4014 , piece_id=FDEIWM2YPK2NHAOHY5NE74Q6XB7YJLGTINJ63KADKRLYPBY2CYYQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.347218937 (389h38m43.830676136s) [noise=true , address=185.167.204.205:4027 , piece_id=JMAGQ7KEDUDN7MBXQEGQGVB5NNWUO5HU7BZATHBORQ7EJN46UF6A, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.352289470 (389h38m43.835746665s) [noise=true , address=185.167.204.206:4066 , piece_id=HCYOMLOPP6PLWUD3H5VQ3PGUJUIG7RQAS2QKEPUOCRGWPCYSVUJQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.352722623 (389h38m43.836179824s) [noise=true , address=185.167.204.206:4072 , piece_id=4VVQE6RWCILAA4JTRKSFXKN677CXXE2JFVL3NYHC7K6R54237PXQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.353039283 (389h38m43.83649647s) [noise=true , address=185.167.204.206:4070 , piece_id=4OFRPTUQ34OGUGGBT6A4J64BWZP4PFKKAJOPMODPLPI4QMNLCLAA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.353362981 (389h38m43.836820185s) [noise=true , address=185.167.204.206:4044 , piece_id=QB46W43MJBO2JI2HMWH3RIFC7YKAPTI7HJUM46YF5CTZYJC2T4NQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.354258977 (389h38m43.837716169s) [noise=true , address=185.167.204.205:4019 , piece_id=5AC555XIK5W5UPETOAQEXYBE2YWT3WTCN762EQSR2IYSU54PXDCQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.365410692 (389h38m43.848867902s) [noise=true , address=185.167.204.206:4045 , piece_id=ZZKJVEV43JFLX7L6XRKIOC2JRRC7P4B4XBZXGWLL4SSJW4YFLCZQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.368609194 (389h38m43.852066395s) [noise=true , address=89.149.201.147:4005 , piece_id=65O63ETGC35NG5C3OWPZGQKTYRWJQ7O73P26ZC3ZFKYECETCQ3YA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.371782410 (389h38m43.855239598s) [noise=true , address=185.167.204.205:4032 , piece_id=J5NVOENBLOIS5JTQ62TI62IUR4Y2P52IYLLZDK2OGGMA4UHNHRDQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.373618919 (389h38m43.857076116s) [noise=true , address=185.167.204.205:4010 , piece_id=J7G2ETYDMHDBVZKESZ22AUBEZIC52FGZU7CROLSWTOGF2SGBWQRQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.379704851 (389h38m43.863162051s) [noise=true , address=185.167.204.206:4037 , piece_id=QSQ4ULBDAHRSXUJAWD4PTNVRYXO4F2MY626IKB2FDHKPCCGULXWA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.394563019 (389h38m43.878020215s) [noise=true , address=89.149.201.147:4009 , piece_id=ET7TF7LSLL6NAIYI7DTWXOCWSL2OSQ46DAFGQPM4D7MZC4KL3MTA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.397572775 (389h38m43.881029977s) [noise=true , address=185.167.204.205:4009 , piece_id=L6JZ2XWT2MLGXCWIA55LFTZ2WE5TGBCFTJBWRTVPUTHCEYCOX24A, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.526690895 (389h38m44.010148083s) [noise=true , address=81.171.14.70:4033 , piece_id=AXEWZ3F53TIT6N7P25DHNBCDLHTYGWGLZQRICQP52GA4VTMX6IOA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.807303763 (389h38m44.290760961s) [noise=true , address=89.149.201.147:4010 , piece_id=RG7F5CWJFUAASYE7QS6Y4PGPRT45SADGUOW3W7M5I5E5G7L34FDA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.811867760 (389h38m44.295324957s) [noise=true , address=89.149.201.147:4015 , piece_id=PV7RO2L2JTENDLOVF5NGMANGIVJKKHSAJBW5N4YX5VZMXNGD5XWA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.817491869 (389h38m44.300949066s) [noise=true , address=89.149.201.147:4023 , piece_id=G4IHTFGTLH2FJCMEXJKGAUD2ZD3NSNTBK3ZWPYIKTFRVV46XQTNQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.826882849 (389h38m44.310340046s) [noise=true , address=81.171.14.70:4030 , piece_id=26XAQQBYQGZ2UXPROFLMX5ATVFOUMZ5LEXC4FZ2MGREVROH57PUA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.829999912 (389h38m44.313457109s) [noise=true , address=89.149.201.147:4002 , piece_id=J6DZWEXEPL53SI65XV7S32QNFGPHIORND42UPPQEBEFGDY3QCHJQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.833608524 (389h38m44.31706572s) [noise=true , address=89.149.201.147:4004 , piece_id=METSYHH76QGQFFGUDDPIYTEGFZX5DRBY2LB3FTULP5PQ26ZJA7AQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.833631335 (389h38m44.317088532s) [seg_pos={0 0}, upload=405070]: Satisfied cohort requirements after 26 pieces
Jan 31 21:33:50.833925231 (389h38m44.317382437s) [noise=true , address=89.149.201.147:4013 , piece_id=YA2Q5VWLCWM3HOSVFKB3VJL2JDSX6PUQMNYKOM6RQHJUXQV743JQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ZKP9g2cY7B3b8eyfHqoChUG2nYtz3nbFRGA6szgtACzT4M6J): protocol: expected piece hash; context canceled; EOF
Jan 31 21:33:50.833933563 (389h38m44.31739076s) [noise=true , address=89.149.201.147:4013 , piece_id=YA2Q5VWLCWM3HOSVFKB3VJL2JDSX6PUQMNYKOM6RQHJUXQV743JQ, seg_pos={0 0}, upload=405070]: Long tail cancellation due to optimal threshold
Jan 31 21:33:50.834025355 (389h38m44.317482562s) [noise=true , address=81.171.14.70:4039 , piece_id=TKSY6DCTWWY7UEFMTIWIP6GRG236CLB6CM34JGIGN3FN56KR3TQA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC): protocol: expected piece hash; context canceled; EOF
Jan 31 21:33:50.834046459 (389h38m44.317503672s) [noise=true , address=81.171.14.70:4039 , piece_id=TKSY6DCTWWY7UEFMTIWIP6GRG236CLB6CM34JGIGN3FN56KR3TQA, seg_pos={0 0}, upload=405070]: Long tail cancellation due to optimal threshold
Jan 31 21:33:50.834078259 (389h38m44.317535447s) [seg_pos={0 0}, upload=405070]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Jan 31 21:33:50.834187246 (389h38m44.31764446s) [upload=405070]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jan 31 21:33:50.834196158 (389h38m44.317653366s) [upload=405070]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jan 31 21:33:51.191301790 (389h38m44.67475899s) [upload=405070]: Done uploading segments...
Jan 31 21:33:51.191411925 (389h38m44.674869122s) [upload=405070]: Upload finished. err: metaclient: manager closed: closed: read tcp 159.223.143.39:47078->35.207.121.91:7777: read: connection reset by peer
Duration: 2.592318909s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.] duration:
2.081321983s trace:
https://jaeger.dev.storj.io/trace/43abdc33eb6dc242
FAIL (2026-01-31 21:34:32 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: manager closed: closed: read tcp 159.223.143.39:47078->35.207.121.91:7777: read: connection reset by peer
======LOGS======
.149.201.147:4005 , piece_id=65O63ETGC35NG5C3OWPZGQKTYRWJQ7O73P26ZC3ZFKYECETCQ3YA, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.683232901 (389h38m43.166690107s) [noise=true , address=185.167.204.206:4072 , piece_id=4VVQE6RWCILAA4JTRKSFXKN677CXXE2JFVL3NYHC7K6R54237PXQ, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.683709179 (389h38m43.167166383s) [noise=true , address=89.149.201.147:4004 , piece_id=METSYHH76QGQFFGUDDPIYTEGFZX5DRBY2LB3FTULP5PQ26ZJA7AQ, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.683724493 (389h38m43.167181697s) [noise=true , address=89.149.201.147:4010 , piece_id=RG7F5CWJFUAASYE7QS6Y4PGPRT45SADGUOW3W7M5I5E5G7L34FDA, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.684173473 (389h38m43.16763068s) [noise=true , address=185.167.204.206:4037 , piece_id=QSQ4ULBDAHRSXUJAWD4PTNVRYXO4F2MY626IKB2FDHKPCCGULXWA, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.684694161 (389h38m43.168151369s) [noise=true , address=185.167.204.205:4017 , piece_id=3LM3MACH5OSCSALKAUIOVRSBCX5HHX2VG7RTQZ6BJ4ALHRQOL7KQ, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.686216206 (389h38m43.169673413s) [noise=true , address=81.171.14.70:4034 , piece_id=QQWI3YYGGFGITTEPDDHEYGDLAGPOMHQ5NYG2MLWP6VC5ZJI77BUQ, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.688413710 (389h38m43.171870916s) [noise=true , address=89.149.201.147:4009 , piece_id=ET7TF7LSLL6NAIYI7DTWXOCWSL2OSQ46DAFGQPM4D7MZC4KL3MTA, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.690659778 (389h38m43.174116984s) [noise=true , address=185.167.204.205:4009 , piece_id=L6JZ2XWT2MLGXCWIA55LFTZ2WE5TGBCFTJBWRTVPUTHCEYCOX24A, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.692330711 (389h38m43.175787902s) [noise=true , address=89.149.201.147:4019 , piece_id=SE62AS4I6LN4AA7CSBUK7FWZJTDHUIN56FNTASDN3Y64PVNFL5VA, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:50.141023513 (389h38m43.624480718s) [upload=405070]: Next returned nil segment
Jan 31 21:33:50.141047430 (389h38m43.624504638s) [upload=405070]: Waiting for error group managing segments...
Jan 31 21:33:50.324609708 (389h38m43.808066905s) [noise=true , address=185.167.204.205:4017 , piece_id=3LM3MACH5OSCSALKAUIOVRSBCX5HHX2VG7RTQZ6BJ4ALHRQOL7KQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.325388947 (389h38m43.808846162s) [noise=true , address=81.171.14.70:4034 , piece_id=QQWI3YYGGFGITTEPDDHEYGDLAGPOMHQ5NYG2MLWP6VC5ZJI77BUQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.341012709 (389h38m43.824469907s) [noise=true , address=185.167.204.205:4023 , piece_id=WHKCMG2FOY5XSLLHJN5EWKFKSOYXFS3OBYFO4IGN26Y4DYE3H42Q, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.344607851 (389h38m43.828065048s) [noise=true , address=89.149.201.147:4019 , piece_id=SE62AS4I6LN4AA7CSBUK7FWZJTDHUIN56FNTASDN3Y64PVNFL5VA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.345220277 (389h38m43.828677475s) [noise=true , address=185.167.204.206:4061 , piece_id=OIFH6PI53RPJJ44BDBRYI4GYQYLLHLV4633VJSDZB4U2LRUMEATA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.345872439 (389h38m43.829329637s) [noise=true , address=89.149.201.147:4014 , piece_id=FDEIWM2YPK2NHAOHY5NE74Q6XB7YJLGTINJ63KADKRLYPBY2CYYQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.347218937 (389h38m43.830676136s) [noise=true , address=185.167.204.205:4027 , piece_id=JMAGQ7KEDUDN7MBXQEGQGVB5NNWUO5HU7BZATHBORQ7EJN46UF6A, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.352289470 (389h38m43.835746665s) [noise=true , address=185.167.204.206:4066 , piece_id=HCYOMLOPP6PLWUD3H5VQ3PGUJUIG7RQAS2QKEPUOCRGWPCYSVUJQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.352722623 (389h38m43.836179824s) [noise=true , address=185.167.204.206:4072 , piece_id=4VVQE6RWCILAA4JTRKSFXKN677CXXE2JFVL3NYHC7K6R54237PXQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.353039283 (389h38m43.83649647s) [noise=true , address=185.167.204.206:4070 , piece_id=4OFRPTUQ34OGUGGBT6A4J64BWZP4PFKKAJOPMODPLPI4QMNLCLAA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.353362981 (389h38m43.836820185s) [noise=true , address=185.167.204.206:4044 , piece_id=QB46W43MJBO2JI2HMWH3RIFC7YKAPTI7HJUM46YF5CTZYJC2T4NQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.354258977 (389h38m43.837716169s) [noise=true , address=185.167.204.205:4019 , piece_id=5AC555XIK5W5UPETOAQEXYBE2YWT3WTCN762EQSR2IYSU54PXDCQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.365410692 (389h38m43.848867902s) [noise=true , address=185.167.204.206:4045 , piece_id=ZZKJVEV43JFLX7L6XRKIOC2JRRC7P4B4XBZXGWLL4SSJW4YFLCZQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.368609194 (389h38m43.852066395s) [noise=true , address=89.149.201.147:4005 , piece_id=65O63ETGC35NG5C3OWPZGQKTYRWJQ7O73P26ZC3ZFKYECETCQ3YA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.371782410 (389h38m43.855239598s) [noise=true , address=185.167.204.205:4032 , piece_id=J5NVOENBLOIS5JTQ62TI62IUR4Y2P52IYLLZDK2OGGMA4UHNHRDQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.373618919 (389h38m43.857076116s) [noise=true , address=185.167.204.205:4010 , piece_id=J7G2ETYDMHDBVZKESZ22AUBEZIC52FGZU7CROLSWTOGF2SGBWQRQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.379704851 (389h38m43.863162051s) [noise=true , address=185.167.204.206:4037 , piece_id=QSQ4ULBDAHRSXUJAWD4PTNVRYXO4F2MY626IKB2FDHKPCCGULXWA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.394563019 (389h38m43.878020215s) [noise=true , address=89.149.201.147:4009 , piece_id=ET7TF7LSLL6NAIYI7DTWXOCWSL2OSQ46DAFGQPM4D7MZC4KL3MTA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.397572775 (389h38m43.881029977s) [noise=true , address=185.167.204.205:4009 , piece_id=L6JZ2XWT2MLGXCWIA55LFTZ2WE5TGBCFTJBWRTVPUTHCEYCOX24A, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.526690895 (389h38m44.010148083s) [noise=true , address=81.171.14.70:4033 , piece_id=AXEWZ3F53TIT6N7P25DHNBCDLHTYGWGLZQRICQP52GA4VTMX6IOA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.807303763 (389h38m44.290760961s) [noise=true , address=89.149.201.147:4010 , piece_id=RG7F5CWJFUAASYE7QS6Y4PGPRT45SADGUOW3W7M5I5E5G7L34FDA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.811867760 (389h38m44.295324957s) [noise=true , address=89.149.201.147:4015 , piece_id=PV7RO2L2JTENDLOVF5NGMANGIVJKKHSAJBW5N4YX5VZMXNGD5XWA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.817491869 (389h38m44.300949066s) [noise=true , address=89.149.201.147:4023 , piece_id=G4IHTFGTLH2FJCMEXJKGAUD2ZD3NSNTBK3ZWPYIKTFRVV46XQTNQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.826882849 (389h38m44.310340046s) [noise=true , address=81.171.14.70:4030 , piece_id=26XAQQBYQGZ2UXPROFLMX5ATVFOUMZ5LEXC4FZ2MGREVROH57PUA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.829999912 (389h38m44.313457109s) [noise=true , address=89.149.201.147:4002 , piece_id=J6DZWEXEPL53SI65XV7S32QNFGPHIORND42UPPQEBEFGDY3QCHJQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.833608524 (389h38m44.31706572s) [noise=true , address=89.149.201.147:4004 , piece_id=METSYHH76QGQFFGUDDPIYTEGFZX5DRBY2LB3FTULP5PQ26ZJA7AQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.833631335 (389h38m44.317088532s) [seg_pos={0 0}, upload=405070]: Satisfied cohort requirements after 26 pieces
Jan 31 21:33:50.833925231 (389h38m44.317382437s) [noise=true , address=89.149.201.147:4013 , piece_id=YA2Q5VWLCWM3HOSVFKB3VJL2JDSX6PUQMNYKOM6RQHJUXQV743JQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ZKP9g2cY7B3b8eyfHqoChUG2nYtz3nbFRGA6szgtACzT4M6J): protocol: expected piece hash; context canceled; EOF
Jan 31 21:33:50.833933563 (389h38m44.31739076s) [noise=true , address=89.149.201.147:4013 , piece_id=YA2Q5VWLCWM3HOSVFKB3VJL2JDSX6PUQMNYKOM6RQHJUXQV743JQ, seg_pos={0 0}, upload=405070]: Long tail cancellation due to optimal threshold
Jan 31 21:33:50.834025355 (389h38m44.317482562s) [noise=true , address=81.171.14.70:4039 , piece_id=TKSY6DCTWWY7UEFMTIWIP6GRG236CLB6CM34JGIGN3FN56KR3TQA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC): protocol: expected piece hash; context canceled; EOF
Jan 31 21:33:50.834046459 (389h38m44.317503672s) [noise=true , address=81.171.14.70:4039 , piece_id=TKSY6DCTWWY7UEFMTIWIP6GRG236CLB6CM34JGIGN3FN56KR3TQA, seg_pos={0 0}, upload=405070]: Long tail cancellation due to optimal threshold
Jan 31 21:33:50.834078259 (389h38m44.317535447s) [seg_pos={0 0}, upload=405070]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Jan 31 21:33:50.834187246 (389h38m44.31764446s) [upload=405070]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jan 31 21:33:50.834196158 (389h38m44.317653366s) [upload=405070]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jan 31 21:33:51.191301790 (389h38m44.67475899s) [upload=405070]: Done uploading segments...
Jan 31 21:33:51.191411925 (389h38m44.674869122s) [upload=405070]: Upload finished. err: metaclient: manager closed: closed: read tcp 159.223.143.39:47078->35.207.121.91:7777: read: connection reset by peer
Duration: 2.081415484s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
1.925657918s trace:
https://jaeger.dev.storj.io/trace/43abdc33eb6dc242
FAIL (2026-01-31 21:33:51 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: manager closed: closed: read tcp 159.223.143.39:47078->35.207.121.91:7777: read: connection reset by peer
======LOGS======
.149.201.147:4005 , piece_id=65O63ETGC35NG5C3OWPZGQKTYRWJQ7O73P26ZC3ZFKYECETCQ3YA, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.683232901 (389h38m43.166690107s) [noise=true , address=185.167.204.206:4072 , piece_id=4VVQE6RWCILAA4JTRKSFXKN677CXXE2JFVL3NYHC7K6R54237PXQ, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.683709179 (389h38m43.167166383s) [noise=true , address=89.149.201.147:4004 , piece_id=METSYHH76QGQFFGUDDPIYTEGFZX5DRBY2LB3FTULP5PQ26ZJA7AQ, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.683724493 (389h38m43.167181697s) [noise=true , address=89.149.201.147:4010 , piece_id=RG7F5CWJFUAASYE7QS6Y4PGPRT45SADGUOW3W7M5I5E5G7L34FDA, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.684173473 (389h38m43.16763068s) [noise=true , address=185.167.204.206:4037 , piece_id=QSQ4ULBDAHRSXUJAWD4PTNVRYXO4F2MY626IKB2FDHKPCCGULXWA, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.684694161 (389h38m43.168151369s) [noise=true , address=185.167.204.205:4017 , piece_id=3LM3MACH5OSCSALKAUIOVRSBCX5HHX2VG7RTQZ6BJ4ALHRQOL7KQ, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.686216206 (389h38m43.169673413s) [noise=true , address=81.171.14.70:4034 , piece_id=QQWI3YYGGFGITTEPDDHEYGDLAGPOMHQ5NYG2MLWP6VC5ZJI77BUQ, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.688413710 (389h38m43.171870916s) [noise=true , address=89.149.201.147:4009 , piece_id=ET7TF7LSLL6NAIYI7DTWXOCWSL2OSQ46DAFGQPM4D7MZC4KL3MTA, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.690659778 (389h38m43.174116984s) [noise=true , address=185.167.204.205:4009 , piece_id=L6JZ2XWT2MLGXCWIA55LFTZ2WE5TGBCFTJBWRTVPUTHCEYCOX24A, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:49.692330711 (389h38m43.175787902s) [noise=true , address=89.149.201.147:4019 , piece_id=SE62AS4I6LN4AA7CSBUK7FWZJTDHUIN56FNTASDN3Y64PVNFL5VA, seg_pos={0 0}, upload=405070]: Uploading piece...
Jan 31 21:33:50.141023513 (389h38m43.624480718s) [upload=405070]: Next returned nil segment
Jan 31 21:33:50.141047430 (389h38m43.624504638s) [upload=405070]: Waiting for error group managing segments...
Jan 31 21:33:50.324609708 (389h38m43.808066905s) [noise=true , address=185.167.204.205:4017 , piece_id=3LM3MACH5OSCSALKAUIOVRSBCX5HHX2VG7RTQZ6BJ4ALHRQOL7KQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.325388947 (389h38m43.808846162s) [noise=true , address=81.171.14.70:4034 , piece_id=QQWI3YYGGFGITTEPDDHEYGDLAGPOMHQ5NYG2MLWP6VC5ZJI77BUQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.341012709 (389h38m43.824469907s) [noise=true , address=185.167.204.205:4023 , piece_id=WHKCMG2FOY5XSLLHJN5EWKFKSOYXFS3OBYFO4IGN26Y4DYE3H42Q, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.344607851 (389h38m43.828065048s) [noise=true , address=89.149.201.147:4019 , piece_id=SE62AS4I6LN4AA7CSBUK7FWZJTDHUIN56FNTASDN3Y64PVNFL5VA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.345220277 (389h38m43.828677475s) [noise=true , address=185.167.204.206:4061 , piece_id=OIFH6PI53RPJJ44BDBRYI4GYQYLLHLV4633VJSDZB4U2LRUMEATA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.345872439 (389h38m43.829329637s) [noise=true , address=89.149.201.147:4014 , piece_id=FDEIWM2YPK2NHAOHY5NE74Q6XB7YJLGTINJ63KADKRLYPBY2CYYQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.347218937 (389h38m43.830676136s) [noise=true , address=185.167.204.205:4027 , piece_id=JMAGQ7KEDUDN7MBXQEGQGVB5NNWUO5HU7BZATHBORQ7EJN46UF6A, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.352289470 (389h38m43.835746665s) [noise=true , address=185.167.204.206:4066 , piece_id=HCYOMLOPP6PLWUD3H5VQ3PGUJUIG7RQAS2QKEPUOCRGWPCYSVUJQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.352722623 (389h38m43.836179824s) [noise=true , address=185.167.204.206:4072 , piece_id=4VVQE6RWCILAA4JTRKSFXKN677CXXE2JFVL3NYHC7K6R54237PXQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.353039283 (389h38m43.83649647s) [noise=true , address=185.167.204.206:4070 , piece_id=4OFRPTUQ34OGUGGBT6A4J64BWZP4PFKKAJOPMODPLPI4QMNLCLAA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.353362981 (389h38m43.836820185s) [noise=true , address=185.167.204.206:4044 , piece_id=QB46W43MJBO2JI2HMWH3RIFC7YKAPTI7HJUM46YF5CTZYJC2T4NQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.354258977 (389h38m43.837716169s) [noise=true , address=185.167.204.205:4019 , piece_id=5AC555XIK5W5UPETOAQEXYBE2YWT3WTCN762EQSR2IYSU54PXDCQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.365410692 (389h38m43.848867902s) [noise=true , address=185.167.204.206:4045 , piece_id=ZZKJVEV43JFLX7L6XRKIOC2JRRC7P4B4XBZXGWLL4SSJW4YFLCZQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.368609194 (389h38m43.852066395s) [noise=true , address=89.149.201.147:4005 , piece_id=65O63ETGC35NG5C3OWPZGQKTYRWJQ7O73P26ZC3ZFKYECETCQ3YA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.371782410 (389h38m43.855239598s) [noise=true , address=185.167.204.205:4032 , piece_id=J5NVOENBLOIS5JTQ62TI62IUR4Y2P52IYLLZDK2OGGMA4UHNHRDQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.373618919 (389h38m43.857076116s) [noise=true , address=185.167.204.205:4010 , piece_id=J7G2ETYDMHDBVZKESZ22AUBEZIC52FGZU7CROLSWTOGF2SGBWQRQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.379704851 (389h38m43.863162051s) [noise=true , address=185.167.204.206:4037 , piece_id=QSQ4ULBDAHRSXUJAWD4PTNVRYXO4F2MY626IKB2FDHKPCCGULXWA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.394563019 (389h38m43.878020215s) [noise=true , address=89.149.201.147:4009 , piece_id=ET7TF7LSLL6NAIYI7DTWXOCWSL2OSQ46DAFGQPM4D7MZC4KL3MTA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.397572775 (389h38m43.881029977s) [noise=true , address=185.167.204.205:4009 , piece_id=L6JZ2XWT2MLGXCWIA55LFTZ2WE5TGBCFTJBWRTVPUTHCEYCOX24A, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.526690895 (389h38m44.010148083s) [noise=true , address=81.171.14.70:4033 , piece_id=AXEWZ3F53TIT6N7P25DHNBCDLHTYGWGLZQRICQP52GA4VTMX6IOA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.807303763 (389h38m44.290760961s) [noise=true , address=89.149.201.147:4010 , piece_id=RG7F5CWJFUAASYE7QS6Y4PGPRT45SADGUOW3W7M5I5E5G7L34FDA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.811867760 (389h38m44.295324957s) [noise=true , address=89.149.201.147:4015 , piece_id=PV7RO2L2JTENDLOVF5NGMANGIVJKKHSAJBW5N4YX5VZMXNGD5XWA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.817491869 (389h38m44.300949066s) [noise=true , address=89.149.201.147:4023 , piece_id=G4IHTFGTLH2FJCMEXJKGAUD2ZD3NSNTBK3ZWPYIKTFRVV46XQTNQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.826882849 (389h38m44.310340046s) [noise=true , address=81.171.14.70:4030 , piece_id=26XAQQBYQGZ2UXPROFLMX5ATVFOUMZ5LEXC4FZ2MGREVROH57PUA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.829999912 (389h38m44.313457109s) [noise=true , address=89.149.201.147:4002 , piece_id=J6DZWEXEPL53SI65XV7S32QNFGPHIORND42UPPQEBEFGDY3QCHJQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.833608524 (389h38m44.31706572s) [noise=true , address=89.149.201.147:4004 , piece_id=METSYHH76QGQFFGUDDPIYTEGFZX5DRBY2LB3FTULP5PQ26ZJA7AQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: <nil>
Jan 31 21:33:50.833631335 (389h38m44.317088532s) [seg_pos={0 0}, upload=405070]: Satisfied cohort requirements after 26 pieces
Jan 31 21:33:50.833925231 (389h38m44.317382437s) [noise=true , address=89.149.201.147:4013 , piece_id=YA2Q5VWLCWM3HOSVFKB3VJL2JDSX6PUQMNYKOM6RQHJUXQV743JQ, seg_pos={0 0}, upload=405070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ZKP9g2cY7B3b8eyfHqoChUG2nYtz3nbFRGA6szgtACzT4M6J): protocol: expected piece hash; context canceled; EOF
Jan 31 21:33:50.833933563 (389h38m44.31739076s) [noise=true , address=89.149.201.147:4013 , piece_id=YA2Q5VWLCWM3HOSVFKB3VJL2JDSX6PUQMNYKOM6RQHJUXQV743JQ, seg_pos={0 0}, upload=405070]: Long tail cancellation due to optimal threshold
Jan 31 21:33:50.834025355 (389h38m44.317482562s) [noise=true , address=81.171.14.70:4039 , piece_id=TKSY6DCTWWY7UEFMTIWIP6GRG236CLB6CM34JGIGN3FN56KR3TQA, seg_pos={0 0}, upload=405070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC): protocol: expected piece hash; context canceled; EOF
Jan 31 21:33:50.834046459 (389h38m44.317503672s) [noise=true , address=81.171.14.70:4039 , piece_id=TKSY6DCTWWY7UEFMTIWIP6GRG236CLB6CM34JGIGN3FN56KR3TQA, seg_pos={0 0}, upload=405070]: Long tail cancellation due to optimal threshold
Jan 31 21:33:50.834078259 (389h38m44.317535447s) [seg_pos={0 0}, upload=405070]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Jan 31 21:33:50.834187246 (389h38m44.31764446s) [upload=405070]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jan 31 21:33:50.834196158 (389h38m44.317653366s) [upload=405070]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jan 31 21:33:51.191301790 (389h38m44.67475899s) [upload=405070]: Done uploading segments...
Jan 31 21:33:51.191411925 (389h38m44.674869122s) [upload=405070]: Upload finished. err: metaclient: manager closed: closed: read tcp 159.223.143.39:47078->35.207.121.91:7777: read: connection reset by peer
Duration: 1.925685122s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
497.111537ms trace:
https://jaeger.dev.storj.io/trace/62a1509ecc5b5058
FAIL (2026-01-31 09:28:29 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Jan 31 08:42:01.166771110 (376h46m54.650228297s) [upload=391707]: Starting upload
Jan 31 08:42:01.166986060 (376h46m54.650443258s) [upload=391707]: Done starting upload
Jan 31 08:42:01.167072896 (376h46m54.650530094s) [upload=391707]: Uploading segments...
Jan 31 08:42:01.167397789 (376h46m54.650854991s) [upload=391707]: Got next segment. Inline: false
Jan 31 08:42:01.167473452 (376h46m54.650930657s) [upload=391707]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jan 31 08:42:01.167480315 (376h46m54.650937524s) [upload=391707]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jan 31 08:42:01.195346764 (376h46m54.678803969s) [seg_pos={0 0}, upload=391707]: Begin upload segment...
Jan 31 08:42:01.195428575 (376h46m54.678885757s) [seg_pos={0 0}, upload=391707]: Done begin upload segment.
Jan 31 08:42:01.195463009 (376h46m54.678920214s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.195847779 (376h46m54.679304986s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.197051385 (376h46m54.680508591s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.197594770 (376h46m54.681051978s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.198049760 (376h46m54.681506963s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.198528799 (376h46m54.681986007s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.198996489 (376h46m54.682453701s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.199526016 (376h46m54.682983298s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.199772893 (376h46m54.683230102s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.200006077 (376h46m54.683463277s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.200298977 (376h46m54.68375618s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.200764860 (376h46m54.684222068s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.201330795 (376h46m54.68478802s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.201862657 (376h46m54.685319865s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.202435299 (376h46m54.685892502s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.203149033 (376h46m54.686606237s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.288234960 (376h46m54.771692157s) [upload=391707]: Next returned nil segment
Jan 31 08:42:01.288251604 (376h46m54.771708818s) [upload=391707]: Waiting for error group managing segments...
Jan 31 08:42:01.293400797 (376h46m54.776858002s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.294083673 (376h46m54.777540862s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.294285106 (376h46m54.777742305s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.295108197 (376h46m54.778565394s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.295172272 (376h46m54.778629467s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.295900158 (376h46m54.779357349s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.295955977 (376h46m54.779413173s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.296392614 (376h46m54.779849814s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.296441340 (376h46m54.779898537s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.296874674 (376h46m54.780331861s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.296909101 (376h46m54.780366298s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.297409404 (376h46m54.780866604s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:31.121919846 (376h47m24.605377048s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: read tcp 159.223.143.39:60086->45.91.116.7:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 31 08:42:31.122001831 (376h47m24.605459038s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: read tcp 159.223.143.39:60218->45.91.116.5:4297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 31 08:42:31.122048345 (376h47m24.605505567s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: read tcp 159.223.143.39:45254->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 31 08:42:31.122106214 (376h47m24.605563415s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12qpELr7Yr1aJtgau8q34qPBtADW6QAXPNySAzcRLB39hXoXueo, address:45.91.116.8:4070): protocol: read tcp 159.223.143.39:50716->45.91.116.8:4070: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 31 08:42:31.122131285 (376h47m24.605588469s) [seg_pos={0 0}, upload=391707]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Jan 31 08:42:31.141063957 (376h47m24.624521154s) [upload=391707]: Done uploading segments...
Jan 31 08:42:31.141101292 (376h47m24.624558493s) [upload=391707]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 497.210455ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
499.211217ms trace:
https://jaeger.dev.storj.io/trace/62a1509ecc5b5058
FAIL (2026-01-31 09:27:22 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Jan 31 08:42:01.166771110 (376h46m54.650228297s) [upload=391707]: Starting upload
Jan 31 08:42:01.166986060 (376h46m54.650443258s) [upload=391707]: Done starting upload
Jan 31 08:42:01.167072896 (376h46m54.650530094s) [upload=391707]: Uploading segments...
Jan 31 08:42:01.167397789 (376h46m54.650854991s) [upload=391707]: Got next segment. Inline: false
Jan 31 08:42:01.167473452 (376h46m54.650930657s) [upload=391707]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jan 31 08:42:01.167480315 (376h46m54.650937524s) [upload=391707]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jan 31 08:42:01.195346764 (376h46m54.678803969s) [seg_pos={0 0}, upload=391707]: Begin upload segment...
Jan 31 08:42:01.195428575 (376h46m54.678885757s) [seg_pos={0 0}, upload=391707]: Done begin upload segment.
Jan 31 08:42:01.195463009 (376h46m54.678920214s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.195847779 (376h46m54.679304986s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.197051385 (376h46m54.680508591s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.197594770 (376h46m54.681051978s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.198049760 (376h46m54.681506963s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.198528799 (376h46m54.681986007s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.198996489 (376h46m54.682453701s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.199526016 (376h46m54.682983298s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.199772893 (376h46m54.683230102s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.200006077 (376h46m54.683463277s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.200298977 (376h46m54.68375618s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.200764860 (376h46m54.684222068s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.201330795 (376h46m54.68478802s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.201862657 (376h46m54.685319865s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.202435299 (376h46m54.685892502s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.203149033 (376h46m54.686606237s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.288234960 (376h46m54.771692157s) [upload=391707]: Next returned nil segment
Jan 31 08:42:01.288251604 (376h46m54.771708818s) [upload=391707]: Waiting for error group managing segments...
Jan 31 08:42:01.293400797 (376h46m54.776858002s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.294083673 (376h46m54.777540862s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.294285106 (376h46m54.777742305s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.295108197 (376h46m54.778565394s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.295172272 (376h46m54.778629467s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.295900158 (376h46m54.779357349s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.295955977 (376h46m54.779413173s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.296392614 (376h46m54.779849814s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.296441340 (376h46m54.779898537s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.296874674 (376h46m54.780331861s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.296909101 (376h46m54.780366298s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.297409404 (376h46m54.780866604s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:31.121919846 (376h47m24.605377048s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: read tcp 159.223.143.39:60086->45.91.116.7:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 31 08:42:31.122001831 (376h47m24.605459038s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: read tcp 159.223.143.39:60218->45.91.116.5:4297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 31 08:42:31.122048345 (376h47m24.605505567s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: read tcp 159.223.143.39:45254->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 31 08:42:31.122106214 (376h47m24.605563415s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12qpELr7Yr1aJtgau8q34qPBtADW6QAXPNySAzcRLB39hXoXueo, address:45.91.116.8:4070): protocol: read tcp 159.223.143.39:50716->45.91.116.8:4070: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 31 08:42:31.122131285 (376h47m24.605588469s) [seg_pos={0 0}, upload=391707]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Jan 31 08:42:31.141063957 (376h47m24.624521154s) [upload=391707]: Done uploading segments...
Jan 31 08:42:31.141101292 (376h47m24.624558493s) [upload=391707]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 499.242364ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
618.174081ms trace:
https://jaeger.dev.storj.io/trace/62a1509ecc5b5058
FAIL (2026-01-31 09:26:55 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Jan 31 08:42:01.166771110 (376h46m54.650228297s) [upload=391707]: Starting upload
Jan 31 08:42:01.166986060 (376h46m54.650443258s) [upload=391707]: Done starting upload
Jan 31 08:42:01.167072896 (376h46m54.650530094s) [upload=391707]: Uploading segments...
Jan 31 08:42:01.167397789 (376h46m54.650854991s) [upload=391707]: Got next segment. Inline: false
Jan 31 08:42:01.167473452 (376h46m54.650930657s) [upload=391707]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jan 31 08:42:01.167480315 (376h46m54.650937524s) [upload=391707]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jan 31 08:42:01.195346764 (376h46m54.678803969s) [seg_pos={0 0}, upload=391707]: Begin upload segment...
Jan 31 08:42:01.195428575 (376h46m54.678885757s) [seg_pos={0 0}, upload=391707]: Done begin upload segment.
Jan 31 08:42:01.195463009 (376h46m54.678920214s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.195847779 (376h46m54.679304986s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.197051385 (376h46m54.680508591s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.197594770 (376h46m54.681051978s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.198049760 (376h46m54.681506963s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.198528799 (376h46m54.681986007s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.198996489 (376h46m54.682453701s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.199526016 (376h46m54.682983298s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.199772893 (376h46m54.683230102s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.200006077 (376h46m54.683463277s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.200298977 (376h46m54.68375618s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.200764860 (376h46m54.684222068s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.201330795 (376h46m54.68478802s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.201862657 (376h46m54.685319865s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.202435299 (376h46m54.685892502s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.203149033 (376h46m54.686606237s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.288234960 (376h46m54.771692157s) [upload=391707]: Next returned nil segment
Jan 31 08:42:01.288251604 (376h46m54.771708818s) [upload=391707]: Waiting for error group managing segments...
Jan 31 08:42:01.293400797 (376h46m54.776858002s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.294083673 (376h46m54.777540862s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.294285106 (376h46m54.777742305s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.295108197 (376h46m54.778565394s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.295172272 (376h46m54.778629467s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.295900158 (376h46m54.779357349s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.295955977 (376h46m54.779413173s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.296392614 (376h46m54.779849814s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.296441340 (376h46m54.779898537s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.296874674 (376h46m54.780331861s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.296909101 (376h46m54.780366298s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.297409404 (376h46m54.780866604s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:31.121919846 (376h47m24.605377048s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: read tcp 159.223.143.39:60086->45.91.116.7:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 31 08:42:31.122001831 (376h47m24.605459038s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: read tcp 159.223.143.39:60218->45.91.116.5:4297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 31 08:42:31.122048345 (376h47m24.605505567s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: read tcp 159.223.143.39:45254->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 31 08:42:31.122106214 (376h47m24.605563415s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12qpELr7Yr1aJtgau8q34qPBtADW6QAXPNySAzcRLB39hXoXueo, address:45.91.116.8:4070): protocol: read tcp 159.223.143.39:50716->45.91.116.8:4070: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 31 08:42:31.122131285 (376h47m24.605588469s) [seg_pos={0 0}, upload=391707]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Jan 31 08:42:31.141063957 (376h47m24.624521154s) [upload=391707]: Done uploading segments...
Jan 31 08:42:31.141101292 (376h47m24.624558493s) [upload=391707]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 618.260913ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
466.117108ms trace:
https://jaeger.dev.storj.io/trace/33d822523a7c81ef
FAIL (2026-01-31 09:26:24 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 466.143585ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [X...........................................................] duration:
418.199998ms trace:
https://jaeger.dev.storj.io/trace/6c4cbd1edfce8e6
FAIL (2026-01-31 09:25:56 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 418.306634ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................] duration:
409.960751ms trace:
https://jaeger.dev.storj.io/trace/62a1509ecc5b5058
FAIL (2026-01-31 09:25:53 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Jan 31 08:42:01.166771110 (376h46m54.650228297s) [upload=391707]: Starting upload
Jan 31 08:42:01.166986060 (376h46m54.650443258s) [upload=391707]: Done starting upload
Jan 31 08:42:01.167072896 (376h46m54.650530094s) [upload=391707]: Uploading segments...
Jan 31 08:42:01.167397789 (376h46m54.650854991s) [upload=391707]: Got next segment. Inline: false
Jan 31 08:42:01.167473452 (376h46m54.650930657s) [upload=391707]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jan 31 08:42:01.167480315 (376h46m54.650937524s) [upload=391707]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jan 31 08:42:01.195346764 (376h46m54.678803969s) [seg_pos={0 0}, upload=391707]: Begin upload segment...
Jan 31 08:42:01.195428575 (376h46m54.678885757s) [seg_pos={0 0}, upload=391707]: Done begin upload segment.
Jan 31 08:42:01.195463009 (376h46m54.678920214s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.195847779 (376h46m54.679304986s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.197051385 (376h46m54.680508591s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.197594770 (376h46m54.681051978s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.198049760 (376h46m54.681506963s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.198528799 (376h46m54.681986007s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.198996489 (376h46m54.682453701s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.199526016 (376h46m54.682983298s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.199772893 (376h46m54.683230102s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.200006077 (376h46m54.683463277s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.200298977 (376h46m54.68375618s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.200764860 (376h46m54.684222068s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.201330795 (376h46m54.68478802s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.201862657 (376h46m54.685319865s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.202435299 (376h46m54.685892502s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.203149033 (376h46m54.686606237s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.288234960 (376h46m54.771692157s) [upload=391707]: Next returned nil segment
Jan 31 08:42:01.288251604 (376h46m54.771708818s) [upload=391707]: Waiting for error group managing segments...
Jan 31 08:42:01.293400797 (376h46m54.776858002s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.294083673 (376h46m54.777540862s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.294285106 (376h46m54.777742305s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.295108197 (376h46m54.778565394s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.295172272 (376h46m54.778629467s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.295900158 (376h46m54.779357349s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.295955977 (376h46m54.779413173s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.296392614 (376h46m54.779849814s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.296441340 (376h46m54.779898537s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.296874674 (376h46m54.780331861s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.296909101 (376h46m54.780366298s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.297409404 (376h46m54.780866604s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:31.121919846 (376h47m24.605377048s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: read tcp 159.223.143.39:60086->45.91.116.7:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 31 08:42:31.122001831 (376h47m24.605459038s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: read tcp 159.223.143.39:60218->45.91.116.5:4297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 31 08:42:31.122048345 (376h47m24.605505567s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: read tcp 159.223.143.39:45254->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 31 08:42:31.122106214 (376h47m24.605563415s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12qpELr7Yr1aJtgau8q34qPBtADW6QAXPNySAzcRLB39hXoXueo, address:45.91.116.8:4070): protocol: read tcp 159.223.143.39:50716->45.91.116.8:4070: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 31 08:42:31.122131285 (376h47m24.605588469s) [seg_pos={0 0}, upload=391707]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Jan 31 08:42:31.141063957 (376h47m24.624521154s) [upload=391707]: Done uploading segments...
Jan 31 08:42:31.141101292 (376h47m24.624558493s) [upload=391707]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 409.977778ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.X..........................................................] duration:
485.613709ms trace:
https://jaeger.dev.storj.io/trace/6c4cbd1edfce8e6
FAIL (2026-01-31 09:25:22 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 485.638804ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
448.395552ms trace:
https://jaeger.dev.storj.io/trace/33d822523a7c81ef
FAIL (2026-01-31 09:25:09 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 448.432813ms
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
482.810115ms trace:
https://jaeger.dev.storj.io/trace/62a1509ecc5b5058
FAIL (2026-01-31 09:24:43 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Jan 31 08:42:01.166771110 (376h46m54.650228297s) [upload=391707]: Starting upload
Jan 31 08:42:01.166986060 (376h46m54.650443258s) [upload=391707]: Done starting upload
Jan 31 08:42:01.167072896 (376h46m54.650530094s) [upload=391707]: Uploading segments...
Jan 31 08:42:01.167397789 (376h46m54.650854991s) [upload=391707]: Got next segment. Inline: false
Jan 31 08:42:01.167473452 (376h46m54.650930657s) [upload=391707]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jan 31 08:42:01.167480315 (376h46m54.650937524s) [upload=391707]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jan 31 08:42:01.195346764 (376h46m54.678803969s) [seg_pos={0 0}, upload=391707]: Begin upload segment...
Jan 31 08:42:01.195428575 (376h46m54.678885757s) [seg_pos={0 0}, upload=391707]: Done begin upload segment.
Jan 31 08:42:01.195463009 (376h46m54.678920214s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.195847779 (376h46m54.679304986s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.197051385 (376h46m54.680508591s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.197594770 (376h46m54.681051978s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.198049760 (376h46m54.681506963s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.198528799 (376h46m54.681986007s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.198996489 (376h46m54.682453701s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.199526016 (376h46m54.682983298s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.199772893 (376h46m54.683230102s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.200006077 (376h46m54.683463277s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.200298977 (376h46m54.68375618s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.200764860 (376h46m54.684222068s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.201330795 (376h46m54.68478802s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.201862657 (376h46m54.685319865s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.202435299 (376h46m54.685892502s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.203149033 (376h46m54.686606237s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.288234960 (376h46m54.771692157s) [upload=391707]: Next returned nil segment
Jan 31 08:42:01.288251604 (376h46m54.771708818s) [upload=391707]: Waiting for error group managing segments...
Jan 31 08:42:01.293400797 (376h46m54.776858002s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.294083673 (376h46m54.777540862s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.294285106 (376h46m54.777742305s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.295108197 (376h46m54.778565394s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.295172272 (376h46m54.778629467s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.295900158 (376h46m54.779357349s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.295955977 (376h46m54.779413173s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.296392614 (376h46m54.779849814s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.296441340 (376h46m54.779898537s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.296874674 (376h46m54.780331861s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.296909101 (376h46m54.780366298s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.297409404 (376h46m54.780866604s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:31.121919846 (376h47m24.605377048s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: read tcp 159.223.143.39:60086->45.91.116.7:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 31 08:42:31.122001831 (376h47m24.605459038s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: read tcp 159.223.143.39:60218->45.91.116.5:4297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 31 08:42:31.122048345 (376h47m24.605505567s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: read tcp 159.223.143.39:45254->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 31 08:42:31.122106214 (376h47m24.605563415s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12qpELr7Yr1aJtgau8q34qPBtADW6QAXPNySAzcRLB39hXoXueo, address:45.91.116.8:4070): protocol: read tcp 159.223.143.39:50716->45.91.116.8:4070: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 31 08:42:31.122131285 (376h47m24.605588469s) [seg_pos={0 0}, upload=391707]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Jan 31 08:42:31.141063957 (376h47m24.624521154s) [upload=391707]: Done uploading segments...
Jan 31 08:42:31.141101292 (376h47m24.624558493s) [upload=391707]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 482.842229ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [..X.........................................................] duration:
437.37468ms trace:
https://jaeger.dev.storj.io/trace/6c4cbd1edfce8e6
FAIL (2026-01-31 09:24:30 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 437.402253ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
488.019769ms trace:
https://jaeger.dev.storj.io/trace/33d822523a7c81ef
FAIL (2026-01-31 09:24:17 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 488.051238ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [...X........................................................] duration:
422.494163ms trace:
https://jaeger.dev.storj.io/trace/6c4cbd1edfce8e6
FAIL (2026-01-31 09:24:01 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 422.537051ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................] duration:
517.836343ms trace:
https://jaeger.dev.storj.io/trace/62a1509ecc5b5058
FAIL (2026-01-31 09:23:41 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Jan 31 08:42:01.166771110 (376h46m54.650228297s) [upload=391707]: Starting upload
Jan 31 08:42:01.166986060 (376h46m54.650443258s) [upload=391707]: Done starting upload
Jan 31 08:42:01.167072896 (376h46m54.650530094s) [upload=391707]: Uploading segments...
Jan 31 08:42:01.167397789 (376h46m54.650854991s) [upload=391707]: Got next segment. Inline: false
Jan 31 08:42:01.167473452 (376h46m54.650930657s) [upload=391707]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jan 31 08:42:01.167480315 (376h46m54.650937524s) [upload=391707]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jan 31 08:42:01.195346764 (376h46m54.678803969s) [seg_pos={0 0}, upload=391707]: Begin upload segment...
Jan 31 08:42:01.195428575 (376h46m54.678885757s) [seg_pos={0 0}, upload=391707]: Done begin upload segment.
Jan 31 08:42:01.195463009 (376h46m54.678920214s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.195847779 (376h46m54.679304986s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.197051385 (376h46m54.680508591s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.197594770 (376h46m54.681051978s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.198049760 (376h46m54.681506963s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.198528799 (376h46m54.681986007s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.198996489 (376h46m54.682453701s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.199526016 (376h46m54.682983298s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.199772893 (376h46m54.683230102s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.200006077 (376h46m54.683463277s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.200298977 (376h46m54.68375618s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.200764860 (376h46m54.684222068s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.201330795 (376h46m54.68478802s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.201862657 (376h46m54.685319865s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.202435299 (376h46m54.685892502s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.203149033 (376h46m54.686606237s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.288234960 (376h46m54.771692157s) [upload=391707]: Next returned nil segment
Jan 31 08:42:01.288251604 (376h46m54.771708818s) [upload=391707]: Waiting for error group managing segments...
Jan 31 08:42:01.293400797 (376h46m54.776858002s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.294083673 (376h46m54.777540862s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.294285106 (376h46m54.777742305s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.295108197 (376h46m54.778565394s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.295172272 (376h46m54.778629467s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.295900158 (376h46m54.779357349s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.295955977 (376h46m54.779413173s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.296392614 (376h46m54.779849814s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.296441340 (376h46m54.779898537s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.296874674 (376h46m54.780331861s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.296909101 (376h46m54.780366298s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.297409404 (376h46m54.780866604s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:31.121919846 (376h47m24.605377048s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: read tcp 159.223.143.39:60086->45.91.116.7:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 31 08:42:31.122001831 (376h47m24.605459038s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: read tcp 159.223.143.39:60218->45.91.116.5:4297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 31 08:42:31.122048345 (376h47m24.605505567s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: read tcp 159.223.143.39:45254->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 31 08:42:31.122106214 (376h47m24.605563415s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12qpELr7Yr1aJtgau8q34qPBtADW6QAXPNySAzcRLB39hXoXueo, address:45.91.116.8:4070): protocol: read tcp 159.223.143.39:50716->45.91.116.8:4070: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 31 08:42:31.122131285 (376h47m24.605588469s) [seg_pos={0 0}, upload=391707]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Jan 31 08:42:31.141063957 (376h47m24.624521154s) [upload=391707]: Done uploading segments...
Jan 31 08:42:31.141101292 (376h47m24.624558493s) [upload=391707]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 517.870433ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................] duration:
487.659119ms trace:
https://jaeger.dev.storj.io/trace/33d822523a7c81ef
FAIL (2026-01-31 09:23:21 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 487.687313ms
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......X.....................................................] duration:
415.044304ms trace:
https://jaeger.dev.storj.io/trace/62a1509ecc5b5058
FAIL (2026-01-31 09:23:13 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Jan 31 08:42:01.166771110 (376h46m54.650228297s) [upload=391707]: Starting upload
Jan 31 08:42:01.166986060 (376h46m54.650443258s) [upload=391707]: Done starting upload
Jan 31 08:42:01.167072896 (376h46m54.650530094s) [upload=391707]: Uploading segments...
Jan 31 08:42:01.167397789 (376h46m54.650854991s) [upload=391707]: Got next segment. Inline: false
Jan 31 08:42:01.167473452 (376h46m54.650930657s) [upload=391707]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jan 31 08:42:01.167480315 (376h46m54.650937524s) [upload=391707]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jan 31 08:42:01.195346764 (376h46m54.678803969s) [seg_pos={0 0}, upload=391707]: Begin upload segment...
Jan 31 08:42:01.195428575 (376h46m54.678885757s) [seg_pos={0 0}, upload=391707]: Done begin upload segment.
Jan 31 08:42:01.195463009 (376h46m54.678920214s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.195847779 (376h46m54.679304986s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.197051385 (376h46m54.680508591s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.197594770 (376h46m54.681051978s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.198049760 (376h46m54.681506963s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.198528799 (376h46m54.681986007s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.198996489 (376h46m54.682453701s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.199526016 (376h46m54.682983298s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.199772893 (376h46m54.683230102s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.200006077 (376h46m54.683463277s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.200298977 (376h46m54.68375618s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.200764860 (376h46m54.684222068s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.201330795 (376h46m54.68478802s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.201862657 (376h46m54.685319865s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.202435299 (376h46m54.685892502s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.203149033 (376h46m54.686606237s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.288234960 (376h46m54.771692157s) [upload=391707]: Next returned nil segment
Jan 31 08:42:01.288251604 (376h46m54.771708818s) [upload=391707]: Waiting for error group managing segments...
Jan 31 08:42:01.293400797 (376h46m54.776858002s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.294083673 (376h46m54.777540862s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.294285106 (376h46m54.777742305s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.295108197 (376h46m54.778565394s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.295172272 (376h46m54.778629467s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.295900158 (376h46m54.779357349s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.295955977 (376h46m54.779413173s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.296392614 (376h46m54.779849814s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.296441340 (376h46m54.779898537s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.296874674 (376h46m54.780331861s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.296909101 (376h46m54.780366298s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.297409404 (376h46m54.780866604s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:31.121919846 (376h47m24.605377048s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: read tcp 159.223.143.39:60086->45.91.116.7:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 31 08:42:31.122001831 (376h47m24.605459038s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: read tcp 159.223.143.39:60218->45.91.116.5:4297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 31 08:42:31.122048345 (376h47m24.605505567s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: read tcp 159.223.143.39:45254->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 31 08:42:31.122106214 (376h47m24.605563415s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12qpELr7Yr1aJtgau8q34qPBtADW6QAXPNySAzcRLB39hXoXueo, address:45.91.116.8:4070): protocol: read tcp 159.223.143.39:50716->45.91.116.8:4070: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 31 08:42:31.122131285 (376h47m24.605588469s) [seg_pos={0 0}, upload=391707]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Jan 31 08:42:31.141063957 (376h47m24.624521154s) [upload=391707]: Done uploading segments...
Jan 31 08:42:31.141101292 (376h47m24.624558493s) [upload=391707]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 415.073039ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
504.055577ms trace:
https://jaeger.dev.storj.io/trace/33d822523a7c81ef
FAIL (2026-01-31 09:23:00 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 504.081927ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [....X.......................................................] duration:
393.07722ms trace:
https://jaeger.dev.storj.io/trace/6c4cbd1edfce8e6
FAIL (2026-01-31 09:22:59 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 393.119331ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......X....................................................] duration:
536.508097ms trace:
https://jaeger.dev.storj.io/trace/62a1509ecc5b5058
FAIL (2026-01-31 09:22:22 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Jan 31 08:42:01.166771110 (376h46m54.650228297s) [upload=391707]: Starting upload
Jan 31 08:42:01.166986060 (376h46m54.650443258s) [upload=391707]: Done starting upload
Jan 31 08:42:01.167072896 (376h46m54.650530094s) [upload=391707]: Uploading segments...
Jan 31 08:42:01.167397789 (376h46m54.650854991s) [upload=391707]: Got next segment. Inline: false
Jan 31 08:42:01.167473452 (376h46m54.650930657s) [upload=391707]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jan 31 08:42:01.167480315 (376h46m54.650937524s) [upload=391707]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jan 31 08:42:01.195346764 (376h46m54.678803969s) [seg_pos={0 0}, upload=391707]: Begin upload segment...
Jan 31 08:42:01.195428575 (376h46m54.678885757s) [seg_pos={0 0}, upload=391707]: Done begin upload segment.
Jan 31 08:42:01.195463009 (376h46m54.678920214s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.195847779 (376h46m54.679304986s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.197051385 (376h46m54.680508591s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.197594770 (376h46m54.681051978s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.198049760 (376h46m54.681506963s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.198528799 (376h46m54.681986007s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.198996489 (376h46m54.682453701s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.199526016 (376h46m54.682983298s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.199772893 (376h46m54.683230102s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.200006077 (376h46m54.683463277s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.200298977 (376h46m54.68375618s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.200764860 (376h46m54.684222068s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.201330795 (376h46m54.68478802s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.201862657 (376h46m54.685319865s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.202435299 (376h46m54.685892502s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.203149033 (376h46m54.686606237s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.288234960 (376h46m54.771692157s) [upload=391707]: Next returned nil segment
Jan 31 08:42:01.288251604 (376h46m54.771708818s) [upload=391707]: Waiting for error group managing segments...
Jan 31 08:42:01.293400797 (376h46m54.776858002s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.294083673 (376h46m54.777540862s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.294285106 (376h46m54.777742305s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.295108197 (376h46m54.778565394s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.295172272 (376h46m54.778629467s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.295900158 (376h46m54.779357349s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.295955977 (376h46m54.779413173s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.296392614 (376h46m54.779849814s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.296441340 (376h46m54.779898537s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.296874674 (376h46m54.780331861s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.296909101 (376h46m54.780366298s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.297409404 (376h46m54.780866604s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:31.121919846 (376h47m24.605377048s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: read tcp 159.223.143.39:60086->45.91.116.7:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 31 08:42:31.122001831 (376h47m24.605459038s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: read tcp 159.223.143.39:60218->45.91.116.5:4297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 31 08:42:31.122048345 (376h47m24.605505567s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: read tcp 159.223.143.39:45254->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 31 08:42:31.122106214 (376h47m24.605563415s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12qpELr7Yr1aJtgau8q34qPBtADW6QAXPNySAzcRLB39hXoXueo, address:45.91.116.8:4070): protocol: read tcp 159.223.143.39:50716->45.91.116.8:4070: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 31 08:42:31.122131285 (376h47m24.605588469s) [seg_pos={0 0}, upload=391707]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Jan 31 08:42:31.141063957 (376h47m24.624521154s) [upload=391707]: Done uploading segments...
Jan 31 08:42:31.141101292 (376h47m24.624558493s) [upload=391707]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 536.538283ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................] duration:
440.489941ms trace:
https://jaeger.dev.storj.io/trace/33d822523a7c81ef
FAIL (2026-01-31 09:22:08 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 440.520805ms
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........X...................................................] duration:
385.148563ms trace:
https://jaeger.dev.storj.io/trace/62a1509ecc5b5058
FAIL (2026-01-31 09:22:04 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Jan 31 08:42:01.166771110 (376h46m54.650228297s) [upload=391707]: Starting upload
Jan 31 08:42:01.166986060 (376h46m54.650443258s) [upload=391707]: Done starting upload
Jan 31 08:42:01.167072896 (376h46m54.650530094s) [upload=391707]: Uploading segments...
Jan 31 08:42:01.167397789 (376h46m54.650854991s) [upload=391707]: Got next segment. Inline: false
Jan 31 08:42:01.167473452 (376h46m54.650930657s) [upload=391707]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jan 31 08:42:01.167480315 (376h46m54.650937524s) [upload=391707]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jan 31 08:42:01.195346764 (376h46m54.678803969s) [seg_pos={0 0}, upload=391707]: Begin upload segment...
Jan 31 08:42:01.195428575 (376h46m54.678885757s) [seg_pos={0 0}, upload=391707]: Done begin upload segment.
Jan 31 08:42:01.195463009 (376h46m54.678920214s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.195847779 (376h46m54.679304986s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.197051385 (376h46m54.680508591s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.197594770 (376h46m54.681051978s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.198049760 (376h46m54.681506963s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.198528799 (376h46m54.681986007s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.198996489 (376h46m54.682453701s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.199526016 (376h46m54.682983298s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.199772893 (376h46m54.683230102s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.200006077 (376h46m54.683463277s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.200298977 (376h46m54.68375618s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.200764860 (376h46m54.684222068s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.201330795 (376h46m54.68478802s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.201862657 (376h46m54.685319865s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.202435299 (376h46m54.685892502s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.203149033 (376h46m54.686606237s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.288234960 (376h46m54.771692157s) [upload=391707]: Next returned nil segment
Jan 31 08:42:01.288251604 (376h46m54.771708818s) [upload=391707]: Waiting for error group managing segments...
Jan 31 08:42:01.293400797 (376h46m54.776858002s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.294083673 (376h46m54.777540862s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.294285106 (376h46m54.777742305s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.295108197 (376h46m54.778565394s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.295172272 (376h46m54.778629467s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.295900158 (376h46m54.779357349s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.295955977 (376h46m54.779413173s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.296392614 (376h46m54.779849814s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.296441340 (376h46m54.779898537s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.296874674 (376h46m54.780331861s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.296909101 (376h46m54.780366298s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.297409404 (376h46m54.780866604s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:31.121919846 (376h47m24.605377048s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: read tcp 159.223.143.39:60086->45.91.116.7:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 31 08:42:31.122001831 (376h47m24.605459038s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: read tcp 159.223.143.39:60218->45.91.116.5:4297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 31 08:42:31.122048345 (376h47m24.605505567s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: read tcp 159.223.143.39:45254->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 31 08:42:31.122106214 (376h47m24.605563415s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12qpELr7Yr1aJtgau8q34qPBtADW6QAXPNySAzcRLB39hXoXueo, address:45.91.116.8:4070): protocol: read tcp 159.223.143.39:50716->45.91.116.8:4070: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 31 08:42:31.122131285 (376h47m24.605588469s) [seg_pos={0 0}, upload=391707]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Jan 31 08:42:31.141063957 (376h47m24.624521154s) [upload=391707]: Done uploading segments...
Jan 31 08:42:31.141101292 (376h47m24.624558493s) [upload=391707]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 385.188881ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.....X......................................................] duration:
443.624979ms trace:
https://jaeger.dev.storj.io/trace/6c4cbd1edfce8e6
FAIL (2026-01-31 09:21:50 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 443.673269ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......X.....................................................] duration:
442.512142ms trace:
https://jaeger.dev.storj.io/trace/33d822523a7c81ef
FAIL (2026-01-31 09:21:47 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 442.786285ms
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........X..................................................] duration:
407.165628ms trace:
https://jaeger.dev.storj.io/trace/62a1509ecc5b5058
FAIL (2026-01-31 09:21:32 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Jan 31 08:42:01.166771110 (376h46m54.650228297s) [upload=391707]: Starting upload
Jan 31 08:42:01.166986060 (376h46m54.650443258s) [upload=391707]: Done starting upload
Jan 31 08:42:01.167072896 (376h46m54.650530094s) [upload=391707]: Uploading segments...
Jan 31 08:42:01.167397789 (376h46m54.650854991s) [upload=391707]: Got next segment. Inline: false
Jan 31 08:42:01.167473452 (376h46m54.650930657s) [upload=391707]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jan 31 08:42:01.167480315 (376h46m54.650937524s) [upload=391707]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jan 31 08:42:01.195346764 (376h46m54.678803969s) [seg_pos={0 0}, upload=391707]: Begin upload segment...
Jan 31 08:42:01.195428575 (376h46m54.678885757s) [seg_pos={0 0}, upload=391707]: Done begin upload segment.
Jan 31 08:42:01.195463009 (376h46m54.678920214s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.195847779 (376h46m54.679304986s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.197051385 (376h46m54.680508591s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.197594770 (376h46m54.681051978s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.198049760 (376h46m54.681506963s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.198528799 (376h46m54.681986007s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.198996489 (376h46m54.682453701s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.199526016 (376h46m54.682983298s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.199772893 (376h46m54.683230102s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.200006077 (376h46m54.683463277s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.200298977 (376h46m54.68375618s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.200764860 (376h46m54.684222068s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.201330795 (376h46m54.68478802s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.201862657 (376h46m54.685319865s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.202435299 (376h46m54.685892502s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.203149033 (376h46m54.686606237s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.288234960 (376h46m54.771692157s) [upload=391707]: Next returned nil segment
Jan 31 08:42:01.288251604 (376h46m54.771708818s) [upload=391707]: Waiting for error group managing segments...
Jan 31 08:42:01.293400797 (376h46m54.776858002s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.294083673 (376h46m54.777540862s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.294285106 (376h46m54.777742305s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.295108197 (376h46m54.778565394s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.295172272 (376h46m54.778629467s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.295900158 (376h46m54.779357349s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.295955977 (376h46m54.779413173s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.296392614 (376h46m54.779849814s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.296441340 (376h46m54.779898537s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.296874674 (376h46m54.780331861s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.296909101 (376h46m54.780366298s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.297409404 (376h46m54.780866604s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:31.121919846 (376h47m24.605377048s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: read tcp 159.223.143.39:60086->45.91.116.7:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 31 08:42:31.122001831 (376h47m24.605459038s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: read tcp 159.223.143.39:60218->45.91.116.5:4297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 31 08:42:31.122048345 (376h47m24.605505567s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: read tcp 159.223.143.39:45254->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 31 08:42:31.122106214 (376h47m24.605563415s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12qpELr7Yr1aJtgau8q34qPBtADW6QAXPNySAzcRLB39hXoXueo, address:45.91.116.8:4070): protocol: read tcp 159.223.143.39:50716->45.91.116.8:4070: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 31 08:42:31.122131285 (376h47m24.605588469s) [seg_pos={0 0}, upload=391707]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Jan 31 08:42:31.141063957 (376h47m24.624521154s) [upload=391707]: Done uploading segments...
Jan 31 08:42:31.141101292 (376h47m24.624558493s) [upload=391707]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 407.209354ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [......X.....................................................] duration:
778.891192ms trace:
https://jaeger.dev.storj.io/trace/6c4cbd1edfce8e6
FAIL (2026-01-31 09:21:07 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 778.914546ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......X....................................................] duration:
479.823994ms trace:
https://jaeger.dev.storj.io/trace/33d822523a7c81ef
FAIL (2026-01-31 09:20:51 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 479.850187ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.......X....................................................] duration:
459.1404ms trace:
https://jaeger.dev.storj.io/trace/6c4cbd1edfce8e6
FAIL (2026-01-31 09:20:50 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 459.174955ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........X.................................................] duration:
423.328508ms trace:
https://jaeger.dev.storj.io/trace/62a1509ecc5b5058
FAIL (2026-01-31 09:20:19 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Jan 31 08:42:01.166771110 (376h46m54.650228297s) [upload=391707]: Starting upload
Jan 31 08:42:01.166986060 (376h46m54.650443258s) [upload=391707]: Done starting upload
Jan 31 08:42:01.167072896 (376h46m54.650530094s) [upload=391707]: Uploading segments...
Jan 31 08:42:01.167397789 (376h46m54.650854991s) [upload=391707]: Got next segment. Inline: false
Jan 31 08:42:01.167473452 (376h46m54.650930657s) [upload=391707]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jan 31 08:42:01.167480315 (376h46m54.650937524s) [upload=391707]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jan 31 08:42:01.195346764 (376h46m54.678803969s) [seg_pos={0 0}, upload=391707]: Begin upload segment...
Jan 31 08:42:01.195428575 (376h46m54.678885757s) [seg_pos={0 0}, upload=391707]: Done begin upload segment.
Jan 31 08:42:01.195463009 (376h46m54.678920214s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.195847779 (376h46m54.679304986s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.197051385 (376h46m54.680508591s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.197594770 (376h46m54.681051978s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.198049760 (376h46m54.681506963s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.198528799 (376h46m54.681986007s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.198996489 (376h46m54.682453701s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.199526016 (376h46m54.682983298s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.199772893 (376h46m54.683230102s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.200006077 (376h46m54.683463277s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.200298977 (376h46m54.68375618s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.200764860 (376h46m54.684222068s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.201330795 (376h46m54.68478802s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.201862657 (376h46m54.685319865s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.202435299 (376h46m54.685892502s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.203149033 (376h46m54.686606237s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.288234960 (376h46m54.771692157s) [upload=391707]: Next returned nil segment
Jan 31 08:42:01.288251604 (376h46m54.771708818s) [upload=391707]: Waiting for error group managing segments...
Jan 31 08:42:01.293400797 (376h46m54.776858002s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.294083673 (376h46m54.777540862s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.294285106 (376h46m54.777742305s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.295108197 (376h46m54.778565394s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.295172272 (376h46m54.778629467s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.295900158 (376h46m54.779357349s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.295955977 (376h46m54.779413173s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.296392614 (376h46m54.779849814s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.296441340 (376h46m54.779898537s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.296874674 (376h46m54.780331861s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.296909101 (376h46m54.780366298s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.297409404 (376h46m54.780866604s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:31.121919846 (376h47m24.605377048s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: read tcp 159.223.143.39:60086->45.91.116.7:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 31 08:42:31.122001831 (376h47m24.605459038s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: read tcp 159.223.143.39:60218->45.91.116.5:4297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 31 08:42:31.122048345 (376h47m24.605505567s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: read tcp 159.223.143.39:45254->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 31 08:42:31.122106214 (376h47m24.605563415s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12qpELr7Yr1aJtgau8q34qPBtADW6QAXPNySAzcRLB39hXoXueo, address:45.91.116.8:4070): protocol: read tcp 159.223.143.39:50716->45.91.116.8:4070: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 31 08:42:31.122131285 (376h47m24.605588469s) [seg_pos={0 0}, upload=391707]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Jan 31 08:42:31.141063957 (376h47m24.624521154s) [upload=391707]: Done uploading segments...
Jan 31 08:42:31.141101292 (376h47m24.624558493s) [upload=391707]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 423.357061ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........X...................................................] duration:
532.734973ms trace:
https://jaeger.dev.storj.io/trace/33d822523a7c81ef
FAIL (2026-01-31 09:20:19 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 532.786552ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [........X...................................................] duration:
424.323543ms trace:
https://jaeger.dev.storj.io/trace/6c4cbd1edfce8e6
FAIL (2026-01-31 09:19:59 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 424.40627ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........X..................................................] duration:
494.903987ms trace:
https://jaeger.dev.storj.io/trace/33d822523a7c81ef
FAIL (2026-01-31 09:19:27 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 495.003309ms
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........X................................................] duration:
555.065823ms trace:
https://jaeger.dev.storj.io/trace/62a1509ecc5b5058
FAIL (2026-01-31 09:19:09 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Jan 31 08:42:01.166771110 (376h46m54.650228297s) [upload=391707]: Starting upload
Jan 31 08:42:01.166986060 (376h46m54.650443258s) [upload=391707]: Done starting upload
Jan 31 08:42:01.167072896 (376h46m54.650530094s) [upload=391707]: Uploading segments...
Jan 31 08:42:01.167397789 (376h46m54.650854991s) [upload=391707]: Got next segment. Inline: false
Jan 31 08:42:01.167473452 (376h46m54.650930657s) [upload=391707]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jan 31 08:42:01.167480315 (376h46m54.650937524s) [upload=391707]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jan 31 08:42:01.195346764 (376h46m54.678803969s) [seg_pos={0 0}, upload=391707]: Begin upload segment...
Jan 31 08:42:01.195428575 (376h46m54.678885757s) [seg_pos={0 0}, upload=391707]: Done begin upload segment.
Jan 31 08:42:01.195463009 (376h46m54.678920214s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.195847779 (376h46m54.679304986s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.197051385 (376h46m54.680508591s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.197594770 (376h46m54.681051978s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.198049760 (376h46m54.681506963s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.198528799 (376h46m54.681986007s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.198996489 (376h46m54.682453701s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.199526016 (376h46m54.682983298s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.199772893 (376h46m54.683230102s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.200006077 (376h46m54.683463277s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.200298977 (376h46m54.68375618s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.200764860 (376h46m54.684222068s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.201330795 (376h46m54.68478802s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.201862657 (376h46m54.685319865s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.202435299 (376h46m54.685892502s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.203149033 (376h46m54.686606237s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.288234960 (376h46m54.771692157s) [upload=391707]: Next returned nil segment
Jan 31 08:42:01.288251604 (376h46m54.771708818s) [upload=391707]: Waiting for error group managing segments...
Jan 31 08:42:01.293400797 (376h46m54.776858002s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.294083673 (376h46m54.777540862s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.294285106 (376h46m54.777742305s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.295108197 (376h46m54.778565394s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.295172272 (376h46m54.778629467s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.295900158 (376h46m54.779357349s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.295955977 (376h46m54.779413173s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.296392614 (376h46m54.779849814s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.296441340 (376h46m54.779898537s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.296874674 (376h46m54.780331861s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.296909101 (376h46m54.780366298s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.297409404 (376h46m54.780866604s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:31.121919846 (376h47m24.605377048s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: read tcp 159.223.143.39:60086->45.91.116.7:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 31 08:42:31.122001831 (376h47m24.605459038s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: read tcp 159.223.143.39:60218->45.91.116.5:4297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 31 08:42:31.122048345 (376h47m24.605505567s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: read tcp 159.223.143.39:45254->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 31 08:42:31.122106214 (376h47m24.605563415s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12qpELr7Yr1aJtgau8q34qPBtADW6QAXPNySAzcRLB39hXoXueo, address:45.91.116.8:4070): protocol: read tcp 159.223.143.39:50716->45.91.116.8:4070: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 31 08:42:31.122131285 (376h47m24.605588469s) [seg_pos={0 0}, upload=391707]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Jan 31 08:42:31.141063957 (376h47m24.624521154s) [upload=391707]: Done uploading segments...
Jan 31 08:42:31.141101292 (376h47m24.624558493s) [upload=391707]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 555.109894ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.........X..................................................] duration:
469.764954ms trace:
https://jaeger.dev.storj.io/trace/6c4cbd1edfce8e6
FAIL (2026-01-31 09:18:45 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 469.795639ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [............X...............................................] duration:
382.885034ms trace:
https://jaeger.dev.storj.io/trace/62a1509ecc5b5058
FAIL (2026-01-31 09:18:40 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Jan 31 08:42:01.166771110 (376h46m54.650228297s) [upload=391707]: Starting upload
Jan 31 08:42:01.166986060 (376h46m54.650443258s) [upload=391707]: Done starting upload
Jan 31 08:42:01.167072896 (376h46m54.650530094s) [upload=391707]: Uploading segments...
Jan 31 08:42:01.167397789 (376h46m54.650854991s) [upload=391707]: Got next segment. Inline: false
Jan 31 08:42:01.167473452 (376h46m54.650930657s) [upload=391707]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jan 31 08:42:01.167480315 (376h46m54.650937524s) [upload=391707]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jan 31 08:42:01.195346764 (376h46m54.678803969s) [seg_pos={0 0}, upload=391707]: Begin upload segment...
Jan 31 08:42:01.195428575 (376h46m54.678885757s) [seg_pos={0 0}, upload=391707]: Done begin upload segment.
Jan 31 08:42:01.195463009 (376h46m54.678920214s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.195847779 (376h46m54.679304986s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.197051385 (376h46m54.680508591s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.197594770 (376h46m54.681051978s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.198049760 (376h46m54.681506963s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.198528799 (376h46m54.681986007s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.198996489 (376h46m54.682453701s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.199526016 (376h46m54.682983298s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.199772893 (376h46m54.683230102s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.200006077 (376h46m54.683463277s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.200298977 (376h46m54.68375618s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.200764860 (376h46m54.684222068s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.201330795 (376h46m54.68478802s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.201862657 (376h46m54.685319865s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.202435299 (376h46m54.685892502s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.203149033 (376h46m54.686606237s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.288234960 (376h46m54.771692157s) [upload=391707]: Next returned nil segment
Jan 31 08:42:01.288251604 (376h46m54.771708818s) [upload=391707]: Waiting for error group managing segments...
Jan 31 08:42:01.293400797 (376h46m54.776858002s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.294083673 (376h46m54.777540862s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.294285106 (376h46m54.777742305s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.295108197 (376h46m54.778565394s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.295172272 (376h46m54.778629467s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.295900158 (376h46m54.779357349s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.295955977 (376h46m54.779413173s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.296392614 (376h46m54.779849814s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.296441340 (376h46m54.779898537s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.296874674 (376h46m54.780331861s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.296909101 (376h46m54.780366298s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.297409404 (376h46m54.780866604s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:31.121919846 (376h47m24.605377048s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: read tcp 159.223.143.39:60086->45.91.116.7:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 31 08:42:31.122001831 (376h47m24.605459038s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: read tcp 159.223.143.39:60218->45.91.116.5:4297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 31 08:42:31.122048345 (376h47m24.605505567s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: read tcp 159.223.143.39:45254->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 31 08:42:31.122106214 (376h47m24.605563415s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12qpELr7Yr1aJtgau8q34qPBtADW6QAXPNySAzcRLB39hXoXueo, address:45.91.116.8:4070): protocol: read tcp 159.223.143.39:50716->45.91.116.8:4070: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 31 08:42:31.122131285 (376h47m24.605588469s) [seg_pos={0 0}, upload=391707]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Jan 31 08:42:31.141063957 (376h47m24.624521154s) [upload=391707]: Done uploading segments...
Jan 31 08:42:31.141101292 (376h47m24.624558493s) [upload=391707]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 382.912971ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........X.................................................] duration:
462.485512ms trace:
https://jaeger.dev.storj.io/trace/33d822523a7c81ef
FAIL (2026-01-31 09:18:23 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 462.521634ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........X................................................] duration:
485.189962ms trace:
https://jaeger.dev.storj.io/trace/33d822523a7c81ef
FAIL (2026-01-31 09:18:07 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 485.41421ms
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............X..............................................] duration:
443.011922ms trace:
https://jaeger.dev.storj.io/trace/62a1509ecc5b5058
FAIL (2026-01-31 09:18:04 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Jan 31 08:42:01.166771110 (376h46m54.650228297s) [upload=391707]: Starting upload
Jan 31 08:42:01.166986060 (376h46m54.650443258s) [upload=391707]: Done starting upload
Jan 31 08:42:01.167072896 (376h46m54.650530094s) [upload=391707]: Uploading segments...
Jan 31 08:42:01.167397789 (376h46m54.650854991s) [upload=391707]: Got next segment. Inline: false
Jan 31 08:42:01.167473452 (376h46m54.650930657s) [upload=391707]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jan 31 08:42:01.167480315 (376h46m54.650937524s) [upload=391707]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jan 31 08:42:01.195346764 (376h46m54.678803969s) [seg_pos={0 0}, upload=391707]: Begin upload segment...
Jan 31 08:42:01.195428575 (376h46m54.678885757s) [seg_pos={0 0}, upload=391707]: Done begin upload segment.
Jan 31 08:42:01.195463009 (376h46m54.678920214s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.195847779 (376h46m54.679304986s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.197051385 (376h46m54.680508591s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.197594770 (376h46m54.681051978s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.198049760 (376h46m54.681506963s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.198528799 (376h46m54.681986007s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.198996489 (376h46m54.682453701s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.199526016 (376h46m54.682983298s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.199772893 (376h46m54.683230102s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.200006077 (376h46m54.683463277s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.200298977 (376h46m54.68375618s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.200764860 (376h46m54.684222068s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.201330795 (376h46m54.68478802s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.201862657 (376h46m54.685319865s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.202435299 (376h46m54.685892502s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.203149033 (376h46m54.686606237s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Uploading piece...
Jan 31 08:42:01.288234960 (376h46m54.771692157s) [upload=391707]: Next returned nil segment
Jan 31 08:42:01.288251604 (376h46m54.771708818s) [upload=391707]: Waiting for error group managing segments...
Jan 31 08:42:01.293400797 (376h46m54.776858002s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.294083673 (376h46m54.777540862s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.294285106 (376h46m54.777742305s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.295108197 (376h46m54.778565394s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.295172272 (376h46m54.778629467s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.295900158 (376h46m54.779357349s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.295955977 (376h46m54.779413173s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.296392614 (376h46m54.779849814s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.296441340 (376h46m54.779898537s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.296874674 (376h46m54.780331861s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.296909101 (376h46m54.780366298s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:01.297409404 (376h46m54.780866604s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil>
Jan 31 08:42:31.121919846 (376h47m24.605377048s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: read tcp 159.223.143.39:60086->45.91.116.7:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 31 08:42:31.122001831 (376h47m24.605459038s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: read tcp 159.223.143.39:60218->45.91.116.5:4297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 31 08:42:31.122048345 (376h47m24.605505567s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: read tcp 159.223.143.39:45254->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 31 08:42:31.122106214 (376h47m24.605563415s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12qpELr7Yr1aJtgau8q34qPBtADW6QAXPNySAzcRLB39hXoXueo, address:45.91.116.8:4070): protocol: read tcp 159.223.143.39:50716->45.91.116.8:4070: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 31 08:42:31.122131285 (376h47m24.605588469s) [seg_pos={0 0}, upload=391707]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Jan 31 08:42:31.141063957 (376h47m24.624521154s) [upload=391707]: Done uploading segments...
Jan 31 08:42:31.141101292 (376h47m24.624558493s) [upload=391707]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 443.045904ms
core
uplink
us-select-1
|