Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
3 out of 60 is failed (threshold=6) [.................................................X.......XX.] duration:
563.902292ms trace:
https://jaeger.dev.storj.io/trace/698fd4ac72ce0f8b
FAIL (2026-03-10 21:22:04 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 564.60013ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
3 out of 60 is failed (threshold=6) [..................................................X.......XX] duration:
5.001814236s trace:
https://jaeger.dev.storj.io/trace/698fd4ac72ce0f8b
FAIL (2026-03-10 21:21:20 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.001835703s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [...................................................X.......X] duration:
5.000761344s trace:
https://jaeger.dev.storj.io/trace/650632d1884e8f35
FAIL (2026-03-10 21:20:50 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.000785134s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [....................................................X.......] duration:
1.784856966s trace:
https://jaeger.dev.storj.io/trace/6f320e45406e0ae2
FAIL (2026-03-10 21:19:52 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.784894051s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.....................................................X......] duration:
583.308005ms trace:
https://jaeger.dev.storj.io/trace/6f320e45406e0ae2
FAIL (2026-03-10 21:19:06 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 583.332512ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [......................................................X.....] duration:
1.139934079s trace:
https://jaeger.dev.storj.io/trace/6f320e45406e0ae2
FAIL (2026-03-10 21:18:03 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.140045348s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.......................................................X....] duration:
648.180564ms trace:
https://jaeger.dev.storj.io/trace/6f320e45406e0ae2
FAIL (2026-03-10 21:17:35 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 648.264338ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [........................................................X...] duration:
1.458546794s trace:
https://jaeger.dev.storj.io/trace/6f320e45406e0ae2
FAIL (2026-03-10 21:17:08 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.458577279s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.........................................................X..] duration:
1.470623257s trace:
https://jaeger.dev.storj.io/trace/6f320e45406e0ae2
FAIL (2026-03-10 21:15:55 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.470659549s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [..........................................................X.] duration:
2.570398595s trace:
https://jaeger.dev.storj.io/trace/6f320e45406e0ae2
FAIL (2026-03-10 21:15:36 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.570447978s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [...........................................................X] duration:
5.001864685s trace:
https://jaeger.dev.storj.io/trace/6f320e45406e0ae2
FAIL (2026-03-10 21:15:16 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.00191011s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
2.581668337s trace:
https://jaeger.dev.storj.io/trace/78c3ad63aac6b527
FAIL (2026-03-10 20:21:26 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
:03.719970769 (104h35m1.316741801s) [noise=true , address=185.167.204.205:4033 , piece_id=6JIW66HORXSEPSDWIUONOUZNA2UNXKBA4A6VZ7PBWW3IN2SCJSPQ, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.720006314 (104h35m1.316777347s) [noise=true , address=185.167.204.205:4024 , piece_id=3CKPLBO33LOMNYBF47K2KGQFPO53ZXLMKRKWPVSR4FKQKXKMZMCA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.721245847 (104h35m1.318016883s) [noise=true , address=185.167.204.206:4046 , piece_id=5WVC4R6VY25QUMGSOAORUK3WPGIADBXLKN7ZSLJ4GQEVKMSHZPKA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.722131967 (104h35m1.318902999s) [noise=true , address=89.149.201.147:4004 , piece_id=TMKNRIR6RWRW3TUSA2MQL4LCNTJ3LKEFIT6BAVRX2V54QHNF26FA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.722154312 (104h35m1.318925349s) [noise=true , address=185.167.204.205:4028 , piece_id=IQ4CMHEIUNHIUBQV2PVFQ6BURDMWU2S55VGFP5Y7HPFSDVBY6PYA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.723422869 (104h35m1.320193903s) [noise=true , address=81.171.14.70:4047 , piece_id=DLCUNYJFEKIU3MWFXK24QGJ4GW2H5XE4Y2ARNL2W55EZUMSC4Z7Q, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.724205203 (104h35m1.320976237s) [noise=true , address=185.167.204.205:4022 , piece_id=F5UFUTAFPRBWMSU3QKV7ZSYQR6UUC7SBUDOC7AAYGDXTC5VYUBHA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.727803141 (104h35m1.324574176s) [noise=true , address=185.167.204.206:4067 , piece_id=RH2DWZ3FL7JX6EJ2WTJBPBSWDYVSHLYSLZCMMGRQXUDR6QXVZZ5A, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:04.167055423 (104h35m1.763826466s) [upload=117052]: Next returned nil segment
Mar 10 19:35:04.167072399 (104h35m1.763843424s) [upload=117052]: Waiting for error group managing segments...
Mar 10 19:35:04.350286979 (104h35m1.947058004s) [noise=true , address=81.171.14.70:4035 , piece_id=RNJYIOH4ETYQBSZFYTV3FX73J3PNFR2KBYNZEDXCVBOYMUAN5FCA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.351883208 (104h35m1.948654233s) [noise=true , address=81.171.14.70:4043 , piece_id=LSOHRZMEV5TEQUKZIJPYHVP43CIZKMXBTRVKIZIQNSSFNMAA5FLQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.356149085 (104h35m1.952920098s) [noise=true , address=185.167.204.205:4036 , piece_id=BBGCJ7U7PQDNB5NZTJWHUWXR6TFYF6D7LOR2UUBNV647YQX5JD2Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.369094269 (104h35m1.965865391s) [noise=true , address=89.149.201.147:4011 , piece_id=YHPLXK3ADD6VMYDWQ55PLROBCY2IOLOR2ZWEZ22FRRA4TXKSSWIQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.382982597 (104h35m1.979753611s) [noise=true , address=81.171.14.70:4034 , piece_id=BPXMDI54RTQVH5QKY6JCN5H3AKCPH62V76VBES4GQ6SLIZDIMMOA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.384672279 (104h35m1.981443307s) [noise=true , address=81.171.14.70:4047 , piece_id=DLCUNYJFEKIU3MWFXK24QGJ4GW2H5XE4Y2ARNL2W55EZUMSC4Z7Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.386337288 (104h35m1.98310832s) [noise=true , address=81.171.14.70:4025 , piece_id=R62A3AXM4OKUZFZJ2LB55UPB6VRGHBQLMXXVBWDTHUDWPGPIWEVQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.387583761 (104h35m1.984354793s) [noise=true , address=89.149.201.147:4009 , piece_id=XFSD3W6JWUXGXTHYE57SS5E5322JTRG7RKBI7MBF25GU7SEAKZJQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.388110220 (104h35m1.984881248s) [noise=true , address=81.171.14.70:4026 , piece_id=4CBQ4RZ44USXNYAGA3CIGNC3LBQXR675JVJX6CVTV3X7KVYPRHQA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.389451350 (104h35m1.986222362s) [noise=true , address=185.167.204.205:4012 , piece_id=7UFJIUKCHPKYZF42HTP6Q5D2FMHRSCRUQ7MH2MYJSEOZHZAJRZXA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.404093624 (104h35m2.00086464s) [noise=true , address=81.171.14.70:4048 , piece_id=4F2F4X4ZJ4FXUYQJQKB7YB3JZY53QNVRUSM3N4HQKGEYHMFYQB2A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.413111656 (104h35m2.009882687s) [noise=true , address=185.167.204.206:4071 , piece_id=ZQJLN55G37453GZS54ZDTPM7CEU5GH6LJR3BIKSJDC3C4QOKW5YQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.416183369 (104h35m2.012954404s) [noise=true , address=89.149.201.147:4014 , piece_id=WZOPRPDLFVOO6O2JYZZB75CLIAA4SVM7OEZXBX6WOJV3PUL2YSAQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.449375785 (104h35m2.046146815s) [noise=true , address=185.167.204.206:4039 , piece_id=DKKIZZMA3RAKMLUJTU6OANGBA3F6XWRD5ELVNXAGBIOZMFBVQWZQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.457716093 (104h35m2.054487108s) [noise=true , address=185.167.204.205:4024 , piece_id=3CKPLBO33LOMNYBF47K2KGQFPO53ZXLMKRKWPVSR4FKQKXKMZMCA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.491116161 (104h35m2.087887192s) [noise=true , address=185.167.204.205:4005 , piece_id=E7RO77MCAMFEAIE7ETPALCBV5S4X3DG5ZUD27ZE55RXYICIIXNEQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.507099744 (104h35m2.103870782s) [noise=true , address=185.167.204.205:4029 , piece_id=R33ONZELVLYL76YTJKOZASDJKFBACNNMC3PDUIITQRDGBWWZUNKA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.537139239 (104h35m2.133910269s) [noise=true , address=185.167.204.206:4067 , piece_id=RH2DWZ3FL7JX6EJ2WTJBPBSWDYVSHLYSLZCMMGRQXUDR6QXVZZ5A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.787100189 (104h35m2.383871224s) [noise=true , address=81.171.14.70:4033 , piece_id=ZFV477QXRX42OJIG4D5JLOYU3WERTD65PMHI5MFQLRTQJG3ENUTA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.962846444 (104h35m2.559617469s) [noise=true , address=89.149.201.147:4004 , piece_id=TMKNRIR6RWRW3TUSA2MQL4LCNTJ3LKEFIT6BAVRX2V54QHNF26FA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:08.299132133 (104h35m5.895903147s) [noise=true , address=185.167.204.205:4028 , piece_id=IQ4CMHEIUNHIUBQV2PVFQ6BURDMWU2S55VGFP5Y7HPFSDVBY6PYA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:08.481127955 (104h35m6.077898968s) [noise=true , address=185.167.204.206:4051 , piece_id=GO4OTIL5WFOS7V77FAT2NRFG2HRE2HXKMU2C3XEGDBGPQL5U5HUA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:10.314952727 (104h35m7.91172374s) [noise=true , address=185.167.204.206:4046 , piece_id=5WVC4R6VY25QUMGSOAORUK3WPGIADBXLKN7ZSLJ4GQEVKMSHZPKA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:36:03.322697363 (104h36m0.919468398s) [noise=true , address=185.167.204.206:4062 , piece_id=7NP2ZIMW6ZDXW3YHZSBO5PRTAZPLYLDWXDA4QPN5XH2C6KAGYLEA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:12BL6XLqsoUGm2FxrTgYTxLZakt2u2CAhZJG82oJ31eaE4vTtMr, address:185.167.204.206:4062): protocol: write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection; write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection
Mar 10 19:36:03.322771647 (104h36m0.919542676s) [noise=true , address=89.149.201.147:4007 , piece_id=AAJY5QBN55ENHTODH25WBVCFDT2SAQ2F27TKHFCIURCZSBMGOY6A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1Q7d4vj1KoWhDr6xd9gEfFrZ1sV3Yro2AFke2f3Ke1rXsGnvew, address:89.149.201.147:4007): protocol: read tcp 159.223.143.39:54460->89.149.201.147:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.322958851 (104h36m0.919729884s) [noise=true , address=185.167.204.205:4033 , piece_id=6JIW66HORXSEPSDWIUONOUZNA2UNXKBA4A6VZ7PBWW3IN2SCJSPQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:12Q73mXHLEfuX4VUbJ8PTjhMoXCrYSKjJbFxqjDdqF3U99N8FXe, address:185.167.204.205:4033): protocol: write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection; write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection
Mar 10 19:36:03.323134221 (104h36m0.919905244s) [noise=true , address=89.149.201.147:4018 , piece_id=V3KLJ75HJ3NRMBP2FIM33GC45QW4U5XMN2R4IAHSX32COGBJF47Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1Ku7o12bSmJ1YR3dGetGFcWqs1wJEh53Zkg3H7ELJrt4w1hkua, address:89.149.201.147:4018): protocol: read tcp 159.223.143.39:47122->89.149.201.147:4018: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.323461923 (104h36m0.920232956s) [noise=true , address=185.167.204.205:4022 , piece_id=F5UFUTAFPRBWMSU3QKV7ZSYQR6UUC7SBUDOC7AAYGDXTC5VYUBHA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1fT9qH1igPUNfrzpwa2VkBX7heh475sNqoN4Nsf9FYSwi1VmHj, address:185.167.204.205:4022): protocol: read tcp 159.223.143.39:51226->185.167.204.205:4022: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.323503666 (104h36m0.920274689s) [seg_pos={0 0}, upload=117052]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Mar 10 19:36:03.424787357 (104h36m1.021558384s) [upload=117052]: Done uploading segments...
Mar 10 19:36:03.424860036 (104h36m1.021631063s) [upload=117052]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 2.581708851s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
2.577756205s trace:
https://jaeger.dev.storj.io/trace/78c3ad63aac6b527
FAIL (2026-03-10 20:20:45 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
:03.719970769 (104h35m1.316741801s) [noise=true , address=185.167.204.205:4033 , piece_id=6JIW66HORXSEPSDWIUONOUZNA2UNXKBA4A6VZ7PBWW3IN2SCJSPQ, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.720006314 (104h35m1.316777347s) [noise=true , address=185.167.204.205:4024 , piece_id=3CKPLBO33LOMNYBF47K2KGQFPO53ZXLMKRKWPVSR4FKQKXKMZMCA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.721245847 (104h35m1.318016883s) [noise=true , address=185.167.204.206:4046 , piece_id=5WVC4R6VY25QUMGSOAORUK3WPGIADBXLKN7ZSLJ4GQEVKMSHZPKA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.722131967 (104h35m1.318902999s) [noise=true , address=89.149.201.147:4004 , piece_id=TMKNRIR6RWRW3TUSA2MQL4LCNTJ3LKEFIT6BAVRX2V54QHNF26FA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.722154312 (104h35m1.318925349s) [noise=true , address=185.167.204.205:4028 , piece_id=IQ4CMHEIUNHIUBQV2PVFQ6BURDMWU2S55VGFP5Y7HPFSDVBY6PYA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.723422869 (104h35m1.320193903s) [noise=true , address=81.171.14.70:4047 , piece_id=DLCUNYJFEKIU3MWFXK24QGJ4GW2H5XE4Y2ARNL2W55EZUMSC4Z7Q, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.724205203 (104h35m1.320976237s) [noise=true , address=185.167.204.205:4022 , piece_id=F5UFUTAFPRBWMSU3QKV7ZSYQR6UUC7SBUDOC7AAYGDXTC5VYUBHA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.727803141 (104h35m1.324574176s) [noise=true , address=185.167.204.206:4067 , piece_id=RH2DWZ3FL7JX6EJ2WTJBPBSWDYVSHLYSLZCMMGRQXUDR6QXVZZ5A, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:04.167055423 (104h35m1.763826466s) [upload=117052]: Next returned nil segment
Mar 10 19:35:04.167072399 (104h35m1.763843424s) [upload=117052]: Waiting for error group managing segments...
Mar 10 19:35:04.350286979 (104h35m1.947058004s) [noise=true , address=81.171.14.70:4035 , piece_id=RNJYIOH4ETYQBSZFYTV3FX73J3PNFR2KBYNZEDXCVBOYMUAN5FCA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.351883208 (104h35m1.948654233s) [noise=true , address=81.171.14.70:4043 , piece_id=LSOHRZMEV5TEQUKZIJPYHVP43CIZKMXBTRVKIZIQNSSFNMAA5FLQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.356149085 (104h35m1.952920098s) [noise=true , address=185.167.204.205:4036 , piece_id=BBGCJ7U7PQDNB5NZTJWHUWXR6TFYF6D7LOR2UUBNV647YQX5JD2Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.369094269 (104h35m1.965865391s) [noise=true , address=89.149.201.147:4011 , piece_id=YHPLXK3ADD6VMYDWQ55PLROBCY2IOLOR2ZWEZ22FRRA4TXKSSWIQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.382982597 (104h35m1.979753611s) [noise=true , address=81.171.14.70:4034 , piece_id=BPXMDI54RTQVH5QKY6JCN5H3AKCPH62V76VBES4GQ6SLIZDIMMOA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.384672279 (104h35m1.981443307s) [noise=true , address=81.171.14.70:4047 , piece_id=DLCUNYJFEKIU3MWFXK24QGJ4GW2H5XE4Y2ARNL2W55EZUMSC4Z7Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.386337288 (104h35m1.98310832s) [noise=true , address=81.171.14.70:4025 , piece_id=R62A3AXM4OKUZFZJ2LB55UPB6VRGHBQLMXXVBWDTHUDWPGPIWEVQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.387583761 (104h35m1.984354793s) [noise=true , address=89.149.201.147:4009 , piece_id=XFSD3W6JWUXGXTHYE57SS5E5322JTRG7RKBI7MBF25GU7SEAKZJQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.388110220 (104h35m1.984881248s) [noise=true , address=81.171.14.70:4026 , piece_id=4CBQ4RZ44USXNYAGA3CIGNC3LBQXR675JVJX6CVTV3X7KVYPRHQA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.389451350 (104h35m1.986222362s) [noise=true , address=185.167.204.205:4012 , piece_id=7UFJIUKCHPKYZF42HTP6Q5D2FMHRSCRUQ7MH2MYJSEOZHZAJRZXA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.404093624 (104h35m2.00086464s) [noise=true , address=81.171.14.70:4048 , piece_id=4F2F4X4ZJ4FXUYQJQKB7YB3JZY53QNVRUSM3N4HQKGEYHMFYQB2A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.413111656 (104h35m2.009882687s) [noise=true , address=185.167.204.206:4071 , piece_id=ZQJLN55G37453GZS54ZDTPM7CEU5GH6LJR3BIKSJDC3C4QOKW5YQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.416183369 (104h35m2.012954404s) [noise=true , address=89.149.201.147:4014 , piece_id=WZOPRPDLFVOO6O2JYZZB75CLIAA4SVM7OEZXBX6WOJV3PUL2YSAQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.449375785 (104h35m2.046146815s) [noise=true , address=185.167.204.206:4039 , piece_id=DKKIZZMA3RAKMLUJTU6OANGBA3F6XWRD5ELVNXAGBIOZMFBVQWZQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.457716093 (104h35m2.054487108s) [noise=true , address=185.167.204.205:4024 , piece_id=3CKPLBO33LOMNYBF47K2KGQFPO53ZXLMKRKWPVSR4FKQKXKMZMCA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.491116161 (104h35m2.087887192s) [noise=true , address=185.167.204.205:4005 , piece_id=E7RO77MCAMFEAIE7ETPALCBV5S4X3DG5ZUD27ZE55RXYICIIXNEQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.507099744 (104h35m2.103870782s) [noise=true , address=185.167.204.205:4029 , piece_id=R33ONZELVLYL76YTJKOZASDJKFBACNNMC3PDUIITQRDGBWWZUNKA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.537139239 (104h35m2.133910269s) [noise=true , address=185.167.204.206:4067 , piece_id=RH2DWZ3FL7JX6EJ2WTJBPBSWDYVSHLYSLZCMMGRQXUDR6QXVZZ5A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.787100189 (104h35m2.383871224s) [noise=true , address=81.171.14.70:4033 , piece_id=ZFV477QXRX42OJIG4D5JLOYU3WERTD65PMHI5MFQLRTQJG3ENUTA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.962846444 (104h35m2.559617469s) [noise=true , address=89.149.201.147:4004 , piece_id=TMKNRIR6RWRW3TUSA2MQL4LCNTJ3LKEFIT6BAVRX2V54QHNF26FA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:08.299132133 (104h35m5.895903147s) [noise=true , address=185.167.204.205:4028 , piece_id=IQ4CMHEIUNHIUBQV2PVFQ6BURDMWU2S55VGFP5Y7HPFSDVBY6PYA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:08.481127955 (104h35m6.077898968s) [noise=true , address=185.167.204.206:4051 , piece_id=GO4OTIL5WFOS7V77FAT2NRFG2HRE2HXKMU2C3XEGDBGPQL5U5HUA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:10.314952727 (104h35m7.91172374s) [noise=true , address=185.167.204.206:4046 , piece_id=5WVC4R6VY25QUMGSOAORUK3WPGIADBXLKN7ZSLJ4GQEVKMSHZPKA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:36:03.322697363 (104h36m0.919468398s) [noise=true , address=185.167.204.206:4062 , piece_id=7NP2ZIMW6ZDXW3YHZSBO5PRTAZPLYLDWXDA4QPN5XH2C6KAGYLEA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:12BL6XLqsoUGm2FxrTgYTxLZakt2u2CAhZJG82oJ31eaE4vTtMr, address:185.167.204.206:4062): protocol: write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection; write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection
Mar 10 19:36:03.322771647 (104h36m0.919542676s) [noise=true , address=89.149.201.147:4007 , piece_id=AAJY5QBN55ENHTODH25WBVCFDT2SAQ2F27TKHFCIURCZSBMGOY6A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1Q7d4vj1KoWhDr6xd9gEfFrZ1sV3Yro2AFke2f3Ke1rXsGnvew, address:89.149.201.147:4007): protocol: read tcp 159.223.143.39:54460->89.149.201.147:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.322958851 (104h36m0.919729884s) [noise=true , address=185.167.204.205:4033 , piece_id=6JIW66HORXSEPSDWIUONOUZNA2UNXKBA4A6VZ7PBWW3IN2SCJSPQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:12Q73mXHLEfuX4VUbJ8PTjhMoXCrYSKjJbFxqjDdqF3U99N8FXe, address:185.167.204.205:4033): protocol: write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection; write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection
Mar 10 19:36:03.323134221 (104h36m0.919905244s) [noise=true , address=89.149.201.147:4018 , piece_id=V3KLJ75HJ3NRMBP2FIM33GC45QW4U5XMN2R4IAHSX32COGBJF47Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1Ku7o12bSmJ1YR3dGetGFcWqs1wJEh53Zkg3H7ELJrt4w1hkua, address:89.149.201.147:4018): protocol: read tcp 159.223.143.39:47122->89.149.201.147:4018: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.323461923 (104h36m0.920232956s) [noise=true , address=185.167.204.205:4022 , piece_id=F5UFUTAFPRBWMSU3QKV7ZSYQR6UUC7SBUDOC7AAYGDXTC5VYUBHA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1fT9qH1igPUNfrzpwa2VkBX7heh475sNqoN4Nsf9FYSwi1VmHj, address:185.167.204.205:4022): protocol: read tcp 159.223.143.39:51226->185.167.204.205:4022: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.323503666 (104h36m0.920274689s) [seg_pos={0 0}, upload=117052]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Mar 10 19:36:03.424787357 (104h36m1.021558384s) [upload=117052]: Done uploading segments...
Mar 10 19:36:03.424860036 (104h36m1.021631063s) [upload=117052]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 2.577776871s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
2.71735708s trace:
https://jaeger.dev.storj.io/trace/78c3ad63aac6b527
FAIL (2026-03-10 20:20:05 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
:03.719970769 (104h35m1.316741801s) [noise=true , address=185.167.204.205:4033 , piece_id=6JIW66HORXSEPSDWIUONOUZNA2UNXKBA4A6VZ7PBWW3IN2SCJSPQ, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.720006314 (104h35m1.316777347s) [noise=true , address=185.167.204.205:4024 , piece_id=3CKPLBO33LOMNYBF47K2KGQFPO53ZXLMKRKWPVSR4FKQKXKMZMCA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.721245847 (104h35m1.318016883s) [noise=true , address=185.167.204.206:4046 , piece_id=5WVC4R6VY25QUMGSOAORUK3WPGIADBXLKN7ZSLJ4GQEVKMSHZPKA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.722131967 (104h35m1.318902999s) [noise=true , address=89.149.201.147:4004 , piece_id=TMKNRIR6RWRW3TUSA2MQL4LCNTJ3LKEFIT6BAVRX2V54QHNF26FA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.722154312 (104h35m1.318925349s) [noise=true , address=185.167.204.205:4028 , piece_id=IQ4CMHEIUNHIUBQV2PVFQ6BURDMWU2S55VGFP5Y7HPFSDVBY6PYA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.723422869 (104h35m1.320193903s) [noise=true , address=81.171.14.70:4047 , piece_id=DLCUNYJFEKIU3MWFXK24QGJ4GW2H5XE4Y2ARNL2W55EZUMSC4Z7Q, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.724205203 (104h35m1.320976237s) [noise=true , address=185.167.204.205:4022 , piece_id=F5UFUTAFPRBWMSU3QKV7ZSYQR6UUC7SBUDOC7AAYGDXTC5VYUBHA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.727803141 (104h35m1.324574176s) [noise=true , address=185.167.204.206:4067 , piece_id=RH2DWZ3FL7JX6EJ2WTJBPBSWDYVSHLYSLZCMMGRQXUDR6QXVZZ5A, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:04.167055423 (104h35m1.763826466s) [upload=117052]: Next returned nil segment
Mar 10 19:35:04.167072399 (104h35m1.763843424s) [upload=117052]: Waiting for error group managing segments...
Mar 10 19:35:04.350286979 (104h35m1.947058004s) [noise=true , address=81.171.14.70:4035 , piece_id=RNJYIOH4ETYQBSZFYTV3FX73J3PNFR2KBYNZEDXCVBOYMUAN5FCA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.351883208 (104h35m1.948654233s) [noise=true , address=81.171.14.70:4043 , piece_id=LSOHRZMEV5TEQUKZIJPYHVP43CIZKMXBTRVKIZIQNSSFNMAA5FLQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.356149085 (104h35m1.952920098s) [noise=true , address=185.167.204.205:4036 , piece_id=BBGCJ7U7PQDNB5NZTJWHUWXR6TFYF6D7LOR2UUBNV647YQX5JD2Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.369094269 (104h35m1.965865391s) [noise=true , address=89.149.201.147:4011 , piece_id=YHPLXK3ADD6VMYDWQ55PLROBCY2IOLOR2ZWEZ22FRRA4TXKSSWIQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.382982597 (104h35m1.979753611s) [noise=true , address=81.171.14.70:4034 , piece_id=BPXMDI54RTQVH5QKY6JCN5H3AKCPH62V76VBES4GQ6SLIZDIMMOA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.384672279 (104h35m1.981443307s) [noise=true , address=81.171.14.70:4047 , piece_id=DLCUNYJFEKIU3MWFXK24QGJ4GW2H5XE4Y2ARNL2W55EZUMSC4Z7Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.386337288 (104h35m1.98310832s) [noise=true , address=81.171.14.70:4025 , piece_id=R62A3AXM4OKUZFZJ2LB55UPB6VRGHBQLMXXVBWDTHUDWPGPIWEVQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.387583761 (104h35m1.984354793s) [noise=true , address=89.149.201.147:4009 , piece_id=XFSD3W6JWUXGXTHYE57SS5E5322JTRG7RKBI7MBF25GU7SEAKZJQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.388110220 (104h35m1.984881248s) [noise=true , address=81.171.14.70:4026 , piece_id=4CBQ4RZ44USXNYAGA3CIGNC3LBQXR675JVJX6CVTV3X7KVYPRHQA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.389451350 (104h35m1.986222362s) [noise=true , address=185.167.204.205:4012 , piece_id=7UFJIUKCHPKYZF42HTP6Q5D2FMHRSCRUQ7MH2MYJSEOZHZAJRZXA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.404093624 (104h35m2.00086464s) [noise=true , address=81.171.14.70:4048 , piece_id=4F2F4X4ZJ4FXUYQJQKB7YB3JZY53QNVRUSM3N4HQKGEYHMFYQB2A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.413111656 (104h35m2.009882687s) [noise=true , address=185.167.204.206:4071 , piece_id=ZQJLN55G37453GZS54ZDTPM7CEU5GH6LJR3BIKSJDC3C4QOKW5YQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.416183369 (104h35m2.012954404s) [noise=true , address=89.149.201.147:4014 , piece_id=WZOPRPDLFVOO6O2JYZZB75CLIAA4SVM7OEZXBX6WOJV3PUL2YSAQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.449375785 (104h35m2.046146815s) [noise=true , address=185.167.204.206:4039 , piece_id=DKKIZZMA3RAKMLUJTU6OANGBA3F6XWRD5ELVNXAGBIOZMFBVQWZQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.457716093 (104h35m2.054487108s) [noise=true , address=185.167.204.205:4024 , piece_id=3CKPLBO33LOMNYBF47K2KGQFPO53ZXLMKRKWPVSR4FKQKXKMZMCA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.491116161 (104h35m2.087887192s) [noise=true , address=185.167.204.205:4005 , piece_id=E7RO77MCAMFEAIE7ETPALCBV5S4X3DG5ZUD27ZE55RXYICIIXNEQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.507099744 (104h35m2.103870782s) [noise=true , address=185.167.204.205:4029 , piece_id=R33ONZELVLYL76YTJKOZASDJKFBACNNMC3PDUIITQRDGBWWZUNKA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.537139239 (104h35m2.133910269s) [noise=true , address=185.167.204.206:4067 , piece_id=RH2DWZ3FL7JX6EJ2WTJBPBSWDYVSHLYSLZCMMGRQXUDR6QXVZZ5A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.787100189 (104h35m2.383871224s) [noise=true , address=81.171.14.70:4033 , piece_id=ZFV477QXRX42OJIG4D5JLOYU3WERTD65PMHI5MFQLRTQJG3ENUTA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.962846444 (104h35m2.559617469s) [noise=true , address=89.149.201.147:4004 , piece_id=TMKNRIR6RWRW3TUSA2MQL4LCNTJ3LKEFIT6BAVRX2V54QHNF26FA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:08.299132133 (104h35m5.895903147s) [noise=true , address=185.167.204.205:4028 , piece_id=IQ4CMHEIUNHIUBQV2PVFQ6BURDMWU2S55VGFP5Y7HPFSDVBY6PYA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:08.481127955 (104h35m6.077898968s) [noise=true , address=185.167.204.206:4051 , piece_id=GO4OTIL5WFOS7V77FAT2NRFG2HRE2HXKMU2C3XEGDBGPQL5U5HUA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:10.314952727 (104h35m7.91172374s) [noise=true , address=185.167.204.206:4046 , piece_id=5WVC4R6VY25QUMGSOAORUK3WPGIADBXLKN7ZSLJ4GQEVKMSHZPKA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:36:03.322697363 (104h36m0.919468398s) [noise=true , address=185.167.204.206:4062 , piece_id=7NP2ZIMW6ZDXW3YHZSBO5PRTAZPLYLDWXDA4QPN5XH2C6KAGYLEA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:12BL6XLqsoUGm2FxrTgYTxLZakt2u2CAhZJG82oJ31eaE4vTtMr, address:185.167.204.206:4062): protocol: write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection; write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection
Mar 10 19:36:03.322771647 (104h36m0.919542676s) [noise=true , address=89.149.201.147:4007 , piece_id=AAJY5QBN55ENHTODH25WBVCFDT2SAQ2F27TKHFCIURCZSBMGOY6A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1Q7d4vj1KoWhDr6xd9gEfFrZ1sV3Yro2AFke2f3Ke1rXsGnvew, address:89.149.201.147:4007): protocol: read tcp 159.223.143.39:54460->89.149.201.147:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.322958851 (104h36m0.919729884s) [noise=true , address=185.167.204.205:4033 , piece_id=6JIW66HORXSEPSDWIUONOUZNA2UNXKBA4A6VZ7PBWW3IN2SCJSPQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:12Q73mXHLEfuX4VUbJ8PTjhMoXCrYSKjJbFxqjDdqF3U99N8FXe, address:185.167.204.205:4033): protocol: write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection; write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection
Mar 10 19:36:03.323134221 (104h36m0.919905244s) [noise=true , address=89.149.201.147:4018 , piece_id=V3KLJ75HJ3NRMBP2FIM33GC45QW4U5XMN2R4IAHSX32COGBJF47Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1Ku7o12bSmJ1YR3dGetGFcWqs1wJEh53Zkg3H7ELJrt4w1hkua, address:89.149.201.147:4018): protocol: read tcp 159.223.143.39:47122->89.149.201.147:4018: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.323461923 (104h36m0.920232956s) [noise=true , address=185.167.204.205:4022 , piece_id=F5UFUTAFPRBWMSU3QKV7ZSYQR6UUC7SBUDOC7AAYGDXTC5VYUBHA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1fT9qH1igPUNfrzpwa2VkBX7heh475sNqoN4Nsf9FYSwi1VmHj, address:185.167.204.205:4022): protocol: read tcp 159.223.143.39:51226->185.167.204.205:4022: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.323503666 (104h36m0.920274689s) [seg_pos={0 0}, upload=117052]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Mar 10 19:36:03.424787357 (104h36m1.021558384s) [upload=117052]: Done uploading segments...
Mar 10 19:36:03.424860036 (104h36m1.021631063s) [upload=117052]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 2.717436401s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................] duration:
2.222098208s trace:
https://jaeger.dev.storj.io/trace/78c3ad63aac6b527
FAIL (2026-03-10 20:19:01 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
:03.719970769 (104h35m1.316741801s) [noise=true , address=185.167.204.205:4033 , piece_id=6JIW66HORXSEPSDWIUONOUZNA2UNXKBA4A6VZ7PBWW3IN2SCJSPQ, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.720006314 (104h35m1.316777347s) [noise=true , address=185.167.204.205:4024 , piece_id=3CKPLBO33LOMNYBF47K2KGQFPO53ZXLMKRKWPVSR4FKQKXKMZMCA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.721245847 (104h35m1.318016883s) [noise=true , address=185.167.204.206:4046 , piece_id=5WVC4R6VY25QUMGSOAORUK3WPGIADBXLKN7ZSLJ4GQEVKMSHZPKA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.722131967 (104h35m1.318902999s) [noise=true , address=89.149.201.147:4004 , piece_id=TMKNRIR6RWRW3TUSA2MQL4LCNTJ3LKEFIT6BAVRX2V54QHNF26FA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.722154312 (104h35m1.318925349s) [noise=true , address=185.167.204.205:4028 , piece_id=IQ4CMHEIUNHIUBQV2PVFQ6BURDMWU2S55VGFP5Y7HPFSDVBY6PYA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.723422869 (104h35m1.320193903s) [noise=true , address=81.171.14.70:4047 , piece_id=DLCUNYJFEKIU3MWFXK24QGJ4GW2H5XE4Y2ARNL2W55EZUMSC4Z7Q, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.724205203 (104h35m1.320976237s) [noise=true , address=185.167.204.205:4022 , piece_id=F5UFUTAFPRBWMSU3QKV7ZSYQR6UUC7SBUDOC7AAYGDXTC5VYUBHA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.727803141 (104h35m1.324574176s) [noise=true , address=185.167.204.206:4067 , piece_id=RH2DWZ3FL7JX6EJ2WTJBPBSWDYVSHLYSLZCMMGRQXUDR6QXVZZ5A, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:04.167055423 (104h35m1.763826466s) [upload=117052]: Next returned nil segment
Mar 10 19:35:04.167072399 (104h35m1.763843424s) [upload=117052]: Waiting for error group managing segments...
Mar 10 19:35:04.350286979 (104h35m1.947058004s) [noise=true , address=81.171.14.70:4035 , piece_id=RNJYIOH4ETYQBSZFYTV3FX73J3PNFR2KBYNZEDXCVBOYMUAN5FCA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.351883208 (104h35m1.948654233s) [noise=true , address=81.171.14.70:4043 , piece_id=LSOHRZMEV5TEQUKZIJPYHVP43CIZKMXBTRVKIZIQNSSFNMAA5FLQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.356149085 (104h35m1.952920098s) [noise=true , address=185.167.204.205:4036 , piece_id=BBGCJ7U7PQDNB5NZTJWHUWXR6TFYF6D7LOR2UUBNV647YQX5JD2Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.369094269 (104h35m1.965865391s) [noise=true , address=89.149.201.147:4011 , piece_id=YHPLXK3ADD6VMYDWQ55PLROBCY2IOLOR2ZWEZ22FRRA4TXKSSWIQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.382982597 (104h35m1.979753611s) [noise=true , address=81.171.14.70:4034 , piece_id=BPXMDI54RTQVH5QKY6JCN5H3AKCPH62V76VBES4GQ6SLIZDIMMOA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.384672279 (104h35m1.981443307s) [noise=true , address=81.171.14.70:4047 , piece_id=DLCUNYJFEKIU3MWFXK24QGJ4GW2H5XE4Y2ARNL2W55EZUMSC4Z7Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.386337288 (104h35m1.98310832s) [noise=true , address=81.171.14.70:4025 , piece_id=R62A3AXM4OKUZFZJ2LB55UPB6VRGHBQLMXXVBWDTHUDWPGPIWEVQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.387583761 (104h35m1.984354793s) [noise=true , address=89.149.201.147:4009 , piece_id=XFSD3W6JWUXGXTHYE57SS5E5322JTRG7RKBI7MBF25GU7SEAKZJQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.388110220 (104h35m1.984881248s) [noise=true , address=81.171.14.70:4026 , piece_id=4CBQ4RZ44USXNYAGA3CIGNC3LBQXR675JVJX6CVTV3X7KVYPRHQA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.389451350 (104h35m1.986222362s) [noise=true , address=185.167.204.205:4012 , piece_id=7UFJIUKCHPKYZF42HTP6Q5D2FMHRSCRUQ7MH2MYJSEOZHZAJRZXA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.404093624 (104h35m2.00086464s) [noise=true , address=81.171.14.70:4048 , piece_id=4F2F4X4ZJ4FXUYQJQKB7YB3JZY53QNVRUSM3N4HQKGEYHMFYQB2A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.413111656 (104h35m2.009882687s) [noise=true , address=185.167.204.206:4071 , piece_id=ZQJLN55G37453GZS54ZDTPM7CEU5GH6LJR3BIKSJDC3C4QOKW5YQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.416183369 (104h35m2.012954404s) [noise=true , address=89.149.201.147:4014 , piece_id=WZOPRPDLFVOO6O2JYZZB75CLIAA4SVM7OEZXBX6WOJV3PUL2YSAQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.449375785 (104h35m2.046146815s) [noise=true , address=185.167.204.206:4039 , piece_id=DKKIZZMA3RAKMLUJTU6OANGBA3F6XWRD5ELVNXAGBIOZMFBVQWZQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.457716093 (104h35m2.054487108s) [noise=true , address=185.167.204.205:4024 , piece_id=3CKPLBO33LOMNYBF47K2KGQFPO53ZXLMKRKWPVSR4FKQKXKMZMCA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.491116161 (104h35m2.087887192s) [noise=true , address=185.167.204.205:4005 , piece_id=E7RO77MCAMFEAIE7ETPALCBV5S4X3DG5ZUD27ZE55RXYICIIXNEQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.507099744 (104h35m2.103870782s) [noise=true , address=185.167.204.205:4029 , piece_id=R33ONZELVLYL76YTJKOZASDJKFBACNNMC3PDUIITQRDGBWWZUNKA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.537139239 (104h35m2.133910269s) [noise=true , address=185.167.204.206:4067 , piece_id=RH2DWZ3FL7JX6EJ2WTJBPBSWDYVSHLYSLZCMMGRQXUDR6QXVZZ5A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.787100189 (104h35m2.383871224s) [noise=true , address=81.171.14.70:4033 , piece_id=ZFV477QXRX42OJIG4D5JLOYU3WERTD65PMHI5MFQLRTQJG3ENUTA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.962846444 (104h35m2.559617469s) [noise=true , address=89.149.201.147:4004 , piece_id=TMKNRIR6RWRW3TUSA2MQL4LCNTJ3LKEFIT6BAVRX2V54QHNF26FA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:08.299132133 (104h35m5.895903147s) [noise=true , address=185.167.204.205:4028 , piece_id=IQ4CMHEIUNHIUBQV2PVFQ6BURDMWU2S55VGFP5Y7HPFSDVBY6PYA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:08.481127955 (104h35m6.077898968s) [noise=true , address=185.167.204.206:4051 , piece_id=GO4OTIL5WFOS7V77FAT2NRFG2HRE2HXKMU2C3XEGDBGPQL5U5HUA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:10.314952727 (104h35m7.91172374s) [noise=true , address=185.167.204.206:4046 , piece_id=5WVC4R6VY25QUMGSOAORUK3WPGIADBXLKN7ZSLJ4GQEVKMSHZPKA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:36:03.322697363 (104h36m0.919468398s) [noise=true , address=185.167.204.206:4062 , piece_id=7NP2ZIMW6ZDXW3YHZSBO5PRTAZPLYLDWXDA4QPN5XH2C6KAGYLEA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:12BL6XLqsoUGm2FxrTgYTxLZakt2u2CAhZJG82oJ31eaE4vTtMr, address:185.167.204.206:4062): protocol: write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection; write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection
Mar 10 19:36:03.322771647 (104h36m0.919542676s) [noise=true , address=89.149.201.147:4007 , piece_id=AAJY5QBN55ENHTODH25WBVCFDT2SAQ2F27TKHFCIURCZSBMGOY6A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1Q7d4vj1KoWhDr6xd9gEfFrZ1sV3Yro2AFke2f3Ke1rXsGnvew, address:89.149.201.147:4007): protocol: read tcp 159.223.143.39:54460->89.149.201.147:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.322958851 (104h36m0.919729884s) [noise=true , address=185.167.204.205:4033 , piece_id=6JIW66HORXSEPSDWIUONOUZNA2UNXKBA4A6VZ7PBWW3IN2SCJSPQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:12Q73mXHLEfuX4VUbJ8PTjhMoXCrYSKjJbFxqjDdqF3U99N8FXe, address:185.167.204.205:4033): protocol: write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection; write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection
Mar 10 19:36:03.323134221 (104h36m0.919905244s) [noise=true , address=89.149.201.147:4018 , piece_id=V3KLJ75HJ3NRMBP2FIM33GC45QW4U5XMN2R4IAHSX32COGBJF47Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1Ku7o12bSmJ1YR3dGetGFcWqs1wJEh53Zkg3H7ELJrt4w1hkua, address:89.149.201.147:4018): protocol: read tcp 159.223.143.39:47122->89.149.201.147:4018: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.323461923 (104h36m0.920232956s) [noise=true , address=185.167.204.205:4022 , piece_id=F5UFUTAFPRBWMSU3QKV7ZSYQR6UUC7SBUDOC7AAYGDXTC5VYUBHA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1fT9qH1igPUNfrzpwa2VkBX7heh475sNqoN4Nsf9FYSwi1VmHj, address:185.167.204.205:4022): protocol: read tcp 159.223.143.39:51226->185.167.204.205:4022: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.323503666 (104h36m0.920274689s) [seg_pos={0 0}, upload=117052]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Mar 10 19:36:03.424787357 (104h36m1.021558384s) [upload=117052]: Done uploading segments...
Mar 10 19:36:03.424860036 (104h36m1.021631063s) [upload=117052]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 2.222146655s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
2.537538053s trace:
https://jaeger.dev.storj.io/trace/78c3ad63aac6b527
FAIL (2026-03-10 20:18:10 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
:03.719970769 (104h35m1.316741801s) [noise=true , address=185.167.204.205:4033 , piece_id=6JIW66HORXSEPSDWIUONOUZNA2UNXKBA4A6VZ7PBWW3IN2SCJSPQ, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.720006314 (104h35m1.316777347s) [noise=true , address=185.167.204.205:4024 , piece_id=3CKPLBO33LOMNYBF47K2KGQFPO53ZXLMKRKWPVSR4FKQKXKMZMCA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.721245847 (104h35m1.318016883s) [noise=true , address=185.167.204.206:4046 , piece_id=5WVC4R6VY25QUMGSOAORUK3WPGIADBXLKN7ZSLJ4GQEVKMSHZPKA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.722131967 (104h35m1.318902999s) [noise=true , address=89.149.201.147:4004 , piece_id=TMKNRIR6RWRW3TUSA2MQL4LCNTJ3LKEFIT6BAVRX2V54QHNF26FA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.722154312 (104h35m1.318925349s) [noise=true , address=185.167.204.205:4028 , piece_id=IQ4CMHEIUNHIUBQV2PVFQ6BURDMWU2S55VGFP5Y7HPFSDVBY6PYA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.723422869 (104h35m1.320193903s) [noise=true , address=81.171.14.70:4047 , piece_id=DLCUNYJFEKIU3MWFXK24QGJ4GW2H5XE4Y2ARNL2W55EZUMSC4Z7Q, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.724205203 (104h35m1.320976237s) [noise=true , address=185.167.204.205:4022 , piece_id=F5UFUTAFPRBWMSU3QKV7ZSYQR6UUC7SBUDOC7AAYGDXTC5VYUBHA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.727803141 (104h35m1.324574176s) [noise=true , address=185.167.204.206:4067 , piece_id=RH2DWZ3FL7JX6EJ2WTJBPBSWDYVSHLYSLZCMMGRQXUDR6QXVZZ5A, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:04.167055423 (104h35m1.763826466s) [upload=117052]: Next returned nil segment
Mar 10 19:35:04.167072399 (104h35m1.763843424s) [upload=117052]: Waiting for error group managing segments...
Mar 10 19:35:04.350286979 (104h35m1.947058004s) [noise=true , address=81.171.14.70:4035 , piece_id=RNJYIOH4ETYQBSZFYTV3FX73J3PNFR2KBYNZEDXCVBOYMUAN5FCA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.351883208 (104h35m1.948654233s) [noise=true , address=81.171.14.70:4043 , piece_id=LSOHRZMEV5TEQUKZIJPYHVP43CIZKMXBTRVKIZIQNSSFNMAA5FLQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.356149085 (104h35m1.952920098s) [noise=true , address=185.167.204.205:4036 , piece_id=BBGCJ7U7PQDNB5NZTJWHUWXR6TFYF6D7LOR2UUBNV647YQX5JD2Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.369094269 (104h35m1.965865391s) [noise=true , address=89.149.201.147:4011 , piece_id=YHPLXK3ADD6VMYDWQ55PLROBCY2IOLOR2ZWEZ22FRRA4TXKSSWIQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.382982597 (104h35m1.979753611s) [noise=true , address=81.171.14.70:4034 , piece_id=BPXMDI54RTQVH5QKY6JCN5H3AKCPH62V76VBES4GQ6SLIZDIMMOA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.384672279 (104h35m1.981443307s) [noise=true , address=81.171.14.70:4047 , piece_id=DLCUNYJFEKIU3MWFXK24QGJ4GW2H5XE4Y2ARNL2W55EZUMSC4Z7Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.386337288 (104h35m1.98310832s) [noise=true , address=81.171.14.70:4025 , piece_id=R62A3AXM4OKUZFZJ2LB55UPB6VRGHBQLMXXVBWDTHUDWPGPIWEVQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.387583761 (104h35m1.984354793s) [noise=true , address=89.149.201.147:4009 , piece_id=XFSD3W6JWUXGXTHYE57SS5E5322JTRG7RKBI7MBF25GU7SEAKZJQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.388110220 (104h35m1.984881248s) [noise=true , address=81.171.14.70:4026 , piece_id=4CBQ4RZ44USXNYAGA3CIGNC3LBQXR675JVJX6CVTV3X7KVYPRHQA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.389451350 (104h35m1.986222362s) [noise=true , address=185.167.204.205:4012 , piece_id=7UFJIUKCHPKYZF42HTP6Q5D2FMHRSCRUQ7MH2MYJSEOZHZAJRZXA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.404093624 (104h35m2.00086464s) [noise=true , address=81.171.14.70:4048 , piece_id=4F2F4X4ZJ4FXUYQJQKB7YB3JZY53QNVRUSM3N4HQKGEYHMFYQB2A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.413111656 (104h35m2.009882687s) [noise=true , address=185.167.204.206:4071 , piece_id=ZQJLN55G37453GZS54ZDTPM7CEU5GH6LJR3BIKSJDC3C4QOKW5YQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.416183369 (104h35m2.012954404s) [noise=true , address=89.149.201.147:4014 , piece_id=WZOPRPDLFVOO6O2JYZZB75CLIAA4SVM7OEZXBX6WOJV3PUL2YSAQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.449375785 (104h35m2.046146815s) [noise=true , address=185.167.204.206:4039 , piece_id=DKKIZZMA3RAKMLUJTU6OANGBA3F6XWRD5ELVNXAGBIOZMFBVQWZQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.457716093 (104h35m2.054487108s) [noise=true , address=185.167.204.205:4024 , piece_id=3CKPLBO33LOMNYBF47K2KGQFPO53ZXLMKRKWPVSR4FKQKXKMZMCA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.491116161 (104h35m2.087887192s) [noise=true , address=185.167.204.205:4005 , piece_id=E7RO77MCAMFEAIE7ETPALCBV5S4X3DG5ZUD27ZE55RXYICIIXNEQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.507099744 (104h35m2.103870782s) [noise=true , address=185.167.204.205:4029 , piece_id=R33ONZELVLYL76YTJKOZASDJKFBACNNMC3PDUIITQRDGBWWZUNKA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.537139239 (104h35m2.133910269s) [noise=true , address=185.167.204.206:4067 , piece_id=RH2DWZ3FL7JX6EJ2WTJBPBSWDYVSHLYSLZCMMGRQXUDR6QXVZZ5A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.787100189 (104h35m2.383871224s) [noise=true , address=81.171.14.70:4033 , piece_id=ZFV477QXRX42OJIG4D5JLOYU3WERTD65PMHI5MFQLRTQJG3ENUTA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.962846444 (104h35m2.559617469s) [noise=true , address=89.149.201.147:4004 , piece_id=TMKNRIR6RWRW3TUSA2MQL4LCNTJ3LKEFIT6BAVRX2V54QHNF26FA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:08.299132133 (104h35m5.895903147s) [noise=true , address=185.167.204.205:4028 , piece_id=IQ4CMHEIUNHIUBQV2PVFQ6BURDMWU2S55VGFP5Y7HPFSDVBY6PYA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:08.481127955 (104h35m6.077898968s) [noise=true , address=185.167.204.206:4051 , piece_id=GO4OTIL5WFOS7V77FAT2NRFG2HRE2HXKMU2C3XEGDBGPQL5U5HUA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:10.314952727 (104h35m7.91172374s) [noise=true , address=185.167.204.206:4046 , piece_id=5WVC4R6VY25QUMGSOAORUK3WPGIADBXLKN7ZSLJ4GQEVKMSHZPKA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:36:03.322697363 (104h36m0.919468398s) [noise=true , address=185.167.204.206:4062 , piece_id=7NP2ZIMW6ZDXW3YHZSBO5PRTAZPLYLDWXDA4QPN5XH2C6KAGYLEA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:12BL6XLqsoUGm2FxrTgYTxLZakt2u2CAhZJG82oJ31eaE4vTtMr, address:185.167.204.206:4062): protocol: write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection; write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection
Mar 10 19:36:03.322771647 (104h36m0.919542676s) [noise=true , address=89.149.201.147:4007 , piece_id=AAJY5QBN55ENHTODH25WBVCFDT2SAQ2F27TKHFCIURCZSBMGOY6A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1Q7d4vj1KoWhDr6xd9gEfFrZ1sV3Yro2AFke2f3Ke1rXsGnvew, address:89.149.201.147:4007): protocol: read tcp 159.223.143.39:54460->89.149.201.147:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.322958851 (104h36m0.919729884s) [noise=true , address=185.167.204.205:4033 , piece_id=6JIW66HORXSEPSDWIUONOUZNA2UNXKBA4A6VZ7PBWW3IN2SCJSPQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:12Q73mXHLEfuX4VUbJ8PTjhMoXCrYSKjJbFxqjDdqF3U99N8FXe, address:185.167.204.205:4033): protocol: write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection; write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection
Mar 10 19:36:03.323134221 (104h36m0.919905244s) [noise=true , address=89.149.201.147:4018 , piece_id=V3KLJ75HJ3NRMBP2FIM33GC45QW4U5XMN2R4IAHSX32COGBJF47Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1Ku7o12bSmJ1YR3dGetGFcWqs1wJEh53Zkg3H7ELJrt4w1hkua, address:89.149.201.147:4018): protocol: read tcp 159.223.143.39:47122->89.149.201.147:4018: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.323461923 (104h36m0.920232956s) [noise=true , address=185.167.204.205:4022 , piece_id=F5UFUTAFPRBWMSU3QKV7ZSYQR6UUC7SBUDOC7AAYGDXTC5VYUBHA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1fT9qH1igPUNfrzpwa2VkBX7heh475sNqoN4Nsf9FYSwi1VmHj, address:185.167.204.205:4022): protocol: read tcp 159.223.143.39:51226->185.167.204.205:4022: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.323503666 (104h36m0.920274689s) [seg_pos={0 0}, upload=117052]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Mar 10 19:36:03.424787357 (104h36m1.021558384s) [upload=117052]: Done uploading segments...
Mar 10 19:36:03.424860036 (104h36m1.021631063s) [upload=117052]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 2.537569028s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................] duration:
2.761721346s trace:
https://jaeger.dev.storj.io/trace/78c3ad63aac6b527
FAIL (2026-03-10 20:17:00 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
:03.719970769 (104h35m1.316741801s) [noise=true , address=185.167.204.205:4033 , piece_id=6JIW66HORXSEPSDWIUONOUZNA2UNXKBA4A6VZ7PBWW3IN2SCJSPQ, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.720006314 (104h35m1.316777347s) [noise=true , address=185.167.204.205:4024 , piece_id=3CKPLBO33LOMNYBF47K2KGQFPO53ZXLMKRKWPVSR4FKQKXKMZMCA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.721245847 (104h35m1.318016883s) [noise=true , address=185.167.204.206:4046 , piece_id=5WVC4R6VY25QUMGSOAORUK3WPGIADBXLKN7ZSLJ4GQEVKMSHZPKA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.722131967 (104h35m1.318902999s) [noise=true , address=89.149.201.147:4004 , piece_id=TMKNRIR6RWRW3TUSA2MQL4LCNTJ3LKEFIT6BAVRX2V54QHNF26FA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.722154312 (104h35m1.318925349s) [noise=true , address=185.167.204.205:4028 , piece_id=IQ4CMHEIUNHIUBQV2PVFQ6BURDMWU2S55VGFP5Y7HPFSDVBY6PYA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.723422869 (104h35m1.320193903s) [noise=true , address=81.171.14.70:4047 , piece_id=DLCUNYJFEKIU3MWFXK24QGJ4GW2H5XE4Y2ARNL2W55EZUMSC4Z7Q, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.724205203 (104h35m1.320976237s) [noise=true , address=185.167.204.205:4022 , piece_id=F5UFUTAFPRBWMSU3QKV7ZSYQR6UUC7SBUDOC7AAYGDXTC5VYUBHA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.727803141 (104h35m1.324574176s) [noise=true , address=185.167.204.206:4067 , piece_id=RH2DWZ3FL7JX6EJ2WTJBPBSWDYVSHLYSLZCMMGRQXUDR6QXVZZ5A, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:04.167055423 (104h35m1.763826466s) [upload=117052]: Next returned nil segment
Mar 10 19:35:04.167072399 (104h35m1.763843424s) [upload=117052]: Waiting for error group managing segments...
Mar 10 19:35:04.350286979 (104h35m1.947058004s) [noise=true , address=81.171.14.70:4035 , piece_id=RNJYIOH4ETYQBSZFYTV3FX73J3PNFR2KBYNZEDXCVBOYMUAN5FCA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.351883208 (104h35m1.948654233s) [noise=true , address=81.171.14.70:4043 , piece_id=LSOHRZMEV5TEQUKZIJPYHVP43CIZKMXBTRVKIZIQNSSFNMAA5FLQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.356149085 (104h35m1.952920098s) [noise=true , address=185.167.204.205:4036 , piece_id=BBGCJ7U7PQDNB5NZTJWHUWXR6TFYF6D7LOR2UUBNV647YQX5JD2Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.369094269 (104h35m1.965865391s) [noise=true , address=89.149.201.147:4011 , piece_id=YHPLXK3ADD6VMYDWQ55PLROBCY2IOLOR2ZWEZ22FRRA4TXKSSWIQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.382982597 (104h35m1.979753611s) [noise=true , address=81.171.14.70:4034 , piece_id=BPXMDI54RTQVH5QKY6JCN5H3AKCPH62V76VBES4GQ6SLIZDIMMOA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.384672279 (104h35m1.981443307s) [noise=true , address=81.171.14.70:4047 , piece_id=DLCUNYJFEKIU3MWFXK24QGJ4GW2H5XE4Y2ARNL2W55EZUMSC4Z7Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.386337288 (104h35m1.98310832s) [noise=true , address=81.171.14.70:4025 , piece_id=R62A3AXM4OKUZFZJ2LB55UPB6VRGHBQLMXXVBWDTHUDWPGPIWEVQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.387583761 (104h35m1.984354793s) [noise=true , address=89.149.201.147:4009 , piece_id=XFSD3W6JWUXGXTHYE57SS5E5322JTRG7RKBI7MBF25GU7SEAKZJQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.388110220 (104h35m1.984881248s) [noise=true , address=81.171.14.70:4026 , piece_id=4CBQ4RZ44USXNYAGA3CIGNC3LBQXR675JVJX6CVTV3X7KVYPRHQA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.389451350 (104h35m1.986222362s) [noise=true , address=185.167.204.205:4012 , piece_id=7UFJIUKCHPKYZF42HTP6Q5D2FMHRSCRUQ7MH2MYJSEOZHZAJRZXA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.404093624 (104h35m2.00086464s) [noise=true , address=81.171.14.70:4048 , piece_id=4F2F4X4ZJ4FXUYQJQKB7YB3JZY53QNVRUSM3N4HQKGEYHMFYQB2A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.413111656 (104h35m2.009882687s) [noise=true , address=185.167.204.206:4071 , piece_id=ZQJLN55G37453GZS54ZDTPM7CEU5GH6LJR3BIKSJDC3C4QOKW5YQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.416183369 (104h35m2.012954404s) [noise=true , address=89.149.201.147:4014 , piece_id=WZOPRPDLFVOO6O2JYZZB75CLIAA4SVM7OEZXBX6WOJV3PUL2YSAQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.449375785 (104h35m2.046146815s) [noise=true , address=185.167.204.206:4039 , piece_id=DKKIZZMA3RAKMLUJTU6OANGBA3F6XWRD5ELVNXAGBIOZMFBVQWZQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.457716093 (104h35m2.054487108s) [noise=true , address=185.167.204.205:4024 , piece_id=3CKPLBO33LOMNYBF47K2KGQFPO53ZXLMKRKWPVSR4FKQKXKMZMCA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.491116161 (104h35m2.087887192s) [noise=true , address=185.167.204.205:4005 , piece_id=E7RO77MCAMFEAIE7ETPALCBV5S4X3DG5ZUD27ZE55RXYICIIXNEQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.507099744 (104h35m2.103870782s) [noise=true , address=185.167.204.205:4029 , piece_id=R33ONZELVLYL76YTJKOZASDJKFBACNNMC3PDUIITQRDGBWWZUNKA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.537139239 (104h35m2.133910269s) [noise=true , address=185.167.204.206:4067 , piece_id=RH2DWZ3FL7JX6EJ2WTJBPBSWDYVSHLYSLZCMMGRQXUDR6QXVZZ5A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.787100189 (104h35m2.383871224s) [noise=true , address=81.171.14.70:4033 , piece_id=ZFV477QXRX42OJIG4D5JLOYU3WERTD65PMHI5MFQLRTQJG3ENUTA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.962846444 (104h35m2.559617469s) [noise=true , address=89.149.201.147:4004 , piece_id=TMKNRIR6RWRW3TUSA2MQL4LCNTJ3LKEFIT6BAVRX2V54QHNF26FA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:08.299132133 (104h35m5.895903147s) [noise=true , address=185.167.204.205:4028 , piece_id=IQ4CMHEIUNHIUBQV2PVFQ6BURDMWU2S55VGFP5Y7HPFSDVBY6PYA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:08.481127955 (104h35m6.077898968s) [noise=true , address=185.167.204.206:4051 , piece_id=GO4OTIL5WFOS7V77FAT2NRFG2HRE2HXKMU2C3XEGDBGPQL5U5HUA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:10.314952727 (104h35m7.91172374s) [noise=true , address=185.167.204.206:4046 , piece_id=5WVC4R6VY25QUMGSOAORUK3WPGIADBXLKN7ZSLJ4GQEVKMSHZPKA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:36:03.322697363 (104h36m0.919468398s) [noise=true , address=185.167.204.206:4062 , piece_id=7NP2ZIMW6ZDXW3YHZSBO5PRTAZPLYLDWXDA4QPN5XH2C6KAGYLEA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:12BL6XLqsoUGm2FxrTgYTxLZakt2u2CAhZJG82oJ31eaE4vTtMr, address:185.167.204.206:4062): protocol: write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection; write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection
Mar 10 19:36:03.322771647 (104h36m0.919542676s) [noise=true , address=89.149.201.147:4007 , piece_id=AAJY5QBN55ENHTODH25WBVCFDT2SAQ2F27TKHFCIURCZSBMGOY6A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1Q7d4vj1KoWhDr6xd9gEfFrZ1sV3Yro2AFke2f3Ke1rXsGnvew, address:89.149.201.147:4007): protocol: read tcp 159.223.143.39:54460->89.149.201.147:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.322958851 (104h36m0.919729884s) [noise=true , address=185.167.204.205:4033 , piece_id=6JIW66HORXSEPSDWIUONOUZNA2UNXKBA4A6VZ7PBWW3IN2SCJSPQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:12Q73mXHLEfuX4VUbJ8PTjhMoXCrYSKjJbFxqjDdqF3U99N8FXe, address:185.167.204.205:4033): protocol: write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection; write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection
Mar 10 19:36:03.323134221 (104h36m0.919905244s) [noise=true , address=89.149.201.147:4018 , piece_id=V3KLJ75HJ3NRMBP2FIM33GC45QW4U5XMN2R4IAHSX32COGBJF47Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1Ku7o12bSmJ1YR3dGetGFcWqs1wJEh53Zkg3H7ELJrt4w1hkua, address:89.149.201.147:4018): protocol: read tcp 159.223.143.39:47122->89.149.201.147:4018: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.323461923 (104h36m0.920232956s) [noise=true , address=185.167.204.205:4022 , piece_id=F5UFUTAFPRBWMSU3QKV7ZSYQR6UUC7SBUDOC7AAYGDXTC5VYUBHA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1fT9qH1igPUNfrzpwa2VkBX7heh475sNqoN4Nsf9FYSwi1VmHj, address:185.167.204.205:4022): protocol: read tcp 159.223.143.39:51226->185.167.204.205:4022: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.323503666 (104h36m0.920274689s) [seg_pos={0 0}, upload=117052]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Mar 10 19:36:03.424787357 (104h36m1.021558384s) [upload=117052]: Done uploading segments...
Mar 10 19:36:03.424860036 (104h36m1.021631063s) [upload=117052]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 2.761751804s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [......X.....................................................] duration:
2.597916471s trace:
https://jaeger.dev.storj.io/trace/78c3ad63aac6b527
FAIL (2026-03-10 20:16:00 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
:03.719970769 (104h35m1.316741801s) [noise=true , address=185.167.204.205:4033 , piece_id=6JIW66HORXSEPSDWIUONOUZNA2UNXKBA4A6VZ7PBWW3IN2SCJSPQ, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.720006314 (104h35m1.316777347s) [noise=true , address=185.167.204.205:4024 , piece_id=3CKPLBO33LOMNYBF47K2KGQFPO53ZXLMKRKWPVSR4FKQKXKMZMCA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.721245847 (104h35m1.318016883s) [noise=true , address=185.167.204.206:4046 , piece_id=5WVC4R6VY25QUMGSOAORUK3WPGIADBXLKN7ZSLJ4GQEVKMSHZPKA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.722131967 (104h35m1.318902999s) [noise=true , address=89.149.201.147:4004 , piece_id=TMKNRIR6RWRW3TUSA2MQL4LCNTJ3LKEFIT6BAVRX2V54QHNF26FA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.722154312 (104h35m1.318925349s) [noise=true , address=185.167.204.205:4028 , piece_id=IQ4CMHEIUNHIUBQV2PVFQ6BURDMWU2S55VGFP5Y7HPFSDVBY6PYA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.723422869 (104h35m1.320193903s) [noise=true , address=81.171.14.70:4047 , piece_id=DLCUNYJFEKIU3MWFXK24QGJ4GW2H5XE4Y2ARNL2W55EZUMSC4Z7Q, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.724205203 (104h35m1.320976237s) [noise=true , address=185.167.204.205:4022 , piece_id=F5UFUTAFPRBWMSU3QKV7ZSYQR6UUC7SBUDOC7AAYGDXTC5VYUBHA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.727803141 (104h35m1.324574176s) [noise=true , address=185.167.204.206:4067 , piece_id=RH2DWZ3FL7JX6EJ2WTJBPBSWDYVSHLYSLZCMMGRQXUDR6QXVZZ5A, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:04.167055423 (104h35m1.763826466s) [upload=117052]: Next returned nil segment
Mar 10 19:35:04.167072399 (104h35m1.763843424s) [upload=117052]: Waiting for error group managing segments...
Mar 10 19:35:04.350286979 (104h35m1.947058004s) [noise=true , address=81.171.14.70:4035 , piece_id=RNJYIOH4ETYQBSZFYTV3FX73J3PNFR2KBYNZEDXCVBOYMUAN5FCA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.351883208 (104h35m1.948654233s) [noise=true , address=81.171.14.70:4043 , piece_id=LSOHRZMEV5TEQUKZIJPYHVP43CIZKMXBTRVKIZIQNSSFNMAA5FLQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.356149085 (104h35m1.952920098s) [noise=true , address=185.167.204.205:4036 , piece_id=BBGCJ7U7PQDNB5NZTJWHUWXR6TFYF6D7LOR2UUBNV647YQX5JD2Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.369094269 (104h35m1.965865391s) [noise=true , address=89.149.201.147:4011 , piece_id=YHPLXK3ADD6VMYDWQ55PLROBCY2IOLOR2ZWEZ22FRRA4TXKSSWIQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.382982597 (104h35m1.979753611s) [noise=true , address=81.171.14.70:4034 , piece_id=BPXMDI54RTQVH5QKY6JCN5H3AKCPH62V76VBES4GQ6SLIZDIMMOA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.384672279 (104h35m1.981443307s) [noise=true , address=81.171.14.70:4047 , piece_id=DLCUNYJFEKIU3MWFXK24QGJ4GW2H5XE4Y2ARNL2W55EZUMSC4Z7Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.386337288 (104h35m1.98310832s) [noise=true , address=81.171.14.70:4025 , piece_id=R62A3AXM4OKUZFZJ2LB55UPB6VRGHBQLMXXVBWDTHUDWPGPIWEVQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.387583761 (104h35m1.984354793s) [noise=true , address=89.149.201.147:4009 , piece_id=XFSD3W6JWUXGXTHYE57SS5E5322JTRG7RKBI7MBF25GU7SEAKZJQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.388110220 (104h35m1.984881248s) [noise=true , address=81.171.14.70:4026 , piece_id=4CBQ4RZ44USXNYAGA3CIGNC3LBQXR675JVJX6CVTV3X7KVYPRHQA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.389451350 (104h35m1.986222362s) [noise=true , address=185.167.204.205:4012 , piece_id=7UFJIUKCHPKYZF42HTP6Q5D2FMHRSCRUQ7MH2MYJSEOZHZAJRZXA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.404093624 (104h35m2.00086464s) [noise=true , address=81.171.14.70:4048 , piece_id=4F2F4X4ZJ4FXUYQJQKB7YB3JZY53QNVRUSM3N4HQKGEYHMFYQB2A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.413111656 (104h35m2.009882687s) [noise=true , address=185.167.204.206:4071 , piece_id=ZQJLN55G37453GZS54ZDTPM7CEU5GH6LJR3BIKSJDC3C4QOKW5YQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.416183369 (104h35m2.012954404s) [noise=true , address=89.149.201.147:4014 , piece_id=WZOPRPDLFVOO6O2JYZZB75CLIAA4SVM7OEZXBX6WOJV3PUL2YSAQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.449375785 (104h35m2.046146815s) [noise=true , address=185.167.204.206:4039 , piece_id=DKKIZZMA3RAKMLUJTU6OANGBA3F6XWRD5ELVNXAGBIOZMFBVQWZQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.457716093 (104h35m2.054487108s) [noise=true , address=185.167.204.205:4024 , piece_id=3CKPLBO33LOMNYBF47K2KGQFPO53ZXLMKRKWPVSR4FKQKXKMZMCA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.491116161 (104h35m2.087887192s) [noise=true , address=185.167.204.205:4005 , piece_id=E7RO77MCAMFEAIE7ETPALCBV5S4X3DG5ZUD27ZE55RXYICIIXNEQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.507099744 (104h35m2.103870782s) [noise=true , address=185.167.204.205:4029 , piece_id=R33ONZELVLYL76YTJKOZASDJKFBACNNMC3PDUIITQRDGBWWZUNKA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.537139239 (104h35m2.133910269s) [noise=true , address=185.167.204.206:4067 , piece_id=RH2DWZ3FL7JX6EJ2WTJBPBSWDYVSHLYSLZCMMGRQXUDR6QXVZZ5A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.787100189 (104h35m2.383871224s) [noise=true , address=81.171.14.70:4033 , piece_id=ZFV477QXRX42OJIG4D5JLOYU3WERTD65PMHI5MFQLRTQJG3ENUTA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.962846444 (104h35m2.559617469s) [noise=true , address=89.149.201.147:4004 , piece_id=TMKNRIR6RWRW3TUSA2MQL4LCNTJ3LKEFIT6BAVRX2V54QHNF26FA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:08.299132133 (104h35m5.895903147s) [noise=true , address=185.167.204.205:4028 , piece_id=IQ4CMHEIUNHIUBQV2PVFQ6BURDMWU2S55VGFP5Y7HPFSDVBY6PYA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:08.481127955 (104h35m6.077898968s) [noise=true , address=185.167.204.206:4051 , piece_id=GO4OTIL5WFOS7V77FAT2NRFG2HRE2HXKMU2C3XEGDBGPQL5U5HUA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:10.314952727 (104h35m7.91172374s) [noise=true , address=185.167.204.206:4046 , piece_id=5WVC4R6VY25QUMGSOAORUK3WPGIADBXLKN7ZSLJ4GQEVKMSHZPKA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:36:03.322697363 (104h36m0.919468398s) [noise=true , address=185.167.204.206:4062 , piece_id=7NP2ZIMW6ZDXW3YHZSBO5PRTAZPLYLDWXDA4QPN5XH2C6KAGYLEA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:12BL6XLqsoUGm2FxrTgYTxLZakt2u2CAhZJG82oJ31eaE4vTtMr, address:185.167.204.206:4062): protocol: write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection; write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection
Mar 10 19:36:03.322771647 (104h36m0.919542676s) [noise=true , address=89.149.201.147:4007 , piece_id=AAJY5QBN55ENHTODH25WBVCFDT2SAQ2F27TKHFCIURCZSBMGOY6A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1Q7d4vj1KoWhDr6xd9gEfFrZ1sV3Yro2AFke2f3Ke1rXsGnvew, address:89.149.201.147:4007): protocol: read tcp 159.223.143.39:54460->89.149.201.147:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.322958851 (104h36m0.919729884s) [noise=true , address=185.167.204.205:4033 , piece_id=6JIW66HORXSEPSDWIUONOUZNA2UNXKBA4A6VZ7PBWW3IN2SCJSPQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:12Q73mXHLEfuX4VUbJ8PTjhMoXCrYSKjJbFxqjDdqF3U99N8FXe, address:185.167.204.205:4033): protocol: write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection; write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection
Mar 10 19:36:03.323134221 (104h36m0.919905244s) [noise=true , address=89.149.201.147:4018 , piece_id=V3KLJ75HJ3NRMBP2FIM33GC45QW4U5XMN2R4IAHSX32COGBJF47Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1Ku7o12bSmJ1YR3dGetGFcWqs1wJEh53Zkg3H7ELJrt4w1hkua, address:89.149.201.147:4018): protocol: read tcp 159.223.143.39:47122->89.149.201.147:4018: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.323461923 (104h36m0.920232956s) [noise=true , address=185.167.204.205:4022 , piece_id=F5UFUTAFPRBWMSU3QKV7ZSYQR6UUC7SBUDOC7AAYGDXTC5VYUBHA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1fT9qH1igPUNfrzpwa2VkBX7heh475sNqoN4Nsf9FYSwi1VmHj, address:185.167.204.205:4022): protocol: read tcp 159.223.143.39:51226->185.167.204.205:4022: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.323503666 (104h36m0.920274689s) [seg_pos={0 0}, upload=117052]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Mar 10 19:36:03.424787357 (104h36m1.021558384s) [upload=117052]: Done uploading segments...
Mar 10 19:36:03.424860036 (104h36m1.021631063s) [upload=117052]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 2.597978379s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.......X....................................................] duration:
2.159283885s trace:
https://jaeger.dev.storj.io/trace/78c3ad63aac6b527
FAIL (2026-03-10 20:15:20 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
:03.719970769 (104h35m1.316741801s) [noise=true , address=185.167.204.205:4033 , piece_id=6JIW66HORXSEPSDWIUONOUZNA2UNXKBA4A6VZ7PBWW3IN2SCJSPQ, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.720006314 (104h35m1.316777347s) [noise=true , address=185.167.204.205:4024 , piece_id=3CKPLBO33LOMNYBF47K2KGQFPO53ZXLMKRKWPVSR4FKQKXKMZMCA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.721245847 (104h35m1.318016883s) [noise=true , address=185.167.204.206:4046 , piece_id=5WVC4R6VY25QUMGSOAORUK3WPGIADBXLKN7ZSLJ4GQEVKMSHZPKA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.722131967 (104h35m1.318902999s) [noise=true , address=89.149.201.147:4004 , piece_id=TMKNRIR6RWRW3TUSA2MQL4LCNTJ3LKEFIT6BAVRX2V54QHNF26FA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.722154312 (104h35m1.318925349s) [noise=true , address=185.167.204.205:4028 , piece_id=IQ4CMHEIUNHIUBQV2PVFQ6BURDMWU2S55VGFP5Y7HPFSDVBY6PYA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.723422869 (104h35m1.320193903s) [noise=true , address=81.171.14.70:4047 , piece_id=DLCUNYJFEKIU3MWFXK24QGJ4GW2H5XE4Y2ARNL2W55EZUMSC4Z7Q, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.724205203 (104h35m1.320976237s) [noise=true , address=185.167.204.205:4022 , piece_id=F5UFUTAFPRBWMSU3QKV7ZSYQR6UUC7SBUDOC7AAYGDXTC5VYUBHA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.727803141 (104h35m1.324574176s) [noise=true , address=185.167.204.206:4067 , piece_id=RH2DWZ3FL7JX6EJ2WTJBPBSWDYVSHLYSLZCMMGRQXUDR6QXVZZ5A, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:04.167055423 (104h35m1.763826466s) [upload=117052]: Next returned nil segment
Mar 10 19:35:04.167072399 (104h35m1.763843424s) [upload=117052]: Waiting for error group managing segments...
Mar 10 19:35:04.350286979 (104h35m1.947058004s) [noise=true , address=81.171.14.70:4035 , piece_id=RNJYIOH4ETYQBSZFYTV3FX73J3PNFR2KBYNZEDXCVBOYMUAN5FCA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.351883208 (104h35m1.948654233s) [noise=true , address=81.171.14.70:4043 , piece_id=LSOHRZMEV5TEQUKZIJPYHVP43CIZKMXBTRVKIZIQNSSFNMAA5FLQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.356149085 (104h35m1.952920098s) [noise=true , address=185.167.204.205:4036 , piece_id=BBGCJ7U7PQDNB5NZTJWHUWXR6TFYF6D7LOR2UUBNV647YQX5JD2Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.369094269 (104h35m1.965865391s) [noise=true , address=89.149.201.147:4011 , piece_id=YHPLXK3ADD6VMYDWQ55PLROBCY2IOLOR2ZWEZ22FRRA4TXKSSWIQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.382982597 (104h35m1.979753611s) [noise=true , address=81.171.14.70:4034 , piece_id=BPXMDI54RTQVH5QKY6JCN5H3AKCPH62V76VBES4GQ6SLIZDIMMOA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.384672279 (104h35m1.981443307s) [noise=true , address=81.171.14.70:4047 , piece_id=DLCUNYJFEKIU3MWFXK24QGJ4GW2H5XE4Y2ARNL2W55EZUMSC4Z7Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.386337288 (104h35m1.98310832s) [noise=true , address=81.171.14.70:4025 , piece_id=R62A3AXM4OKUZFZJ2LB55UPB6VRGHBQLMXXVBWDTHUDWPGPIWEVQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.387583761 (104h35m1.984354793s) [noise=true , address=89.149.201.147:4009 , piece_id=XFSD3W6JWUXGXTHYE57SS5E5322JTRG7RKBI7MBF25GU7SEAKZJQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.388110220 (104h35m1.984881248s) [noise=true , address=81.171.14.70:4026 , piece_id=4CBQ4RZ44USXNYAGA3CIGNC3LBQXR675JVJX6CVTV3X7KVYPRHQA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.389451350 (104h35m1.986222362s) [noise=true , address=185.167.204.205:4012 , piece_id=7UFJIUKCHPKYZF42HTP6Q5D2FMHRSCRUQ7MH2MYJSEOZHZAJRZXA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.404093624 (104h35m2.00086464s) [noise=true , address=81.171.14.70:4048 , piece_id=4F2F4X4ZJ4FXUYQJQKB7YB3JZY53QNVRUSM3N4HQKGEYHMFYQB2A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.413111656 (104h35m2.009882687s) [noise=true , address=185.167.204.206:4071 , piece_id=ZQJLN55G37453GZS54ZDTPM7CEU5GH6LJR3BIKSJDC3C4QOKW5YQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.416183369 (104h35m2.012954404s) [noise=true , address=89.149.201.147:4014 , piece_id=WZOPRPDLFVOO6O2JYZZB75CLIAA4SVM7OEZXBX6WOJV3PUL2YSAQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.449375785 (104h35m2.046146815s) [noise=true , address=185.167.204.206:4039 , piece_id=DKKIZZMA3RAKMLUJTU6OANGBA3F6XWRD5ELVNXAGBIOZMFBVQWZQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.457716093 (104h35m2.054487108s) [noise=true , address=185.167.204.205:4024 , piece_id=3CKPLBO33LOMNYBF47K2KGQFPO53ZXLMKRKWPVSR4FKQKXKMZMCA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.491116161 (104h35m2.087887192s) [noise=true , address=185.167.204.205:4005 , piece_id=E7RO77MCAMFEAIE7ETPALCBV5S4X3DG5ZUD27ZE55RXYICIIXNEQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.507099744 (104h35m2.103870782s) [noise=true , address=185.167.204.205:4029 , piece_id=R33ONZELVLYL76YTJKOZASDJKFBACNNMC3PDUIITQRDGBWWZUNKA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.537139239 (104h35m2.133910269s) [noise=true , address=185.167.204.206:4067 , piece_id=RH2DWZ3FL7JX6EJ2WTJBPBSWDYVSHLYSLZCMMGRQXUDR6QXVZZ5A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.787100189 (104h35m2.383871224s) [noise=true , address=81.171.14.70:4033 , piece_id=ZFV477QXRX42OJIG4D5JLOYU3WERTD65PMHI5MFQLRTQJG3ENUTA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.962846444 (104h35m2.559617469s) [noise=true , address=89.149.201.147:4004 , piece_id=TMKNRIR6RWRW3TUSA2MQL4LCNTJ3LKEFIT6BAVRX2V54QHNF26FA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:08.299132133 (104h35m5.895903147s) [noise=true , address=185.167.204.205:4028 , piece_id=IQ4CMHEIUNHIUBQV2PVFQ6BURDMWU2S55VGFP5Y7HPFSDVBY6PYA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:08.481127955 (104h35m6.077898968s) [noise=true , address=185.167.204.206:4051 , piece_id=GO4OTIL5WFOS7V77FAT2NRFG2HRE2HXKMU2C3XEGDBGPQL5U5HUA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:10.314952727 (104h35m7.91172374s) [noise=true , address=185.167.204.206:4046 , piece_id=5WVC4R6VY25QUMGSOAORUK3WPGIADBXLKN7ZSLJ4GQEVKMSHZPKA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:36:03.322697363 (104h36m0.919468398s) [noise=true , address=185.167.204.206:4062 , piece_id=7NP2ZIMW6ZDXW3YHZSBO5PRTAZPLYLDWXDA4QPN5XH2C6KAGYLEA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:12BL6XLqsoUGm2FxrTgYTxLZakt2u2CAhZJG82oJ31eaE4vTtMr, address:185.167.204.206:4062): protocol: write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection; write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection
Mar 10 19:36:03.322771647 (104h36m0.919542676s) [noise=true , address=89.149.201.147:4007 , piece_id=AAJY5QBN55ENHTODH25WBVCFDT2SAQ2F27TKHFCIURCZSBMGOY6A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1Q7d4vj1KoWhDr6xd9gEfFrZ1sV3Yro2AFke2f3Ke1rXsGnvew, address:89.149.201.147:4007): protocol: read tcp 159.223.143.39:54460->89.149.201.147:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.322958851 (104h36m0.919729884s) [noise=true , address=185.167.204.205:4033 , piece_id=6JIW66HORXSEPSDWIUONOUZNA2UNXKBA4A6VZ7PBWW3IN2SCJSPQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:12Q73mXHLEfuX4VUbJ8PTjhMoXCrYSKjJbFxqjDdqF3U99N8FXe, address:185.167.204.205:4033): protocol: write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection; write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection
Mar 10 19:36:03.323134221 (104h36m0.919905244s) [noise=true , address=89.149.201.147:4018 , piece_id=V3KLJ75HJ3NRMBP2FIM33GC45QW4U5XMN2R4IAHSX32COGBJF47Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1Ku7o12bSmJ1YR3dGetGFcWqs1wJEh53Zkg3H7ELJrt4w1hkua, address:89.149.201.147:4018): protocol: read tcp 159.223.143.39:47122->89.149.201.147:4018: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.323461923 (104h36m0.920232956s) [noise=true , address=185.167.204.205:4022 , piece_id=F5UFUTAFPRBWMSU3QKV7ZSYQR6UUC7SBUDOC7AAYGDXTC5VYUBHA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1fT9qH1igPUNfrzpwa2VkBX7heh475sNqoN4Nsf9FYSwi1VmHj, address:185.167.204.205:4022): protocol: read tcp 159.223.143.39:51226->185.167.204.205:4022: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.323503666 (104h36m0.920274689s) [seg_pos={0 0}, upload=117052]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Mar 10 19:36:03.424787357 (104h36m1.021558384s) [upload=117052]: Done uploading segments...
Mar 10 19:36:03.424860036 (104h36m1.021631063s) [upload=117052]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 2.159314601s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [X.......X...................................................] duration:
2.167822563s trace:
https://jaeger.dev.storj.io/trace/78c3ad63aac6b527
FAIL (2026-03-10 20:14:18 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
:03.719970769 (104h35m1.316741801s) [noise=true , address=185.167.204.205:4033 , piece_id=6JIW66HORXSEPSDWIUONOUZNA2UNXKBA4A6VZ7PBWW3IN2SCJSPQ, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.720006314 (104h35m1.316777347s) [noise=true , address=185.167.204.205:4024 , piece_id=3CKPLBO33LOMNYBF47K2KGQFPO53ZXLMKRKWPVSR4FKQKXKMZMCA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.721245847 (104h35m1.318016883s) [noise=true , address=185.167.204.206:4046 , piece_id=5WVC4R6VY25QUMGSOAORUK3WPGIADBXLKN7ZSLJ4GQEVKMSHZPKA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.722131967 (104h35m1.318902999s) [noise=true , address=89.149.201.147:4004 , piece_id=TMKNRIR6RWRW3TUSA2MQL4LCNTJ3LKEFIT6BAVRX2V54QHNF26FA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.722154312 (104h35m1.318925349s) [noise=true , address=185.167.204.205:4028 , piece_id=IQ4CMHEIUNHIUBQV2PVFQ6BURDMWU2S55VGFP5Y7HPFSDVBY6PYA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.723422869 (104h35m1.320193903s) [noise=true , address=81.171.14.70:4047 , piece_id=DLCUNYJFEKIU3MWFXK24QGJ4GW2H5XE4Y2ARNL2W55EZUMSC4Z7Q, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.724205203 (104h35m1.320976237s) [noise=true , address=185.167.204.205:4022 , piece_id=F5UFUTAFPRBWMSU3QKV7ZSYQR6UUC7SBUDOC7AAYGDXTC5VYUBHA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.727803141 (104h35m1.324574176s) [noise=true , address=185.167.204.206:4067 , piece_id=RH2DWZ3FL7JX6EJ2WTJBPBSWDYVSHLYSLZCMMGRQXUDR6QXVZZ5A, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:04.167055423 (104h35m1.763826466s) [upload=117052]: Next returned nil segment
Mar 10 19:35:04.167072399 (104h35m1.763843424s) [upload=117052]: Waiting for error group managing segments...
Mar 10 19:35:04.350286979 (104h35m1.947058004s) [noise=true , address=81.171.14.70:4035 , piece_id=RNJYIOH4ETYQBSZFYTV3FX73J3PNFR2KBYNZEDXCVBOYMUAN5FCA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.351883208 (104h35m1.948654233s) [noise=true , address=81.171.14.70:4043 , piece_id=LSOHRZMEV5TEQUKZIJPYHVP43CIZKMXBTRVKIZIQNSSFNMAA5FLQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.356149085 (104h35m1.952920098s) [noise=true , address=185.167.204.205:4036 , piece_id=BBGCJ7U7PQDNB5NZTJWHUWXR6TFYF6D7LOR2UUBNV647YQX5JD2Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.369094269 (104h35m1.965865391s) [noise=true , address=89.149.201.147:4011 , piece_id=YHPLXK3ADD6VMYDWQ55PLROBCY2IOLOR2ZWEZ22FRRA4TXKSSWIQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.382982597 (104h35m1.979753611s) [noise=true , address=81.171.14.70:4034 , piece_id=BPXMDI54RTQVH5QKY6JCN5H3AKCPH62V76VBES4GQ6SLIZDIMMOA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.384672279 (104h35m1.981443307s) [noise=true , address=81.171.14.70:4047 , piece_id=DLCUNYJFEKIU3MWFXK24QGJ4GW2H5XE4Y2ARNL2W55EZUMSC4Z7Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.386337288 (104h35m1.98310832s) [noise=true , address=81.171.14.70:4025 , piece_id=R62A3AXM4OKUZFZJ2LB55UPB6VRGHBQLMXXVBWDTHUDWPGPIWEVQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.387583761 (104h35m1.984354793s) [noise=true , address=89.149.201.147:4009 , piece_id=XFSD3W6JWUXGXTHYE57SS5E5322JTRG7RKBI7MBF25GU7SEAKZJQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.388110220 (104h35m1.984881248s) [noise=true , address=81.171.14.70:4026 , piece_id=4CBQ4RZ44USXNYAGA3CIGNC3LBQXR675JVJX6CVTV3X7KVYPRHQA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.389451350 (104h35m1.986222362s) [noise=true , address=185.167.204.205:4012 , piece_id=7UFJIUKCHPKYZF42HTP6Q5D2FMHRSCRUQ7MH2MYJSEOZHZAJRZXA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.404093624 (104h35m2.00086464s) [noise=true , address=81.171.14.70:4048 , piece_id=4F2F4X4ZJ4FXUYQJQKB7YB3JZY53QNVRUSM3N4HQKGEYHMFYQB2A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.413111656 (104h35m2.009882687s) [noise=true , address=185.167.204.206:4071 , piece_id=ZQJLN55G37453GZS54ZDTPM7CEU5GH6LJR3BIKSJDC3C4QOKW5YQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.416183369 (104h35m2.012954404s) [noise=true , address=89.149.201.147:4014 , piece_id=WZOPRPDLFVOO6O2JYZZB75CLIAA4SVM7OEZXBX6WOJV3PUL2YSAQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.449375785 (104h35m2.046146815s) [noise=true , address=185.167.204.206:4039 , piece_id=DKKIZZMA3RAKMLUJTU6OANGBA3F6XWRD5ELVNXAGBIOZMFBVQWZQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.457716093 (104h35m2.054487108s) [noise=true , address=185.167.204.205:4024 , piece_id=3CKPLBO33LOMNYBF47K2KGQFPO53ZXLMKRKWPVSR4FKQKXKMZMCA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.491116161 (104h35m2.087887192s) [noise=true , address=185.167.204.205:4005 , piece_id=E7RO77MCAMFEAIE7ETPALCBV5S4X3DG5ZUD27ZE55RXYICIIXNEQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.507099744 (104h35m2.103870782s) [noise=true , address=185.167.204.205:4029 , piece_id=R33ONZELVLYL76YTJKOZASDJKFBACNNMC3PDUIITQRDGBWWZUNKA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.537139239 (104h35m2.133910269s) [noise=true , address=185.167.204.206:4067 , piece_id=RH2DWZ3FL7JX6EJ2WTJBPBSWDYVSHLYSLZCMMGRQXUDR6QXVZZ5A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.787100189 (104h35m2.383871224s) [noise=true , address=81.171.14.70:4033 , piece_id=ZFV477QXRX42OJIG4D5JLOYU3WERTD65PMHI5MFQLRTQJG3ENUTA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.962846444 (104h35m2.559617469s) [noise=true , address=89.149.201.147:4004 , piece_id=TMKNRIR6RWRW3TUSA2MQL4LCNTJ3LKEFIT6BAVRX2V54QHNF26FA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:08.299132133 (104h35m5.895903147s) [noise=true , address=185.167.204.205:4028 , piece_id=IQ4CMHEIUNHIUBQV2PVFQ6BURDMWU2S55VGFP5Y7HPFSDVBY6PYA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:08.481127955 (104h35m6.077898968s) [noise=true , address=185.167.204.206:4051 , piece_id=GO4OTIL5WFOS7V77FAT2NRFG2HRE2HXKMU2C3XEGDBGPQL5U5HUA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:10.314952727 (104h35m7.91172374s) [noise=true , address=185.167.204.206:4046 , piece_id=5WVC4R6VY25QUMGSOAORUK3WPGIADBXLKN7ZSLJ4GQEVKMSHZPKA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:36:03.322697363 (104h36m0.919468398s) [noise=true , address=185.167.204.206:4062 , piece_id=7NP2ZIMW6ZDXW3YHZSBO5PRTAZPLYLDWXDA4QPN5XH2C6KAGYLEA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:12BL6XLqsoUGm2FxrTgYTxLZakt2u2CAhZJG82oJ31eaE4vTtMr, address:185.167.204.206:4062): protocol: write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection; write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection
Mar 10 19:36:03.322771647 (104h36m0.919542676s) [noise=true , address=89.149.201.147:4007 , piece_id=AAJY5QBN55ENHTODH25WBVCFDT2SAQ2F27TKHFCIURCZSBMGOY6A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1Q7d4vj1KoWhDr6xd9gEfFrZ1sV3Yro2AFke2f3Ke1rXsGnvew, address:89.149.201.147:4007): protocol: read tcp 159.223.143.39:54460->89.149.201.147:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.322958851 (104h36m0.919729884s) [noise=true , address=185.167.204.205:4033 , piece_id=6JIW66HORXSEPSDWIUONOUZNA2UNXKBA4A6VZ7PBWW3IN2SCJSPQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:12Q73mXHLEfuX4VUbJ8PTjhMoXCrYSKjJbFxqjDdqF3U99N8FXe, address:185.167.204.205:4033): protocol: write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection; write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection
Mar 10 19:36:03.323134221 (104h36m0.919905244s) [noise=true , address=89.149.201.147:4018 , piece_id=V3KLJ75HJ3NRMBP2FIM33GC45QW4U5XMN2R4IAHSX32COGBJF47Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1Ku7o12bSmJ1YR3dGetGFcWqs1wJEh53Zkg3H7ELJrt4w1hkua, address:89.149.201.147:4018): protocol: read tcp 159.223.143.39:47122->89.149.201.147:4018: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.323461923 (104h36m0.920232956s) [noise=true , address=185.167.204.205:4022 , piece_id=F5UFUTAFPRBWMSU3QKV7ZSYQR6UUC7SBUDOC7AAYGDXTC5VYUBHA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1fT9qH1igPUNfrzpwa2VkBX7heh475sNqoN4Nsf9FYSwi1VmHj, address:185.167.204.205:4022): protocol: read tcp 159.223.143.39:51226->185.167.204.205:4022: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.323503666 (104h36m0.920274689s) [seg_pos={0 0}, upload=117052]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Mar 10 19:36:03.424787357 (104h36m1.021558384s) [upload=117052]: Done uploading segments...
Mar 10 19:36:03.424860036 (104h36m1.021631063s) [upload=117052]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 2.167849741s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [.X.......X..................................................] duration:
2.276275634s trace:
https://jaeger.dev.storj.io/trace/78c3ad63aac6b527
FAIL (2026-03-10 20:13:05 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
:03.719970769 (104h35m1.316741801s) [noise=true , address=185.167.204.205:4033 , piece_id=6JIW66HORXSEPSDWIUONOUZNA2UNXKBA4A6VZ7PBWW3IN2SCJSPQ, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.720006314 (104h35m1.316777347s) [noise=true , address=185.167.204.205:4024 , piece_id=3CKPLBO33LOMNYBF47K2KGQFPO53ZXLMKRKWPVSR4FKQKXKMZMCA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.721245847 (104h35m1.318016883s) [noise=true , address=185.167.204.206:4046 , piece_id=5WVC4R6VY25QUMGSOAORUK3WPGIADBXLKN7ZSLJ4GQEVKMSHZPKA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.722131967 (104h35m1.318902999s) [noise=true , address=89.149.201.147:4004 , piece_id=TMKNRIR6RWRW3TUSA2MQL4LCNTJ3LKEFIT6BAVRX2V54QHNF26FA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.722154312 (104h35m1.318925349s) [noise=true , address=185.167.204.205:4028 , piece_id=IQ4CMHEIUNHIUBQV2PVFQ6BURDMWU2S55VGFP5Y7HPFSDVBY6PYA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.723422869 (104h35m1.320193903s) [noise=true , address=81.171.14.70:4047 , piece_id=DLCUNYJFEKIU3MWFXK24QGJ4GW2H5XE4Y2ARNL2W55EZUMSC4Z7Q, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.724205203 (104h35m1.320976237s) [noise=true , address=185.167.204.205:4022 , piece_id=F5UFUTAFPRBWMSU3QKV7ZSYQR6UUC7SBUDOC7AAYGDXTC5VYUBHA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.727803141 (104h35m1.324574176s) [noise=true , address=185.167.204.206:4067 , piece_id=RH2DWZ3FL7JX6EJ2WTJBPBSWDYVSHLYSLZCMMGRQXUDR6QXVZZ5A, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:04.167055423 (104h35m1.763826466s) [upload=117052]: Next returned nil segment
Mar 10 19:35:04.167072399 (104h35m1.763843424s) [upload=117052]: Waiting for error group managing segments...
Mar 10 19:35:04.350286979 (104h35m1.947058004s) [noise=true , address=81.171.14.70:4035 , piece_id=RNJYIOH4ETYQBSZFYTV3FX73J3PNFR2KBYNZEDXCVBOYMUAN5FCA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.351883208 (104h35m1.948654233s) [noise=true , address=81.171.14.70:4043 , piece_id=LSOHRZMEV5TEQUKZIJPYHVP43CIZKMXBTRVKIZIQNSSFNMAA5FLQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.356149085 (104h35m1.952920098s) [noise=true , address=185.167.204.205:4036 , piece_id=BBGCJ7U7PQDNB5NZTJWHUWXR6TFYF6D7LOR2UUBNV647YQX5JD2Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.369094269 (104h35m1.965865391s) [noise=true , address=89.149.201.147:4011 , piece_id=YHPLXK3ADD6VMYDWQ55PLROBCY2IOLOR2ZWEZ22FRRA4TXKSSWIQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.382982597 (104h35m1.979753611s) [noise=true , address=81.171.14.70:4034 , piece_id=BPXMDI54RTQVH5QKY6JCN5H3AKCPH62V76VBES4GQ6SLIZDIMMOA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.384672279 (104h35m1.981443307s) [noise=true , address=81.171.14.70:4047 , piece_id=DLCUNYJFEKIU3MWFXK24QGJ4GW2H5XE4Y2ARNL2W55EZUMSC4Z7Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.386337288 (104h35m1.98310832s) [noise=true , address=81.171.14.70:4025 , piece_id=R62A3AXM4OKUZFZJ2LB55UPB6VRGHBQLMXXVBWDTHUDWPGPIWEVQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.387583761 (104h35m1.984354793s) [noise=true , address=89.149.201.147:4009 , piece_id=XFSD3W6JWUXGXTHYE57SS5E5322JTRG7RKBI7MBF25GU7SEAKZJQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.388110220 (104h35m1.984881248s) [noise=true , address=81.171.14.70:4026 , piece_id=4CBQ4RZ44USXNYAGA3CIGNC3LBQXR675JVJX6CVTV3X7KVYPRHQA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.389451350 (104h35m1.986222362s) [noise=true , address=185.167.204.205:4012 , piece_id=7UFJIUKCHPKYZF42HTP6Q5D2FMHRSCRUQ7MH2MYJSEOZHZAJRZXA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.404093624 (104h35m2.00086464s) [noise=true , address=81.171.14.70:4048 , piece_id=4F2F4X4ZJ4FXUYQJQKB7YB3JZY53QNVRUSM3N4HQKGEYHMFYQB2A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.413111656 (104h35m2.009882687s) [noise=true , address=185.167.204.206:4071 , piece_id=ZQJLN55G37453GZS54ZDTPM7CEU5GH6LJR3BIKSJDC3C4QOKW5YQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.416183369 (104h35m2.012954404s) [noise=true , address=89.149.201.147:4014 , piece_id=WZOPRPDLFVOO6O2JYZZB75CLIAA4SVM7OEZXBX6WOJV3PUL2YSAQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.449375785 (104h35m2.046146815s) [noise=true , address=185.167.204.206:4039 , piece_id=DKKIZZMA3RAKMLUJTU6OANGBA3F6XWRD5ELVNXAGBIOZMFBVQWZQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.457716093 (104h35m2.054487108s) [noise=true , address=185.167.204.205:4024 , piece_id=3CKPLBO33LOMNYBF47K2KGQFPO53ZXLMKRKWPVSR4FKQKXKMZMCA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.491116161 (104h35m2.087887192s) [noise=true , address=185.167.204.205:4005 , piece_id=E7RO77MCAMFEAIE7ETPALCBV5S4X3DG5ZUD27ZE55RXYICIIXNEQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.507099744 (104h35m2.103870782s) [noise=true , address=185.167.204.205:4029 , piece_id=R33ONZELVLYL76YTJKOZASDJKFBACNNMC3PDUIITQRDGBWWZUNKA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.537139239 (104h35m2.133910269s) [noise=true , address=185.167.204.206:4067 , piece_id=RH2DWZ3FL7JX6EJ2WTJBPBSWDYVSHLYSLZCMMGRQXUDR6QXVZZ5A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.787100189 (104h35m2.383871224s) [noise=true , address=81.171.14.70:4033 , piece_id=ZFV477QXRX42OJIG4D5JLOYU3WERTD65PMHI5MFQLRTQJG3ENUTA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.962846444 (104h35m2.559617469s) [noise=true , address=89.149.201.147:4004 , piece_id=TMKNRIR6RWRW3TUSA2MQL4LCNTJ3LKEFIT6BAVRX2V54QHNF26FA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:08.299132133 (104h35m5.895903147s) [noise=true , address=185.167.204.205:4028 , piece_id=IQ4CMHEIUNHIUBQV2PVFQ6BURDMWU2S55VGFP5Y7HPFSDVBY6PYA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:08.481127955 (104h35m6.077898968s) [noise=true , address=185.167.204.206:4051 , piece_id=GO4OTIL5WFOS7V77FAT2NRFG2HRE2HXKMU2C3XEGDBGPQL5U5HUA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:10.314952727 (104h35m7.91172374s) [noise=true , address=185.167.204.206:4046 , piece_id=5WVC4R6VY25QUMGSOAORUK3WPGIADBXLKN7ZSLJ4GQEVKMSHZPKA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:36:03.322697363 (104h36m0.919468398s) [noise=true , address=185.167.204.206:4062 , piece_id=7NP2ZIMW6ZDXW3YHZSBO5PRTAZPLYLDWXDA4QPN5XH2C6KAGYLEA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:12BL6XLqsoUGm2FxrTgYTxLZakt2u2CAhZJG82oJ31eaE4vTtMr, address:185.167.204.206:4062): protocol: write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection; write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection
Mar 10 19:36:03.322771647 (104h36m0.919542676s) [noise=true , address=89.149.201.147:4007 , piece_id=AAJY5QBN55ENHTODH25WBVCFDT2SAQ2F27TKHFCIURCZSBMGOY6A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1Q7d4vj1KoWhDr6xd9gEfFrZ1sV3Yro2AFke2f3Ke1rXsGnvew, address:89.149.201.147:4007): protocol: read tcp 159.223.143.39:54460->89.149.201.147:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.322958851 (104h36m0.919729884s) [noise=true , address=185.167.204.205:4033 , piece_id=6JIW66HORXSEPSDWIUONOUZNA2UNXKBA4A6VZ7PBWW3IN2SCJSPQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:12Q73mXHLEfuX4VUbJ8PTjhMoXCrYSKjJbFxqjDdqF3U99N8FXe, address:185.167.204.205:4033): protocol: write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection; write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection
Mar 10 19:36:03.323134221 (104h36m0.919905244s) [noise=true , address=89.149.201.147:4018 , piece_id=V3KLJ75HJ3NRMBP2FIM33GC45QW4U5XMN2R4IAHSX32COGBJF47Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1Ku7o12bSmJ1YR3dGetGFcWqs1wJEh53Zkg3H7ELJrt4w1hkua, address:89.149.201.147:4018): protocol: read tcp 159.223.143.39:47122->89.149.201.147:4018: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.323461923 (104h36m0.920232956s) [noise=true , address=185.167.204.205:4022 , piece_id=F5UFUTAFPRBWMSU3QKV7ZSYQR6UUC7SBUDOC7AAYGDXTC5VYUBHA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1fT9qH1igPUNfrzpwa2VkBX7heh475sNqoN4Nsf9FYSwi1VmHj, address:185.167.204.205:4022): protocol: read tcp 159.223.143.39:51226->185.167.204.205:4022: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.323503666 (104h36m0.920274689s) [seg_pos={0 0}, upload=117052]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Mar 10 19:36:03.424787357 (104h36m1.021558384s) [upload=117052]: Done uploading segments...
Mar 10 19:36:03.424860036 (104h36m1.021631063s) [upload=117052]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 2.276309169s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [..X.......X.................................................] duration:
3.719604438s trace:
https://jaeger.dev.storj.io/trace/78c3ad63aac6b527
FAIL (2026-03-10 20:11:59 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
:03.719970769 (104h35m1.316741801s) [noise=true , address=185.167.204.205:4033 , piece_id=6JIW66HORXSEPSDWIUONOUZNA2UNXKBA4A6VZ7PBWW3IN2SCJSPQ, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.720006314 (104h35m1.316777347s) [noise=true , address=185.167.204.205:4024 , piece_id=3CKPLBO33LOMNYBF47K2KGQFPO53ZXLMKRKWPVSR4FKQKXKMZMCA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.721245847 (104h35m1.318016883s) [noise=true , address=185.167.204.206:4046 , piece_id=5WVC4R6VY25QUMGSOAORUK3WPGIADBXLKN7ZSLJ4GQEVKMSHZPKA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.722131967 (104h35m1.318902999s) [noise=true , address=89.149.201.147:4004 , piece_id=TMKNRIR6RWRW3TUSA2MQL4LCNTJ3LKEFIT6BAVRX2V54QHNF26FA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.722154312 (104h35m1.318925349s) [noise=true , address=185.167.204.205:4028 , piece_id=IQ4CMHEIUNHIUBQV2PVFQ6BURDMWU2S55VGFP5Y7HPFSDVBY6PYA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.723422869 (104h35m1.320193903s) [noise=true , address=81.171.14.70:4047 , piece_id=DLCUNYJFEKIU3MWFXK24QGJ4GW2H5XE4Y2ARNL2W55EZUMSC4Z7Q, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.724205203 (104h35m1.320976237s) [noise=true , address=185.167.204.205:4022 , piece_id=F5UFUTAFPRBWMSU3QKV7ZSYQR6UUC7SBUDOC7AAYGDXTC5VYUBHA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.727803141 (104h35m1.324574176s) [noise=true , address=185.167.204.206:4067 , piece_id=RH2DWZ3FL7JX6EJ2WTJBPBSWDYVSHLYSLZCMMGRQXUDR6QXVZZ5A, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:04.167055423 (104h35m1.763826466s) [upload=117052]: Next returned nil segment
Mar 10 19:35:04.167072399 (104h35m1.763843424s) [upload=117052]: Waiting for error group managing segments...
Mar 10 19:35:04.350286979 (104h35m1.947058004s) [noise=true , address=81.171.14.70:4035 , piece_id=RNJYIOH4ETYQBSZFYTV3FX73J3PNFR2KBYNZEDXCVBOYMUAN5FCA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.351883208 (104h35m1.948654233s) [noise=true , address=81.171.14.70:4043 , piece_id=LSOHRZMEV5TEQUKZIJPYHVP43CIZKMXBTRVKIZIQNSSFNMAA5FLQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.356149085 (104h35m1.952920098s) [noise=true , address=185.167.204.205:4036 , piece_id=BBGCJ7U7PQDNB5NZTJWHUWXR6TFYF6D7LOR2UUBNV647YQX5JD2Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.369094269 (104h35m1.965865391s) [noise=true , address=89.149.201.147:4011 , piece_id=YHPLXK3ADD6VMYDWQ55PLROBCY2IOLOR2ZWEZ22FRRA4TXKSSWIQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.382982597 (104h35m1.979753611s) [noise=true , address=81.171.14.70:4034 , piece_id=BPXMDI54RTQVH5QKY6JCN5H3AKCPH62V76VBES4GQ6SLIZDIMMOA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.384672279 (104h35m1.981443307s) [noise=true , address=81.171.14.70:4047 , piece_id=DLCUNYJFEKIU3MWFXK24QGJ4GW2H5XE4Y2ARNL2W55EZUMSC4Z7Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.386337288 (104h35m1.98310832s) [noise=true , address=81.171.14.70:4025 , piece_id=R62A3AXM4OKUZFZJ2LB55UPB6VRGHBQLMXXVBWDTHUDWPGPIWEVQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.387583761 (104h35m1.984354793s) [noise=true , address=89.149.201.147:4009 , piece_id=XFSD3W6JWUXGXTHYE57SS5E5322JTRG7RKBI7MBF25GU7SEAKZJQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.388110220 (104h35m1.984881248s) [noise=true , address=81.171.14.70:4026 , piece_id=4CBQ4RZ44USXNYAGA3CIGNC3LBQXR675JVJX6CVTV3X7KVYPRHQA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.389451350 (104h35m1.986222362s) [noise=true , address=185.167.204.205:4012 , piece_id=7UFJIUKCHPKYZF42HTP6Q5D2FMHRSCRUQ7MH2MYJSEOZHZAJRZXA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.404093624 (104h35m2.00086464s) [noise=true , address=81.171.14.70:4048 , piece_id=4F2F4X4ZJ4FXUYQJQKB7YB3JZY53QNVRUSM3N4HQKGEYHMFYQB2A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.413111656 (104h35m2.009882687s) [noise=true , address=185.167.204.206:4071 , piece_id=ZQJLN55G37453GZS54ZDTPM7CEU5GH6LJR3BIKSJDC3C4QOKW5YQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.416183369 (104h35m2.012954404s) [noise=true , address=89.149.201.147:4014 , piece_id=WZOPRPDLFVOO6O2JYZZB75CLIAA4SVM7OEZXBX6WOJV3PUL2YSAQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.449375785 (104h35m2.046146815s) [noise=true , address=185.167.204.206:4039 , piece_id=DKKIZZMA3RAKMLUJTU6OANGBA3F6XWRD5ELVNXAGBIOZMFBVQWZQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.457716093 (104h35m2.054487108s) [noise=true , address=185.167.204.205:4024 , piece_id=3CKPLBO33LOMNYBF47K2KGQFPO53ZXLMKRKWPVSR4FKQKXKMZMCA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.491116161 (104h35m2.087887192s) [noise=true , address=185.167.204.205:4005 , piece_id=E7RO77MCAMFEAIE7ETPALCBV5S4X3DG5ZUD27ZE55RXYICIIXNEQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.507099744 (104h35m2.103870782s) [noise=true , address=185.167.204.205:4029 , piece_id=R33ONZELVLYL76YTJKOZASDJKFBACNNMC3PDUIITQRDGBWWZUNKA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.537139239 (104h35m2.133910269s) [noise=true , address=185.167.204.206:4067 , piece_id=RH2DWZ3FL7JX6EJ2WTJBPBSWDYVSHLYSLZCMMGRQXUDR6QXVZZ5A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.787100189 (104h35m2.383871224s) [noise=true , address=81.171.14.70:4033 , piece_id=ZFV477QXRX42OJIG4D5JLOYU3WERTD65PMHI5MFQLRTQJG3ENUTA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.962846444 (104h35m2.559617469s) [noise=true , address=89.149.201.147:4004 , piece_id=TMKNRIR6RWRW3TUSA2MQL4LCNTJ3LKEFIT6BAVRX2V54QHNF26FA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:08.299132133 (104h35m5.895903147s) [noise=true , address=185.167.204.205:4028 , piece_id=IQ4CMHEIUNHIUBQV2PVFQ6BURDMWU2S55VGFP5Y7HPFSDVBY6PYA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:08.481127955 (104h35m6.077898968s) [noise=true , address=185.167.204.206:4051 , piece_id=GO4OTIL5WFOS7V77FAT2NRFG2HRE2HXKMU2C3XEGDBGPQL5U5HUA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:10.314952727 (104h35m7.91172374s) [noise=true , address=185.167.204.206:4046 , piece_id=5WVC4R6VY25QUMGSOAORUK3WPGIADBXLKN7ZSLJ4GQEVKMSHZPKA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:36:03.322697363 (104h36m0.919468398s) [noise=true , address=185.167.204.206:4062 , piece_id=7NP2ZIMW6ZDXW3YHZSBO5PRTAZPLYLDWXDA4QPN5XH2C6KAGYLEA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:12BL6XLqsoUGm2FxrTgYTxLZakt2u2CAhZJG82oJ31eaE4vTtMr, address:185.167.204.206:4062): protocol: write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection; write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection
Mar 10 19:36:03.322771647 (104h36m0.919542676s) [noise=true , address=89.149.201.147:4007 , piece_id=AAJY5QBN55ENHTODH25WBVCFDT2SAQ2F27TKHFCIURCZSBMGOY6A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1Q7d4vj1KoWhDr6xd9gEfFrZ1sV3Yro2AFke2f3Ke1rXsGnvew, address:89.149.201.147:4007): protocol: read tcp 159.223.143.39:54460->89.149.201.147:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.322958851 (104h36m0.919729884s) [noise=true , address=185.167.204.205:4033 , piece_id=6JIW66HORXSEPSDWIUONOUZNA2UNXKBA4A6VZ7PBWW3IN2SCJSPQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:12Q73mXHLEfuX4VUbJ8PTjhMoXCrYSKjJbFxqjDdqF3U99N8FXe, address:185.167.204.205:4033): protocol: write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection; write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection
Mar 10 19:36:03.323134221 (104h36m0.919905244s) [noise=true , address=89.149.201.147:4018 , piece_id=V3KLJ75HJ3NRMBP2FIM33GC45QW4U5XMN2R4IAHSX32COGBJF47Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1Ku7o12bSmJ1YR3dGetGFcWqs1wJEh53Zkg3H7ELJrt4w1hkua, address:89.149.201.147:4018): protocol: read tcp 159.223.143.39:47122->89.149.201.147:4018: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.323461923 (104h36m0.920232956s) [noise=true , address=185.167.204.205:4022 , piece_id=F5UFUTAFPRBWMSU3QKV7ZSYQR6UUC7SBUDOC7AAYGDXTC5VYUBHA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1fT9qH1igPUNfrzpwa2VkBX7heh475sNqoN4Nsf9FYSwi1VmHj, address:185.167.204.205:4022): protocol: read tcp 159.223.143.39:51226->185.167.204.205:4022: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.323503666 (104h36m0.920274689s) [seg_pos={0 0}, upload=117052]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Mar 10 19:36:03.424787357 (104h36m1.021558384s) [upload=117052]: Done uploading segments...
Mar 10 19:36:03.424860036 (104h36m1.021631063s) [upload=117052]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 3.719630342s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [...X.......X................................................] duration:
3.183440484s trace:
https://jaeger.dev.storj.io/trace/78c3ad63aac6b527
FAIL (2026-03-10 20:11:40 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
:03.719970769 (104h35m1.316741801s) [noise=true , address=185.167.204.205:4033 , piece_id=6JIW66HORXSEPSDWIUONOUZNA2UNXKBA4A6VZ7PBWW3IN2SCJSPQ, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.720006314 (104h35m1.316777347s) [noise=true , address=185.167.204.205:4024 , piece_id=3CKPLBO33LOMNYBF47K2KGQFPO53ZXLMKRKWPVSR4FKQKXKMZMCA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.721245847 (104h35m1.318016883s) [noise=true , address=185.167.204.206:4046 , piece_id=5WVC4R6VY25QUMGSOAORUK3WPGIADBXLKN7ZSLJ4GQEVKMSHZPKA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.722131967 (104h35m1.318902999s) [noise=true , address=89.149.201.147:4004 , piece_id=TMKNRIR6RWRW3TUSA2MQL4LCNTJ3LKEFIT6BAVRX2V54QHNF26FA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.722154312 (104h35m1.318925349s) [noise=true , address=185.167.204.205:4028 , piece_id=IQ4CMHEIUNHIUBQV2PVFQ6BURDMWU2S55VGFP5Y7HPFSDVBY6PYA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.723422869 (104h35m1.320193903s) [noise=true , address=81.171.14.70:4047 , piece_id=DLCUNYJFEKIU3MWFXK24QGJ4GW2H5XE4Y2ARNL2W55EZUMSC4Z7Q, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.724205203 (104h35m1.320976237s) [noise=true , address=185.167.204.205:4022 , piece_id=F5UFUTAFPRBWMSU3QKV7ZSYQR6UUC7SBUDOC7AAYGDXTC5VYUBHA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.727803141 (104h35m1.324574176s) [noise=true , address=185.167.204.206:4067 , piece_id=RH2DWZ3FL7JX6EJ2WTJBPBSWDYVSHLYSLZCMMGRQXUDR6QXVZZ5A, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:04.167055423 (104h35m1.763826466s) [upload=117052]: Next returned nil segment
Mar 10 19:35:04.167072399 (104h35m1.763843424s) [upload=117052]: Waiting for error group managing segments...
Mar 10 19:35:04.350286979 (104h35m1.947058004s) [noise=true , address=81.171.14.70:4035 , piece_id=RNJYIOH4ETYQBSZFYTV3FX73J3PNFR2KBYNZEDXCVBOYMUAN5FCA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.351883208 (104h35m1.948654233s) [noise=true , address=81.171.14.70:4043 , piece_id=LSOHRZMEV5TEQUKZIJPYHVP43CIZKMXBTRVKIZIQNSSFNMAA5FLQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.356149085 (104h35m1.952920098s) [noise=true , address=185.167.204.205:4036 , piece_id=BBGCJ7U7PQDNB5NZTJWHUWXR6TFYF6D7LOR2UUBNV647YQX5JD2Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.369094269 (104h35m1.965865391s) [noise=true , address=89.149.201.147:4011 , piece_id=YHPLXK3ADD6VMYDWQ55PLROBCY2IOLOR2ZWEZ22FRRA4TXKSSWIQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.382982597 (104h35m1.979753611s) [noise=true , address=81.171.14.70:4034 , piece_id=BPXMDI54RTQVH5QKY6JCN5H3AKCPH62V76VBES4GQ6SLIZDIMMOA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.384672279 (104h35m1.981443307s) [noise=true , address=81.171.14.70:4047 , piece_id=DLCUNYJFEKIU3MWFXK24QGJ4GW2H5XE4Y2ARNL2W55EZUMSC4Z7Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.386337288 (104h35m1.98310832s) [noise=true , address=81.171.14.70:4025 , piece_id=R62A3AXM4OKUZFZJ2LB55UPB6VRGHBQLMXXVBWDTHUDWPGPIWEVQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.387583761 (104h35m1.984354793s) [noise=true , address=89.149.201.147:4009 , piece_id=XFSD3W6JWUXGXTHYE57SS5E5322JTRG7RKBI7MBF25GU7SEAKZJQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.388110220 (104h35m1.984881248s) [noise=true , address=81.171.14.70:4026 , piece_id=4CBQ4RZ44USXNYAGA3CIGNC3LBQXR675JVJX6CVTV3X7KVYPRHQA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.389451350 (104h35m1.986222362s) [noise=true , address=185.167.204.205:4012 , piece_id=7UFJIUKCHPKYZF42HTP6Q5D2FMHRSCRUQ7MH2MYJSEOZHZAJRZXA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.404093624 (104h35m2.00086464s) [noise=true , address=81.171.14.70:4048 , piece_id=4F2F4X4ZJ4FXUYQJQKB7YB3JZY53QNVRUSM3N4HQKGEYHMFYQB2A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.413111656 (104h35m2.009882687s) [noise=true , address=185.167.204.206:4071 , piece_id=ZQJLN55G37453GZS54ZDTPM7CEU5GH6LJR3BIKSJDC3C4QOKW5YQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.416183369 (104h35m2.012954404s) [noise=true , address=89.149.201.147:4014 , piece_id=WZOPRPDLFVOO6O2JYZZB75CLIAA4SVM7OEZXBX6WOJV3PUL2YSAQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.449375785 (104h35m2.046146815s) [noise=true , address=185.167.204.206:4039 , piece_id=DKKIZZMA3RAKMLUJTU6OANGBA3F6XWRD5ELVNXAGBIOZMFBVQWZQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.457716093 (104h35m2.054487108s) [noise=true , address=185.167.204.205:4024 , piece_id=3CKPLBO33LOMNYBF47K2KGQFPO53ZXLMKRKWPVSR4FKQKXKMZMCA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.491116161 (104h35m2.087887192s) [noise=true , address=185.167.204.205:4005 , piece_id=E7RO77MCAMFEAIE7ETPALCBV5S4X3DG5ZUD27ZE55RXYICIIXNEQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.507099744 (104h35m2.103870782s) [noise=true , address=185.167.204.205:4029 , piece_id=R33ONZELVLYL76YTJKOZASDJKFBACNNMC3PDUIITQRDGBWWZUNKA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.537139239 (104h35m2.133910269s) [noise=true , address=185.167.204.206:4067 , piece_id=RH2DWZ3FL7JX6EJ2WTJBPBSWDYVSHLYSLZCMMGRQXUDR6QXVZZ5A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.787100189 (104h35m2.383871224s) [noise=true , address=81.171.14.70:4033 , piece_id=ZFV477QXRX42OJIG4D5JLOYU3WERTD65PMHI5MFQLRTQJG3ENUTA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.962846444 (104h35m2.559617469s) [noise=true , address=89.149.201.147:4004 , piece_id=TMKNRIR6RWRW3TUSA2MQL4LCNTJ3LKEFIT6BAVRX2V54QHNF26FA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:08.299132133 (104h35m5.895903147s) [noise=true , address=185.167.204.205:4028 , piece_id=IQ4CMHEIUNHIUBQV2PVFQ6BURDMWU2S55VGFP5Y7HPFSDVBY6PYA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:08.481127955 (104h35m6.077898968s) [noise=true , address=185.167.204.206:4051 , piece_id=GO4OTIL5WFOS7V77FAT2NRFG2HRE2HXKMU2C3XEGDBGPQL5U5HUA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:10.314952727 (104h35m7.91172374s) [noise=true , address=185.167.204.206:4046 , piece_id=5WVC4R6VY25QUMGSOAORUK3WPGIADBXLKN7ZSLJ4GQEVKMSHZPKA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:36:03.322697363 (104h36m0.919468398s) [noise=true , address=185.167.204.206:4062 , piece_id=7NP2ZIMW6ZDXW3YHZSBO5PRTAZPLYLDWXDA4QPN5XH2C6KAGYLEA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:12BL6XLqsoUGm2FxrTgYTxLZakt2u2CAhZJG82oJ31eaE4vTtMr, address:185.167.204.206:4062): protocol: write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection; write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection
Mar 10 19:36:03.322771647 (104h36m0.919542676s) [noise=true , address=89.149.201.147:4007 , piece_id=AAJY5QBN55ENHTODH25WBVCFDT2SAQ2F27TKHFCIURCZSBMGOY6A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1Q7d4vj1KoWhDr6xd9gEfFrZ1sV3Yro2AFke2f3Ke1rXsGnvew, address:89.149.201.147:4007): protocol: read tcp 159.223.143.39:54460->89.149.201.147:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.322958851 (104h36m0.919729884s) [noise=true , address=185.167.204.205:4033 , piece_id=6JIW66HORXSEPSDWIUONOUZNA2UNXKBA4A6VZ7PBWW3IN2SCJSPQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:12Q73mXHLEfuX4VUbJ8PTjhMoXCrYSKjJbFxqjDdqF3U99N8FXe, address:185.167.204.205:4033): protocol: write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection; write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection
Mar 10 19:36:03.323134221 (104h36m0.919905244s) [noise=true , address=89.149.201.147:4018 , piece_id=V3KLJ75HJ3NRMBP2FIM33GC45QW4U5XMN2R4IAHSX32COGBJF47Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1Ku7o12bSmJ1YR3dGetGFcWqs1wJEh53Zkg3H7ELJrt4w1hkua, address:89.149.201.147:4018): protocol: read tcp 159.223.143.39:47122->89.149.201.147:4018: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.323461923 (104h36m0.920232956s) [noise=true , address=185.167.204.205:4022 , piece_id=F5UFUTAFPRBWMSU3QKV7ZSYQR6UUC7SBUDOC7AAYGDXTC5VYUBHA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1fT9qH1igPUNfrzpwa2VkBX7heh475sNqoN4Nsf9FYSwi1VmHj, address:185.167.204.205:4022): protocol: read tcp 159.223.143.39:51226->185.167.204.205:4022: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.323503666 (104h36m0.920274689s) [seg_pos={0 0}, upload=117052]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Mar 10 19:36:03.424787357 (104h36m1.021558384s) [upload=117052]: Done uploading segments...
Mar 10 19:36:03.424860036 (104h36m1.021631063s) [upload=117052]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 3.183459285s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [....X.......X...............................................] duration:
2.180778443s trace:
https://jaeger.dev.storj.io/trace/78c3ad63aac6b527
FAIL (2026-03-10 20:10:56 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
:03.719970769 (104h35m1.316741801s) [noise=true , address=185.167.204.205:4033 , piece_id=6JIW66HORXSEPSDWIUONOUZNA2UNXKBA4A6VZ7PBWW3IN2SCJSPQ, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.720006314 (104h35m1.316777347s) [noise=true , address=185.167.204.205:4024 , piece_id=3CKPLBO33LOMNYBF47K2KGQFPO53ZXLMKRKWPVSR4FKQKXKMZMCA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.721245847 (104h35m1.318016883s) [noise=true , address=185.167.204.206:4046 , piece_id=5WVC4R6VY25QUMGSOAORUK3WPGIADBXLKN7ZSLJ4GQEVKMSHZPKA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.722131967 (104h35m1.318902999s) [noise=true , address=89.149.201.147:4004 , piece_id=TMKNRIR6RWRW3TUSA2MQL4LCNTJ3LKEFIT6BAVRX2V54QHNF26FA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.722154312 (104h35m1.318925349s) [noise=true , address=185.167.204.205:4028 , piece_id=IQ4CMHEIUNHIUBQV2PVFQ6BURDMWU2S55VGFP5Y7HPFSDVBY6PYA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.723422869 (104h35m1.320193903s) [noise=true , address=81.171.14.70:4047 , piece_id=DLCUNYJFEKIU3MWFXK24QGJ4GW2H5XE4Y2ARNL2W55EZUMSC4Z7Q, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.724205203 (104h35m1.320976237s) [noise=true , address=185.167.204.205:4022 , piece_id=F5UFUTAFPRBWMSU3QKV7ZSYQR6UUC7SBUDOC7AAYGDXTC5VYUBHA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.727803141 (104h35m1.324574176s) [noise=true , address=185.167.204.206:4067 , piece_id=RH2DWZ3FL7JX6EJ2WTJBPBSWDYVSHLYSLZCMMGRQXUDR6QXVZZ5A, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:04.167055423 (104h35m1.763826466s) [upload=117052]: Next returned nil segment
Mar 10 19:35:04.167072399 (104h35m1.763843424s) [upload=117052]: Waiting for error group managing segments...
Mar 10 19:35:04.350286979 (104h35m1.947058004s) [noise=true , address=81.171.14.70:4035 , piece_id=RNJYIOH4ETYQBSZFYTV3FX73J3PNFR2KBYNZEDXCVBOYMUAN5FCA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.351883208 (104h35m1.948654233s) [noise=true , address=81.171.14.70:4043 , piece_id=LSOHRZMEV5TEQUKZIJPYHVP43CIZKMXBTRVKIZIQNSSFNMAA5FLQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.356149085 (104h35m1.952920098s) [noise=true , address=185.167.204.205:4036 , piece_id=BBGCJ7U7PQDNB5NZTJWHUWXR6TFYF6D7LOR2UUBNV647YQX5JD2Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.369094269 (104h35m1.965865391s) [noise=true , address=89.149.201.147:4011 , piece_id=YHPLXK3ADD6VMYDWQ55PLROBCY2IOLOR2ZWEZ22FRRA4TXKSSWIQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.382982597 (104h35m1.979753611s) [noise=true , address=81.171.14.70:4034 , piece_id=BPXMDI54RTQVH5QKY6JCN5H3AKCPH62V76VBES4GQ6SLIZDIMMOA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.384672279 (104h35m1.981443307s) [noise=true , address=81.171.14.70:4047 , piece_id=DLCUNYJFEKIU3MWFXK24QGJ4GW2H5XE4Y2ARNL2W55EZUMSC4Z7Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.386337288 (104h35m1.98310832s) [noise=true , address=81.171.14.70:4025 , piece_id=R62A3AXM4OKUZFZJ2LB55UPB6VRGHBQLMXXVBWDTHUDWPGPIWEVQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.387583761 (104h35m1.984354793s) [noise=true , address=89.149.201.147:4009 , piece_id=XFSD3W6JWUXGXTHYE57SS5E5322JTRG7RKBI7MBF25GU7SEAKZJQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.388110220 (104h35m1.984881248s) [noise=true , address=81.171.14.70:4026 , piece_id=4CBQ4RZ44USXNYAGA3CIGNC3LBQXR675JVJX6CVTV3X7KVYPRHQA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.389451350 (104h35m1.986222362s) [noise=true , address=185.167.204.205:4012 , piece_id=7UFJIUKCHPKYZF42HTP6Q5D2FMHRSCRUQ7MH2MYJSEOZHZAJRZXA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.404093624 (104h35m2.00086464s) [noise=true , address=81.171.14.70:4048 , piece_id=4F2F4X4ZJ4FXUYQJQKB7YB3JZY53QNVRUSM3N4HQKGEYHMFYQB2A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.413111656 (104h35m2.009882687s) [noise=true , address=185.167.204.206:4071 , piece_id=ZQJLN55G37453GZS54ZDTPM7CEU5GH6LJR3BIKSJDC3C4QOKW5YQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.416183369 (104h35m2.012954404s) [noise=true , address=89.149.201.147:4014 , piece_id=WZOPRPDLFVOO6O2JYZZB75CLIAA4SVM7OEZXBX6WOJV3PUL2YSAQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.449375785 (104h35m2.046146815s) [noise=true , address=185.167.204.206:4039 , piece_id=DKKIZZMA3RAKMLUJTU6OANGBA3F6XWRD5ELVNXAGBIOZMFBVQWZQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.457716093 (104h35m2.054487108s) [noise=true , address=185.167.204.205:4024 , piece_id=3CKPLBO33LOMNYBF47K2KGQFPO53ZXLMKRKWPVSR4FKQKXKMZMCA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.491116161 (104h35m2.087887192s) [noise=true , address=185.167.204.205:4005 , piece_id=E7RO77MCAMFEAIE7ETPALCBV5S4X3DG5ZUD27ZE55RXYICIIXNEQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.507099744 (104h35m2.103870782s) [noise=true , address=185.167.204.205:4029 , piece_id=R33ONZELVLYL76YTJKOZASDJKFBACNNMC3PDUIITQRDGBWWZUNKA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.537139239 (104h35m2.133910269s) [noise=true , address=185.167.204.206:4067 , piece_id=RH2DWZ3FL7JX6EJ2WTJBPBSWDYVSHLYSLZCMMGRQXUDR6QXVZZ5A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.787100189 (104h35m2.383871224s) [noise=true , address=81.171.14.70:4033 , piece_id=ZFV477QXRX42OJIG4D5JLOYU3WERTD65PMHI5MFQLRTQJG3ENUTA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.962846444 (104h35m2.559617469s) [noise=true , address=89.149.201.147:4004 , piece_id=TMKNRIR6RWRW3TUSA2MQL4LCNTJ3LKEFIT6BAVRX2V54QHNF26FA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:08.299132133 (104h35m5.895903147s) [noise=true , address=185.167.204.205:4028 , piece_id=IQ4CMHEIUNHIUBQV2PVFQ6BURDMWU2S55VGFP5Y7HPFSDVBY6PYA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:08.481127955 (104h35m6.077898968s) [noise=true , address=185.167.204.206:4051 , piece_id=GO4OTIL5WFOS7V77FAT2NRFG2HRE2HXKMU2C3XEGDBGPQL5U5HUA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:10.314952727 (104h35m7.91172374s) [noise=true , address=185.167.204.206:4046 , piece_id=5WVC4R6VY25QUMGSOAORUK3WPGIADBXLKN7ZSLJ4GQEVKMSHZPKA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:36:03.322697363 (104h36m0.919468398s) [noise=true , address=185.167.204.206:4062 , piece_id=7NP2ZIMW6ZDXW3YHZSBO5PRTAZPLYLDWXDA4QPN5XH2C6KAGYLEA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:12BL6XLqsoUGm2FxrTgYTxLZakt2u2CAhZJG82oJ31eaE4vTtMr, address:185.167.204.206:4062): protocol: write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection; write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection
Mar 10 19:36:03.322771647 (104h36m0.919542676s) [noise=true , address=89.149.201.147:4007 , piece_id=AAJY5QBN55ENHTODH25WBVCFDT2SAQ2F27TKHFCIURCZSBMGOY6A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1Q7d4vj1KoWhDr6xd9gEfFrZ1sV3Yro2AFke2f3Ke1rXsGnvew, address:89.149.201.147:4007): protocol: read tcp 159.223.143.39:54460->89.149.201.147:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.322958851 (104h36m0.919729884s) [noise=true , address=185.167.204.205:4033 , piece_id=6JIW66HORXSEPSDWIUONOUZNA2UNXKBA4A6VZ7PBWW3IN2SCJSPQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:12Q73mXHLEfuX4VUbJ8PTjhMoXCrYSKjJbFxqjDdqF3U99N8FXe, address:185.167.204.205:4033): protocol: write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection; write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection
Mar 10 19:36:03.323134221 (104h36m0.919905244s) [noise=true , address=89.149.201.147:4018 , piece_id=V3KLJ75HJ3NRMBP2FIM33GC45QW4U5XMN2R4IAHSX32COGBJF47Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1Ku7o12bSmJ1YR3dGetGFcWqs1wJEh53Zkg3H7ELJrt4w1hkua, address:89.149.201.147:4018): protocol: read tcp 159.223.143.39:47122->89.149.201.147:4018: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.323461923 (104h36m0.920232956s) [noise=true , address=185.167.204.205:4022 , piece_id=F5UFUTAFPRBWMSU3QKV7ZSYQR6UUC7SBUDOC7AAYGDXTC5VYUBHA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1fT9qH1igPUNfrzpwa2VkBX7heh475sNqoN4Nsf9FYSwi1VmHj, address:185.167.204.205:4022): protocol: read tcp 159.223.143.39:51226->185.167.204.205:4022: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.323503666 (104h36m0.920274689s) [seg_pos={0 0}, upload=117052]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Mar 10 19:36:03.424787357 (104h36m1.021558384s) [upload=117052]: Done uploading segments...
Mar 10 19:36:03.424860036 (104h36m1.021631063s) [upload=117052]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 2.180803661s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [.....X.......X..............................................] duration:
2.708084992s trace:
https://jaeger.dev.storj.io/trace/78c3ad63aac6b527
FAIL (2026-03-10 20:10:04 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
:03.719970769 (104h35m1.316741801s) [noise=true , address=185.167.204.205:4033 , piece_id=6JIW66HORXSEPSDWIUONOUZNA2UNXKBA4A6VZ7PBWW3IN2SCJSPQ, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.720006314 (104h35m1.316777347s) [noise=true , address=185.167.204.205:4024 , piece_id=3CKPLBO33LOMNYBF47K2KGQFPO53ZXLMKRKWPVSR4FKQKXKMZMCA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.721245847 (104h35m1.318016883s) [noise=true , address=185.167.204.206:4046 , piece_id=5WVC4R6VY25QUMGSOAORUK3WPGIADBXLKN7ZSLJ4GQEVKMSHZPKA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.722131967 (104h35m1.318902999s) [noise=true , address=89.149.201.147:4004 , piece_id=TMKNRIR6RWRW3TUSA2MQL4LCNTJ3LKEFIT6BAVRX2V54QHNF26FA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.722154312 (104h35m1.318925349s) [noise=true , address=185.167.204.205:4028 , piece_id=IQ4CMHEIUNHIUBQV2PVFQ6BURDMWU2S55VGFP5Y7HPFSDVBY6PYA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.723422869 (104h35m1.320193903s) [noise=true , address=81.171.14.70:4047 , piece_id=DLCUNYJFEKIU3MWFXK24QGJ4GW2H5XE4Y2ARNL2W55EZUMSC4Z7Q, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.724205203 (104h35m1.320976237s) [noise=true , address=185.167.204.205:4022 , piece_id=F5UFUTAFPRBWMSU3QKV7ZSYQR6UUC7SBUDOC7AAYGDXTC5VYUBHA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.727803141 (104h35m1.324574176s) [noise=true , address=185.167.204.206:4067 , piece_id=RH2DWZ3FL7JX6EJ2WTJBPBSWDYVSHLYSLZCMMGRQXUDR6QXVZZ5A, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:04.167055423 (104h35m1.763826466s) [upload=117052]: Next returned nil segment
Mar 10 19:35:04.167072399 (104h35m1.763843424s) [upload=117052]: Waiting for error group managing segments...
Mar 10 19:35:04.350286979 (104h35m1.947058004s) [noise=true , address=81.171.14.70:4035 , piece_id=RNJYIOH4ETYQBSZFYTV3FX73J3PNFR2KBYNZEDXCVBOYMUAN5FCA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.351883208 (104h35m1.948654233s) [noise=true , address=81.171.14.70:4043 , piece_id=LSOHRZMEV5TEQUKZIJPYHVP43CIZKMXBTRVKIZIQNSSFNMAA5FLQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.356149085 (104h35m1.952920098s) [noise=true , address=185.167.204.205:4036 , piece_id=BBGCJ7U7PQDNB5NZTJWHUWXR6TFYF6D7LOR2UUBNV647YQX5JD2Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.369094269 (104h35m1.965865391s) [noise=true , address=89.149.201.147:4011 , piece_id=YHPLXK3ADD6VMYDWQ55PLROBCY2IOLOR2ZWEZ22FRRA4TXKSSWIQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.382982597 (104h35m1.979753611s) [noise=true , address=81.171.14.70:4034 , piece_id=BPXMDI54RTQVH5QKY6JCN5H3AKCPH62V76VBES4GQ6SLIZDIMMOA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.384672279 (104h35m1.981443307s) [noise=true , address=81.171.14.70:4047 , piece_id=DLCUNYJFEKIU3MWFXK24QGJ4GW2H5XE4Y2ARNL2W55EZUMSC4Z7Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.386337288 (104h35m1.98310832s) [noise=true , address=81.171.14.70:4025 , piece_id=R62A3AXM4OKUZFZJ2LB55UPB6VRGHBQLMXXVBWDTHUDWPGPIWEVQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.387583761 (104h35m1.984354793s) [noise=true , address=89.149.201.147:4009 , piece_id=XFSD3W6JWUXGXTHYE57SS5E5322JTRG7RKBI7MBF25GU7SEAKZJQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.388110220 (104h35m1.984881248s) [noise=true , address=81.171.14.70:4026 , piece_id=4CBQ4RZ44USXNYAGA3CIGNC3LBQXR675JVJX6CVTV3X7KVYPRHQA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.389451350 (104h35m1.986222362s) [noise=true , address=185.167.204.205:4012 , piece_id=7UFJIUKCHPKYZF42HTP6Q5D2FMHRSCRUQ7MH2MYJSEOZHZAJRZXA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.404093624 (104h35m2.00086464s) [noise=true , address=81.171.14.70:4048 , piece_id=4F2F4X4ZJ4FXUYQJQKB7YB3JZY53QNVRUSM3N4HQKGEYHMFYQB2A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.413111656 (104h35m2.009882687s) [noise=true , address=185.167.204.206:4071 , piece_id=ZQJLN55G37453GZS54ZDTPM7CEU5GH6LJR3BIKSJDC3C4QOKW5YQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.416183369 (104h35m2.012954404s) [noise=true , address=89.149.201.147:4014 , piece_id=WZOPRPDLFVOO6O2JYZZB75CLIAA4SVM7OEZXBX6WOJV3PUL2YSAQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.449375785 (104h35m2.046146815s) [noise=true , address=185.167.204.206:4039 , piece_id=DKKIZZMA3RAKMLUJTU6OANGBA3F6XWRD5ELVNXAGBIOZMFBVQWZQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.457716093 (104h35m2.054487108s) [noise=true , address=185.167.204.205:4024 , piece_id=3CKPLBO33LOMNYBF47K2KGQFPO53ZXLMKRKWPVSR4FKQKXKMZMCA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.491116161 (104h35m2.087887192s) [noise=true , address=185.167.204.205:4005 , piece_id=E7RO77MCAMFEAIE7ETPALCBV5S4X3DG5ZUD27ZE55RXYICIIXNEQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.507099744 (104h35m2.103870782s) [noise=true , address=185.167.204.205:4029 , piece_id=R33ONZELVLYL76YTJKOZASDJKFBACNNMC3PDUIITQRDGBWWZUNKA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.537139239 (104h35m2.133910269s) [noise=true , address=185.167.204.206:4067 , piece_id=RH2DWZ3FL7JX6EJ2WTJBPBSWDYVSHLYSLZCMMGRQXUDR6QXVZZ5A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.787100189 (104h35m2.383871224s) [noise=true , address=81.171.14.70:4033 , piece_id=ZFV477QXRX42OJIG4D5JLOYU3WERTD65PMHI5MFQLRTQJG3ENUTA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.962846444 (104h35m2.559617469s) [noise=true , address=89.149.201.147:4004 , piece_id=TMKNRIR6RWRW3TUSA2MQL4LCNTJ3LKEFIT6BAVRX2V54QHNF26FA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:08.299132133 (104h35m5.895903147s) [noise=true , address=185.167.204.205:4028 , piece_id=IQ4CMHEIUNHIUBQV2PVFQ6BURDMWU2S55VGFP5Y7HPFSDVBY6PYA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:08.481127955 (104h35m6.077898968s) [noise=true , address=185.167.204.206:4051 , piece_id=GO4OTIL5WFOS7V77FAT2NRFG2HRE2HXKMU2C3XEGDBGPQL5U5HUA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:10.314952727 (104h35m7.91172374s) [noise=true , address=185.167.204.206:4046 , piece_id=5WVC4R6VY25QUMGSOAORUK3WPGIADBXLKN7ZSLJ4GQEVKMSHZPKA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:36:03.322697363 (104h36m0.919468398s) [noise=true , address=185.167.204.206:4062 , piece_id=7NP2ZIMW6ZDXW3YHZSBO5PRTAZPLYLDWXDA4QPN5XH2C6KAGYLEA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:12BL6XLqsoUGm2FxrTgYTxLZakt2u2CAhZJG82oJ31eaE4vTtMr, address:185.167.204.206:4062): protocol: write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection; write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection
Mar 10 19:36:03.322771647 (104h36m0.919542676s) [noise=true , address=89.149.201.147:4007 , piece_id=AAJY5QBN55ENHTODH25WBVCFDT2SAQ2F27TKHFCIURCZSBMGOY6A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1Q7d4vj1KoWhDr6xd9gEfFrZ1sV3Yro2AFke2f3Ke1rXsGnvew, address:89.149.201.147:4007): protocol: read tcp 159.223.143.39:54460->89.149.201.147:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.322958851 (104h36m0.919729884s) [noise=true , address=185.167.204.205:4033 , piece_id=6JIW66HORXSEPSDWIUONOUZNA2UNXKBA4A6VZ7PBWW3IN2SCJSPQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:12Q73mXHLEfuX4VUbJ8PTjhMoXCrYSKjJbFxqjDdqF3U99N8FXe, address:185.167.204.205:4033): protocol: write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection; write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection
Mar 10 19:36:03.323134221 (104h36m0.919905244s) [noise=true , address=89.149.201.147:4018 , piece_id=V3KLJ75HJ3NRMBP2FIM33GC45QW4U5XMN2R4IAHSX32COGBJF47Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1Ku7o12bSmJ1YR3dGetGFcWqs1wJEh53Zkg3H7ELJrt4w1hkua, address:89.149.201.147:4018): protocol: read tcp 159.223.143.39:47122->89.149.201.147:4018: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.323461923 (104h36m0.920232956s) [noise=true , address=185.167.204.205:4022 , piece_id=F5UFUTAFPRBWMSU3QKV7ZSYQR6UUC7SBUDOC7AAYGDXTC5VYUBHA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1fT9qH1igPUNfrzpwa2VkBX7heh475sNqoN4Nsf9FYSwi1VmHj, address:185.167.204.205:4022): protocol: read tcp 159.223.143.39:51226->185.167.204.205:4022: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.323503666 (104h36m0.920274689s) [seg_pos={0 0}, upload=117052]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Mar 10 19:36:03.424787357 (104h36m1.021558384s) [upload=117052]: Done uploading segments...
Mar 10 19:36:03.424860036 (104h36m1.021631063s) [upload=117052]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 2.708132288s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [......X.......X.............................................] duration:
2.145317073s trace:
https://jaeger.dev.storj.io/trace/78c3ad63aac6b527
FAIL (2026-03-10 20:09:09 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
:03.719970769 (104h35m1.316741801s) [noise=true , address=185.167.204.205:4033 , piece_id=6JIW66HORXSEPSDWIUONOUZNA2UNXKBA4A6VZ7PBWW3IN2SCJSPQ, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.720006314 (104h35m1.316777347s) [noise=true , address=185.167.204.205:4024 , piece_id=3CKPLBO33LOMNYBF47K2KGQFPO53ZXLMKRKWPVSR4FKQKXKMZMCA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.721245847 (104h35m1.318016883s) [noise=true , address=185.167.204.206:4046 , piece_id=5WVC4R6VY25QUMGSOAORUK3WPGIADBXLKN7ZSLJ4GQEVKMSHZPKA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.722131967 (104h35m1.318902999s) [noise=true , address=89.149.201.147:4004 , piece_id=TMKNRIR6RWRW3TUSA2MQL4LCNTJ3LKEFIT6BAVRX2V54QHNF26FA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.722154312 (104h35m1.318925349s) [noise=true , address=185.167.204.205:4028 , piece_id=IQ4CMHEIUNHIUBQV2PVFQ6BURDMWU2S55VGFP5Y7HPFSDVBY6PYA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.723422869 (104h35m1.320193903s) [noise=true , address=81.171.14.70:4047 , piece_id=DLCUNYJFEKIU3MWFXK24QGJ4GW2H5XE4Y2ARNL2W55EZUMSC4Z7Q, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.724205203 (104h35m1.320976237s) [noise=true , address=185.167.204.205:4022 , piece_id=F5UFUTAFPRBWMSU3QKV7ZSYQR6UUC7SBUDOC7AAYGDXTC5VYUBHA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.727803141 (104h35m1.324574176s) [noise=true , address=185.167.204.206:4067 , piece_id=RH2DWZ3FL7JX6EJ2WTJBPBSWDYVSHLYSLZCMMGRQXUDR6QXVZZ5A, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:04.167055423 (104h35m1.763826466s) [upload=117052]: Next returned nil segment
Mar 10 19:35:04.167072399 (104h35m1.763843424s) [upload=117052]: Waiting for error group managing segments...
Mar 10 19:35:04.350286979 (104h35m1.947058004s) [noise=true , address=81.171.14.70:4035 , piece_id=RNJYIOH4ETYQBSZFYTV3FX73J3PNFR2KBYNZEDXCVBOYMUAN5FCA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.351883208 (104h35m1.948654233s) [noise=true , address=81.171.14.70:4043 , piece_id=LSOHRZMEV5TEQUKZIJPYHVP43CIZKMXBTRVKIZIQNSSFNMAA5FLQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.356149085 (104h35m1.952920098s) [noise=true , address=185.167.204.205:4036 , piece_id=BBGCJ7U7PQDNB5NZTJWHUWXR6TFYF6D7LOR2UUBNV647YQX5JD2Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.369094269 (104h35m1.965865391s) [noise=true , address=89.149.201.147:4011 , piece_id=YHPLXK3ADD6VMYDWQ55PLROBCY2IOLOR2ZWEZ22FRRA4TXKSSWIQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.382982597 (104h35m1.979753611s) [noise=true , address=81.171.14.70:4034 , piece_id=BPXMDI54RTQVH5QKY6JCN5H3AKCPH62V76VBES4GQ6SLIZDIMMOA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.384672279 (104h35m1.981443307s) [noise=true , address=81.171.14.70:4047 , piece_id=DLCUNYJFEKIU3MWFXK24QGJ4GW2H5XE4Y2ARNL2W55EZUMSC4Z7Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.386337288 (104h35m1.98310832s) [noise=true , address=81.171.14.70:4025 , piece_id=R62A3AXM4OKUZFZJ2LB55UPB6VRGHBQLMXXVBWDTHUDWPGPIWEVQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.387583761 (104h35m1.984354793s) [noise=true , address=89.149.201.147:4009 , piece_id=XFSD3W6JWUXGXTHYE57SS5E5322JTRG7RKBI7MBF25GU7SEAKZJQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.388110220 (104h35m1.984881248s) [noise=true , address=81.171.14.70:4026 , piece_id=4CBQ4RZ44USXNYAGA3CIGNC3LBQXR675JVJX6CVTV3X7KVYPRHQA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.389451350 (104h35m1.986222362s) [noise=true , address=185.167.204.205:4012 , piece_id=7UFJIUKCHPKYZF42HTP6Q5D2FMHRSCRUQ7MH2MYJSEOZHZAJRZXA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.404093624 (104h35m2.00086464s) [noise=true , address=81.171.14.70:4048 , piece_id=4F2F4X4ZJ4FXUYQJQKB7YB3JZY53QNVRUSM3N4HQKGEYHMFYQB2A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.413111656 (104h35m2.009882687s) [noise=true , address=185.167.204.206:4071 , piece_id=ZQJLN55G37453GZS54ZDTPM7CEU5GH6LJR3BIKSJDC3C4QOKW5YQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.416183369 (104h35m2.012954404s) [noise=true , address=89.149.201.147:4014 , piece_id=WZOPRPDLFVOO6O2JYZZB75CLIAA4SVM7OEZXBX6WOJV3PUL2YSAQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.449375785 (104h35m2.046146815s) [noise=true , address=185.167.204.206:4039 , piece_id=DKKIZZMA3RAKMLUJTU6OANGBA3F6XWRD5ELVNXAGBIOZMFBVQWZQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.457716093 (104h35m2.054487108s) [noise=true , address=185.167.204.205:4024 , piece_id=3CKPLBO33LOMNYBF47K2KGQFPO53ZXLMKRKWPVSR4FKQKXKMZMCA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.491116161 (104h35m2.087887192s) [noise=true , address=185.167.204.205:4005 , piece_id=E7RO77MCAMFEAIE7ETPALCBV5S4X3DG5ZUD27ZE55RXYICIIXNEQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.507099744 (104h35m2.103870782s) [noise=true , address=185.167.204.205:4029 , piece_id=R33ONZELVLYL76YTJKOZASDJKFBACNNMC3PDUIITQRDGBWWZUNKA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.537139239 (104h35m2.133910269s) [noise=true , address=185.167.204.206:4067 , piece_id=RH2DWZ3FL7JX6EJ2WTJBPBSWDYVSHLYSLZCMMGRQXUDR6QXVZZ5A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.787100189 (104h35m2.383871224s) [noise=true , address=81.171.14.70:4033 , piece_id=ZFV477QXRX42OJIG4D5JLOYU3WERTD65PMHI5MFQLRTQJG3ENUTA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.962846444 (104h35m2.559617469s) [noise=true , address=89.149.201.147:4004 , piece_id=TMKNRIR6RWRW3TUSA2MQL4LCNTJ3LKEFIT6BAVRX2V54QHNF26FA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:08.299132133 (104h35m5.895903147s) [noise=true , address=185.167.204.205:4028 , piece_id=IQ4CMHEIUNHIUBQV2PVFQ6BURDMWU2S55VGFP5Y7HPFSDVBY6PYA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:08.481127955 (104h35m6.077898968s) [noise=true , address=185.167.204.206:4051 , piece_id=GO4OTIL5WFOS7V77FAT2NRFG2HRE2HXKMU2C3XEGDBGPQL5U5HUA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:10.314952727 (104h35m7.91172374s) [noise=true , address=185.167.204.206:4046 , piece_id=5WVC4R6VY25QUMGSOAORUK3WPGIADBXLKN7ZSLJ4GQEVKMSHZPKA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:36:03.322697363 (104h36m0.919468398s) [noise=true , address=185.167.204.206:4062 , piece_id=7NP2ZIMW6ZDXW3YHZSBO5PRTAZPLYLDWXDA4QPN5XH2C6KAGYLEA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:12BL6XLqsoUGm2FxrTgYTxLZakt2u2CAhZJG82oJ31eaE4vTtMr, address:185.167.204.206:4062): protocol: write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection; write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection
Mar 10 19:36:03.322771647 (104h36m0.919542676s) [noise=true , address=89.149.201.147:4007 , piece_id=AAJY5QBN55ENHTODH25WBVCFDT2SAQ2F27TKHFCIURCZSBMGOY6A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1Q7d4vj1KoWhDr6xd9gEfFrZ1sV3Yro2AFke2f3Ke1rXsGnvew, address:89.149.201.147:4007): protocol: read tcp 159.223.143.39:54460->89.149.201.147:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.322958851 (104h36m0.919729884s) [noise=true , address=185.167.204.205:4033 , piece_id=6JIW66HORXSEPSDWIUONOUZNA2UNXKBA4A6VZ7PBWW3IN2SCJSPQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:12Q73mXHLEfuX4VUbJ8PTjhMoXCrYSKjJbFxqjDdqF3U99N8FXe, address:185.167.204.205:4033): protocol: write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection; write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection
Mar 10 19:36:03.323134221 (104h36m0.919905244s) [noise=true , address=89.149.201.147:4018 , piece_id=V3KLJ75HJ3NRMBP2FIM33GC45QW4U5XMN2R4IAHSX32COGBJF47Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1Ku7o12bSmJ1YR3dGetGFcWqs1wJEh53Zkg3H7ELJrt4w1hkua, address:89.149.201.147:4018): protocol: read tcp 159.223.143.39:47122->89.149.201.147:4018: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.323461923 (104h36m0.920232956s) [noise=true , address=185.167.204.205:4022 , piece_id=F5UFUTAFPRBWMSU3QKV7ZSYQR6UUC7SBUDOC7AAYGDXTC5VYUBHA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1fT9qH1igPUNfrzpwa2VkBX7heh475sNqoN4Nsf9FYSwi1VmHj, address:185.167.204.205:4022): protocol: read tcp 159.223.143.39:51226->185.167.204.205:4022: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.323503666 (104h36m0.920274689s) [seg_pos={0 0}, upload=117052]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Mar 10 19:36:03.424787357 (104h36m1.021558384s) [upload=117052]: Done uploading segments...
Mar 10 19:36:03.424860036 (104h36m1.021631063s) [upload=117052]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 2.145353696s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [.......X.......X............................................] duration:
2.606409887s trace:
https://jaeger.dev.storj.io/trace/78c3ad63aac6b527
FAIL (2026-03-10 20:08:42 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
:03.719970769 (104h35m1.316741801s) [noise=true , address=185.167.204.205:4033 , piece_id=6JIW66HORXSEPSDWIUONOUZNA2UNXKBA4A6VZ7PBWW3IN2SCJSPQ, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.720006314 (104h35m1.316777347s) [noise=true , address=185.167.204.205:4024 , piece_id=3CKPLBO33LOMNYBF47K2KGQFPO53ZXLMKRKWPVSR4FKQKXKMZMCA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.721245847 (104h35m1.318016883s) [noise=true , address=185.167.204.206:4046 , piece_id=5WVC4R6VY25QUMGSOAORUK3WPGIADBXLKN7ZSLJ4GQEVKMSHZPKA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.722131967 (104h35m1.318902999s) [noise=true , address=89.149.201.147:4004 , piece_id=TMKNRIR6RWRW3TUSA2MQL4LCNTJ3LKEFIT6BAVRX2V54QHNF26FA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.722154312 (104h35m1.318925349s) [noise=true , address=185.167.204.205:4028 , piece_id=IQ4CMHEIUNHIUBQV2PVFQ6BURDMWU2S55VGFP5Y7HPFSDVBY6PYA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.723422869 (104h35m1.320193903s) [noise=true , address=81.171.14.70:4047 , piece_id=DLCUNYJFEKIU3MWFXK24QGJ4GW2H5XE4Y2ARNL2W55EZUMSC4Z7Q, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.724205203 (104h35m1.320976237s) [noise=true , address=185.167.204.205:4022 , piece_id=F5UFUTAFPRBWMSU3QKV7ZSYQR6UUC7SBUDOC7AAYGDXTC5VYUBHA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.727803141 (104h35m1.324574176s) [noise=true , address=185.167.204.206:4067 , piece_id=RH2DWZ3FL7JX6EJ2WTJBPBSWDYVSHLYSLZCMMGRQXUDR6QXVZZ5A, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:04.167055423 (104h35m1.763826466s) [upload=117052]: Next returned nil segment
Mar 10 19:35:04.167072399 (104h35m1.763843424s) [upload=117052]: Waiting for error group managing segments...
Mar 10 19:35:04.350286979 (104h35m1.947058004s) [noise=true , address=81.171.14.70:4035 , piece_id=RNJYIOH4ETYQBSZFYTV3FX73J3PNFR2KBYNZEDXCVBOYMUAN5FCA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.351883208 (104h35m1.948654233s) [noise=true , address=81.171.14.70:4043 , piece_id=LSOHRZMEV5TEQUKZIJPYHVP43CIZKMXBTRVKIZIQNSSFNMAA5FLQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.356149085 (104h35m1.952920098s) [noise=true , address=185.167.204.205:4036 , piece_id=BBGCJ7U7PQDNB5NZTJWHUWXR6TFYF6D7LOR2UUBNV647YQX5JD2Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.369094269 (104h35m1.965865391s) [noise=true , address=89.149.201.147:4011 , piece_id=YHPLXK3ADD6VMYDWQ55PLROBCY2IOLOR2ZWEZ22FRRA4TXKSSWIQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.382982597 (104h35m1.979753611s) [noise=true , address=81.171.14.70:4034 , piece_id=BPXMDI54RTQVH5QKY6JCN5H3AKCPH62V76VBES4GQ6SLIZDIMMOA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.384672279 (104h35m1.981443307s) [noise=true , address=81.171.14.70:4047 , piece_id=DLCUNYJFEKIU3MWFXK24QGJ4GW2H5XE4Y2ARNL2W55EZUMSC4Z7Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.386337288 (104h35m1.98310832s) [noise=true , address=81.171.14.70:4025 , piece_id=R62A3AXM4OKUZFZJ2LB55UPB6VRGHBQLMXXVBWDTHUDWPGPIWEVQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.387583761 (104h35m1.984354793s) [noise=true , address=89.149.201.147:4009 , piece_id=XFSD3W6JWUXGXTHYE57SS5E5322JTRG7RKBI7MBF25GU7SEAKZJQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.388110220 (104h35m1.984881248s) [noise=true , address=81.171.14.70:4026 , piece_id=4CBQ4RZ44USXNYAGA3CIGNC3LBQXR675JVJX6CVTV3X7KVYPRHQA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.389451350 (104h35m1.986222362s) [noise=true , address=185.167.204.205:4012 , piece_id=7UFJIUKCHPKYZF42HTP6Q5D2FMHRSCRUQ7MH2MYJSEOZHZAJRZXA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.404093624 (104h35m2.00086464s) [noise=true , address=81.171.14.70:4048 , piece_id=4F2F4X4ZJ4FXUYQJQKB7YB3JZY53QNVRUSM3N4HQKGEYHMFYQB2A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.413111656 (104h35m2.009882687s) [noise=true , address=185.167.204.206:4071 , piece_id=ZQJLN55G37453GZS54ZDTPM7CEU5GH6LJR3BIKSJDC3C4QOKW5YQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.416183369 (104h35m2.012954404s) [noise=true , address=89.149.201.147:4014 , piece_id=WZOPRPDLFVOO6O2JYZZB75CLIAA4SVM7OEZXBX6WOJV3PUL2YSAQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.449375785 (104h35m2.046146815s) [noise=true , address=185.167.204.206:4039 , piece_id=DKKIZZMA3RAKMLUJTU6OANGBA3F6XWRD5ELVNXAGBIOZMFBVQWZQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.457716093 (104h35m2.054487108s) [noise=true , address=185.167.204.205:4024 , piece_id=3CKPLBO33LOMNYBF47K2KGQFPO53ZXLMKRKWPVSR4FKQKXKMZMCA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.491116161 (104h35m2.087887192s) [noise=true , address=185.167.204.205:4005 , piece_id=E7RO77MCAMFEAIE7ETPALCBV5S4X3DG5ZUD27ZE55RXYICIIXNEQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.507099744 (104h35m2.103870782s) [noise=true , address=185.167.204.205:4029 , piece_id=R33ONZELVLYL76YTJKOZASDJKFBACNNMC3PDUIITQRDGBWWZUNKA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.537139239 (104h35m2.133910269s) [noise=true , address=185.167.204.206:4067 , piece_id=RH2DWZ3FL7JX6EJ2WTJBPBSWDYVSHLYSLZCMMGRQXUDR6QXVZZ5A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.787100189 (104h35m2.383871224s) [noise=true , address=81.171.14.70:4033 , piece_id=ZFV477QXRX42OJIG4D5JLOYU3WERTD65PMHI5MFQLRTQJG3ENUTA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.962846444 (104h35m2.559617469s) [noise=true , address=89.149.201.147:4004 , piece_id=TMKNRIR6RWRW3TUSA2MQL4LCNTJ3LKEFIT6BAVRX2V54QHNF26FA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:08.299132133 (104h35m5.895903147s) [noise=true , address=185.167.204.205:4028 , piece_id=IQ4CMHEIUNHIUBQV2PVFQ6BURDMWU2S55VGFP5Y7HPFSDVBY6PYA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:08.481127955 (104h35m6.077898968s) [noise=true , address=185.167.204.206:4051 , piece_id=GO4OTIL5WFOS7V77FAT2NRFG2HRE2HXKMU2C3XEGDBGPQL5U5HUA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:10.314952727 (104h35m7.91172374s) [noise=true , address=185.167.204.206:4046 , piece_id=5WVC4R6VY25QUMGSOAORUK3WPGIADBXLKN7ZSLJ4GQEVKMSHZPKA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:36:03.322697363 (104h36m0.919468398s) [noise=true , address=185.167.204.206:4062 , piece_id=7NP2ZIMW6ZDXW3YHZSBO5PRTAZPLYLDWXDA4QPN5XH2C6KAGYLEA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:12BL6XLqsoUGm2FxrTgYTxLZakt2u2CAhZJG82oJ31eaE4vTtMr, address:185.167.204.206:4062): protocol: write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection; write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection
Mar 10 19:36:03.322771647 (104h36m0.919542676s) [noise=true , address=89.149.201.147:4007 , piece_id=AAJY5QBN55ENHTODH25WBVCFDT2SAQ2F27TKHFCIURCZSBMGOY6A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1Q7d4vj1KoWhDr6xd9gEfFrZ1sV3Yro2AFke2f3Ke1rXsGnvew, address:89.149.201.147:4007): protocol: read tcp 159.223.143.39:54460->89.149.201.147:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.322958851 (104h36m0.919729884s) [noise=true , address=185.167.204.205:4033 , piece_id=6JIW66HORXSEPSDWIUONOUZNA2UNXKBA4A6VZ7PBWW3IN2SCJSPQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:12Q73mXHLEfuX4VUbJ8PTjhMoXCrYSKjJbFxqjDdqF3U99N8FXe, address:185.167.204.205:4033): protocol: write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection; write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection
Mar 10 19:36:03.323134221 (104h36m0.919905244s) [noise=true , address=89.149.201.147:4018 , piece_id=V3KLJ75HJ3NRMBP2FIM33GC45QW4U5XMN2R4IAHSX32COGBJF47Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1Ku7o12bSmJ1YR3dGetGFcWqs1wJEh53Zkg3H7ELJrt4w1hkua, address:89.149.201.147:4018): protocol: read tcp 159.223.143.39:47122->89.149.201.147:4018: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.323461923 (104h36m0.920232956s) [noise=true , address=185.167.204.205:4022 , piece_id=F5UFUTAFPRBWMSU3QKV7ZSYQR6UUC7SBUDOC7AAYGDXTC5VYUBHA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1fT9qH1igPUNfrzpwa2VkBX7heh475sNqoN4Nsf9FYSwi1VmHj, address:185.167.204.205:4022): protocol: read tcp 159.223.143.39:51226->185.167.204.205:4022: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.323503666 (104h36m0.920274689s) [seg_pos={0 0}, upload=117052]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Mar 10 19:36:03.424787357 (104h36m1.021558384s) [upload=117052]: Done uploading segments...
Mar 10 19:36:03.424860036 (104h36m1.021631063s) [upload=117052]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 2.60644335s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [........X.......X...........................................] duration:
2.149787455s trace:
https://jaeger.dev.storj.io/trace/78c3ad63aac6b527
FAIL (2026-03-10 20:07:39 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
:03.719970769 (104h35m1.316741801s) [noise=true , address=185.167.204.205:4033 , piece_id=6JIW66HORXSEPSDWIUONOUZNA2UNXKBA4A6VZ7PBWW3IN2SCJSPQ, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.720006314 (104h35m1.316777347s) [noise=true , address=185.167.204.205:4024 , piece_id=3CKPLBO33LOMNYBF47K2KGQFPO53ZXLMKRKWPVSR4FKQKXKMZMCA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.721245847 (104h35m1.318016883s) [noise=true , address=185.167.204.206:4046 , piece_id=5WVC4R6VY25QUMGSOAORUK3WPGIADBXLKN7ZSLJ4GQEVKMSHZPKA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.722131967 (104h35m1.318902999s) [noise=true , address=89.149.201.147:4004 , piece_id=TMKNRIR6RWRW3TUSA2MQL4LCNTJ3LKEFIT6BAVRX2V54QHNF26FA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.722154312 (104h35m1.318925349s) [noise=true , address=185.167.204.205:4028 , piece_id=IQ4CMHEIUNHIUBQV2PVFQ6BURDMWU2S55VGFP5Y7HPFSDVBY6PYA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.723422869 (104h35m1.320193903s) [noise=true , address=81.171.14.70:4047 , piece_id=DLCUNYJFEKIU3MWFXK24QGJ4GW2H5XE4Y2ARNL2W55EZUMSC4Z7Q, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.724205203 (104h35m1.320976237s) [noise=true , address=185.167.204.205:4022 , piece_id=F5UFUTAFPRBWMSU3QKV7ZSYQR6UUC7SBUDOC7AAYGDXTC5VYUBHA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.727803141 (104h35m1.324574176s) [noise=true , address=185.167.204.206:4067 , piece_id=RH2DWZ3FL7JX6EJ2WTJBPBSWDYVSHLYSLZCMMGRQXUDR6QXVZZ5A, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:04.167055423 (104h35m1.763826466s) [upload=117052]: Next returned nil segment
Mar 10 19:35:04.167072399 (104h35m1.763843424s) [upload=117052]: Waiting for error group managing segments...
Mar 10 19:35:04.350286979 (104h35m1.947058004s) [noise=true , address=81.171.14.70:4035 , piece_id=RNJYIOH4ETYQBSZFYTV3FX73J3PNFR2KBYNZEDXCVBOYMUAN5FCA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.351883208 (104h35m1.948654233s) [noise=true , address=81.171.14.70:4043 , piece_id=LSOHRZMEV5TEQUKZIJPYHVP43CIZKMXBTRVKIZIQNSSFNMAA5FLQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.356149085 (104h35m1.952920098s) [noise=true , address=185.167.204.205:4036 , piece_id=BBGCJ7U7PQDNB5NZTJWHUWXR6TFYF6D7LOR2UUBNV647YQX5JD2Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.369094269 (104h35m1.965865391s) [noise=true , address=89.149.201.147:4011 , piece_id=YHPLXK3ADD6VMYDWQ55PLROBCY2IOLOR2ZWEZ22FRRA4TXKSSWIQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.382982597 (104h35m1.979753611s) [noise=true , address=81.171.14.70:4034 , piece_id=BPXMDI54RTQVH5QKY6JCN5H3AKCPH62V76VBES4GQ6SLIZDIMMOA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.384672279 (104h35m1.981443307s) [noise=true , address=81.171.14.70:4047 , piece_id=DLCUNYJFEKIU3MWFXK24QGJ4GW2H5XE4Y2ARNL2W55EZUMSC4Z7Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.386337288 (104h35m1.98310832s) [noise=true , address=81.171.14.70:4025 , piece_id=R62A3AXM4OKUZFZJ2LB55UPB6VRGHBQLMXXVBWDTHUDWPGPIWEVQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.387583761 (104h35m1.984354793s) [noise=true , address=89.149.201.147:4009 , piece_id=XFSD3W6JWUXGXTHYE57SS5E5322JTRG7RKBI7MBF25GU7SEAKZJQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.388110220 (104h35m1.984881248s) [noise=true , address=81.171.14.70:4026 , piece_id=4CBQ4RZ44USXNYAGA3CIGNC3LBQXR675JVJX6CVTV3X7KVYPRHQA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.389451350 (104h35m1.986222362s) [noise=true , address=185.167.204.205:4012 , piece_id=7UFJIUKCHPKYZF42HTP6Q5D2FMHRSCRUQ7MH2MYJSEOZHZAJRZXA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.404093624 (104h35m2.00086464s) [noise=true , address=81.171.14.70:4048 , piece_id=4F2F4X4ZJ4FXUYQJQKB7YB3JZY53QNVRUSM3N4HQKGEYHMFYQB2A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.413111656 (104h35m2.009882687s) [noise=true , address=185.167.204.206:4071 , piece_id=ZQJLN55G37453GZS54ZDTPM7CEU5GH6LJR3BIKSJDC3C4QOKW5YQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.416183369 (104h35m2.012954404s) [noise=true , address=89.149.201.147:4014 , piece_id=WZOPRPDLFVOO6O2JYZZB75CLIAA4SVM7OEZXBX6WOJV3PUL2YSAQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.449375785 (104h35m2.046146815s) [noise=true , address=185.167.204.206:4039 , piece_id=DKKIZZMA3RAKMLUJTU6OANGBA3F6XWRD5ELVNXAGBIOZMFBVQWZQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.457716093 (104h35m2.054487108s) [noise=true , address=185.167.204.205:4024 , piece_id=3CKPLBO33LOMNYBF47K2KGQFPO53ZXLMKRKWPVSR4FKQKXKMZMCA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.491116161 (104h35m2.087887192s) [noise=true , address=185.167.204.205:4005 , piece_id=E7RO77MCAMFEAIE7ETPALCBV5S4X3DG5ZUD27ZE55RXYICIIXNEQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.507099744 (104h35m2.103870782s) [noise=true , address=185.167.204.205:4029 , piece_id=R33ONZELVLYL76YTJKOZASDJKFBACNNMC3PDUIITQRDGBWWZUNKA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.537139239 (104h35m2.133910269s) [noise=true , address=185.167.204.206:4067 , piece_id=RH2DWZ3FL7JX6EJ2WTJBPBSWDYVSHLYSLZCMMGRQXUDR6QXVZZ5A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.787100189 (104h35m2.383871224s) [noise=true , address=81.171.14.70:4033 , piece_id=ZFV477QXRX42OJIG4D5JLOYU3WERTD65PMHI5MFQLRTQJG3ENUTA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.962846444 (104h35m2.559617469s) [noise=true , address=89.149.201.147:4004 , piece_id=TMKNRIR6RWRW3TUSA2MQL4LCNTJ3LKEFIT6BAVRX2V54QHNF26FA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:08.299132133 (104h35m5.895903147s) [noise=true , address=185.167.204.205:4028 , piece_id=IQ4CMHEIUNHIUBQV2PVFQ6BURDMWU2S55VGFP5Y7HPFSDVBY6PYA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:08.481127955 (104h35m6.077898968s) [noise=true , address=185.167.204.206:4051 , piece_id=GO4OTIL5WFOS7V77FAT2NRFG2HRE2HXKMU2C3XEGDBGPQL5U5HUA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:10.314952727 (104h35m7.91172374s) [noise=true , address=185.167.204.206:4046 , piece_id=5WVC4R6VY25QUMGSOAORUK3WPGIADBXLKN7ZSLJ4GQEVKMSHZPKA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:36:03.322697363 (104h36m0.919468398s) [noise=true , address=185.167.204.206:4062 , piece_id=7NP2ZIMW6ZDXW3YHZSBO5PRTAZPLYLDWXDA4QPN5XH2C6KAGYLEA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:12BL6XLqsoUGm2FxrTgYTxLZakt2u2CAhZJG82oJ31eaE4vTtMr, address:185.167.204.206:4062): protocol: write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection; write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection
Mar 10 19:36:03.322771647 (104h36m0.919542676s) [noise=true , address=89.149.201.147:4007 , piece_id=AAJY5QBN55ENHTODH25WBVCFDT2SAQ2F27TKHFCIURCZSBMGOY6A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1Q7d4vj1KoWhDr6xd9gEfFrZ1sV3Yro2AFke2f3Ke1rXsGnvew, address:89.149.201.147:4007): protocol: read tcp 159.223.143.39:54460->89.149.201.147:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.322958851 (104h36m0.919729884s) [noise=true , address=185.167.204.205:4033 , piece_id=6JIW66HORXSEPSDWIUONOUZNA2UNXKBA4A6VZ7PBWW3IN2SCJSPQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:12Q73mXHLEfuX4VUbJ8PTjhMoXCrYSKjJbFxqjDdqF3U99N8FXe, address:185.167.204.205:4033): protocol: write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection; write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection
Mar 10 19:36:03.323134221 (104h36m0.919905244s) [noise=true , address=89.149.201.147:4018 , piece_id=V3KLJ75HJ3NRMBP2FIM33GC45QW4U5XMN2R4IAHSX32COGBJF47Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1Ku7o12bSmJ1YR3dGetGFcWqs1wJEh53Zkg3H7ELJrt4w1hkua, address:89.149.201.147:4018): protocol: read tcp 159.223.143.39:47122->89.149.201.147:4018: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.323461923 (104h36m0.920232956s) [noise=true , address=185.167.204.205:4022 , piece_id=F5UFUTAFPRBWMSU3QKV7ZSYQR6UUC7SBUDOC7AAYGDXTC5VYUBHA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1fT9qH1igPUNfrzpwa2VkBX7heh475sNqoN4Nsf9FYSwi1VmHj, address:185.167.204.205:4022): protocol: read tcp 159.223.143.39:51226->185.167.204.205:4022: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.323503666 (104h36m0.920274689s) [seg_pos={0 0}, upload=117052]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Mar 10 19:36:03.424787357 (104h36m1.021558384s) [upload=117052]: Done uploading segments...
Mar 10 19:36:03.424860036 (104h36m1.021631063s) [upload=117052]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 2.149815798s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [.........X.......X..........................................] duration:
2.505512833s trace:
https://jaeger.dev.storj.io/trace/78c3ad63aac6b527
FAIL (2026-03-10 20:07:22 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
:03.719970769 (104h35m1.316741801s) [noise=true , address=185.167.204.205:4033 , piece_id=6JIW66HORXSEPSDWIUONOUZNA2UNXKBA4A6VZ7PBWW3IN2SCJSPQ, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.720006314 (104h35m1.316777347s) [noise=true , address=185.167.204.205:4024 , piece_id=3CKPLBO33LOMNYBF47K2KGQFPO53ZXLMKRKWPVSR4FKQKXKMZMCA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.721245847 (104h35m1.318016883s) [noise=true , address=185.167.204.206:4046 , piece_id=5WVC4R6VY25QUMGSOAORUK3WPGIADBXLKN7ZSLJ4GQEVKMSHZPKA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.722131967 (104h35m1.318902999s) [noise=true , address=89.149.201.147:4004 , piece_id=TMKNRIR6RWRW3TUSA2MQL4LCNTJ3LKEFIT6BAVRX2V54QHNF26FA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.722154312 (104h35m1.318925349s) [noise=true , address=185.167.204.205:4028 , piece_id=IQ4CMHEIUNHIUBQV2PVFQ6BURDMWU2S55VGFP5Y7HPFSDVBY6PYA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.723422869 (104h35m1.320193903s) [noise=true , address=81.171.14.70:4047 , piece_id=DLCUNYJFEKIU3MWFXK24QGJ4GW2H5XE4Y2ARNL2W55EZUMSC4Z7Q, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.724205203 (104h35m1.320976237s) [noise=true , address=185.167.204.205:4022 , piece_id=F5UFUTAFPRBWMSU3QKV7ZSYQR6UUC7SBUDOC7AAYGDXTC5VYUBHA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.727803141 (104h35m1.324574176s) [noise=true , address=185.167.204.206:4067 , piece_id=RH2DWZ3FL7JX6EJ2WTJBPBSWDYVSHLYSLZCMMGRQXUDR6QXVZZ5A, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:04.167055423 (104h35m1.763826466s) [upload=117052]: Next returned nil segment
Mar 10 19:35:04.167072399 (104h35m1.763843424s) [upload=117052]: Waiting for error group managing segments...
Mar 10 19:35:04.350286979 (104h35m1.947058004s) [noise=true , address=81.171.14.70:4035 , piece_id=RNJYIOH4ETYQBSZFYTV3FX73J3PNFR2KBYNZEDXCVBOYMUAN5FCA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.351883208 (104h35m1.948654233s) [noise=true , address=81.171.14.70:4043 , piece_id=LSOHRZMEV5TEQUKZIJPYHVP43CIZKMXBTRVKIZIQNSSFNMAA5FLQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.356149085 (104h35m1.952920098s) [noise=true , address=185.167.204.205:4036 , piece_id=BBGCJ7U7PQDNB5NZTJWHUWXR6TFYF6D7LOR2UUBNV647YQX5JD2Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.369094269 (104h35m1.965865391s) [noise=true , address=89.149.201.147:4011 , piece_id=YHPLXK3ADD6VMYDWQ55PLROBCY2IOLOR2ZWEZ22FRRA4TXKSSWIQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.382982597 (104h35m1.979753611s) [noise=true , address=81.171.14.70:4034 , piece_id=BPXMDI54RTQVH5QKY6JCN5H3AKCPH62V76VBES4GQ6SLIZDIMMOA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.384672279 (104h35m1.981443307s) [noise=true , address=81.171.14.70:4047 , piece_id=DLCUNYJFEKIU3MWFXK24QGJ4GW2H5XE4Y2ARNL2W55EZUMSC4Z7Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.386337288 (104h35m1.98310832s) [noise=true , address=81.171.14.70:4025 , piece_id=R62A3AXM4OKUZFZJ2LB55UPB6VRGHBQLMXXVBWDTHUDWPGPIWEVQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.387583761 (104h35m1.984354793s) [noise=true , address=89.149.201.147:4009 , piece_id=XFSD3W6JWUXGXTHYE57SS5E5322JTRG7RKBI7MBF25GU7SEAKZJQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.388110220 (104h35m1.984881248s) [noise=true , address=81.171.14.70:4026 , piece_id=4CBQ4RZ44USXNYAGA3CIGNC3LBQXR675JVJX6CVTV3X7KVYPRHQA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.389451350 (104h35m1.986222362s) [noise=true , address=185.167.204.205:4012 , piece_id=7UFJIUKCHPKYZF42HTP6Q5D2FMHRSCRUQ7MH2MYJSEOZHZAJRZXA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.404093624 (104h35m2.00086464s) [noise=true , address=81.171.14.70:4048 , piece_id=4F2F4X4ZJ4FXUYQJQKB7YB3JZY53QNVRUSM3N4HQKGEYHMFYQB2A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.413111656 (104h35m2.009882687s) [noise=true , address=185.167.204.206:4071 , piece_id=ZQJLN55G37453GZS54ZDTPM7CEU5GH6LJR3BIKSJDC3C4QOKW5YQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.416183369 (104h35m2.012954404s) [noise=true , address=89.149.201.147:4014 , piece_id=WZOPRPDLFVOO6O2JYZZB75CLIAA4SVM7OEZXBX6WOJV3PUL2YSAQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.449375785 (104h35m2.046146815s) [noise=true , address=185.167.204.206:4039 , piece_id=DKKIZZMA3RAKMLUJTU6OANGBA3F6XWRD5ELVNXAGBIOZMFBVQWZQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.457716093 (104h35m2.054487108s) [noise=true , address=185.167.204.205:4024 , piece_id=3CKPLBO33LOMNYBF47K2KGQFPO53ZXLMKRKWPVSR4FKQKXKMZMCA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.491116161 (104h35m2.087887192s) [noise=true , address=185.167.204.205:4005 , piece_id=E7RO77MCAMFEAIE7ETPALCBV5S4X3DG5ZUD27ZE55RXYICIIXNEQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.507099744 (104h35m2.103870782s) [noise=true , address=185.167.204.205:4029 , piece_id=R33ONZELVLYL76YTJKOZASDJKFBACNNMC3PDUIITQRDGBWWZUNKA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.537139239 (104h35m2.133910269s) [noise=true , address=185.167.204.206:4067 , piece_id=RH2DWZ3FL7JX6EJ2WTJBPBSWDYVSHLYSLZCMMGRQXUDR6QXVZZ5A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.787100189 (104h35m2.383871224s) [noise=true , address=81.171.14.70:4033 , piece_id=ZFV477QXRX42OJIG4D5JLOYU3WERTD65PMHI5MFQLRTQJG3ENUTA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.962846444 (104h35m2.559617469s) [noise=true , address=89.149.201.147:4004 , piece_id=TMKNRIR6RWRW3TUSA2MQL4LCNTJ3LKEFIT6BAVRX2V54QHNF26FA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:08.299132133 (104h35m5.895903147s) [noise=true , address=185.167.204.205:4028 , piece_id=IQ4CMHEIUNHIUBQV2PVFQ6BURDMWU2S55VGFP5Y7HPFSDVBY6PYA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:08.481127955 (104h35m6.077898968s) [noise=true , address=185.167.204.206:4051 , piece_id=GO4OTIL5WFOS7V77FAT2NRFG2HRE2HXKMU2C3XEGDBGPQL5U5HUA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:10.314952727 (104h35m7.91172374s) [noise=true , address=185.167.204.206:4046 , piece_id=5WVC4R6VY25QUMGSOAORUK3WPGIADBXLKN7ZSLJ4GQEVKMSHZPKA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:36:03.322697363 (104h36m0.919468398s) [noise=true , address=185.167.204.206:4062 , piece_id=7NP2ZIMW6ZDXW3YHZSBO5PRTAZPLYLDWXDA4QPN5XH2C6KAGYLEA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:12BL6XLqsoUGm2FxrTgYTxLZakt2u2CAhZJG82oJ31eaE4vTtMr, address:185.167.204.206:4062): protocol: write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection; write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection
Mar 10 19:36:03.322771647 (104h36m0.919542676s) [noise=true , address=89.149.201.147:4007 , piece_id=AAJY5QBN55ENHTODH25WBVCFDT2SAQ2F27TKHFCIURCZSBMGOY6A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1Q7d4vj1KoWhDr6xd9gEfFrZ1sV3Yro2AFke2f3Ke1rXsGnvew, address:89.149.201.147:4007): protocol: read tcp 159.223.143.39:54460->89.149.201.147:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.322958851 (104h36m0.919729884s) [noise=true , address=185.167.204.205:4033 , piece_id=6JIW66HORXSEPSDWIUONOUZNA2UNXKBA4A6VZ7PBWW3IN2SCJSPQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:12Q73mXHLEfuX4VUbJ8PTjhMoXCrYSKjJbFxqjDdqF3U99N8FXe, address:185.167.204.205:4033): protocol: write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection; write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection
Mar 10 19:36:03.323134221 (104h36m0.919905244s) [noise=true , address=89.149.201.147:4018 , piece_id=V3KLJ75HJ3NRMBP2FIM33GC45QW4U5XMN2R4IAHSX32COGBJF47Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1Ku7o12bSmJ1YR3dGetGFcWqs1wJEh53Zkg3H7ELJrt4w1hkua, address:89.149.201.147:4018): protocol: read tcp 159.223.143.39:47122->89.149.201.147:4018: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.323461923 (104h36m0.920232956s) [noise=true , address=185.167.204.205:4022 , piece_id=F5UFUTAFPRBWMSU3QKV7ZSYQR6UUC7SBUDOC7AAYGDXTC5VYUBHA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1fT9qH1igPUNfrzpwa2VkBX7heh475sNqoN4Nsf9FYSwi1VmHj, address:185.167.204.205:4022): protocol: read tcp 159.223.143.39:51226->185.167.204.205:4022: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.323503666 (104h36m0.920274689s) [seg_pos={0 0}, upload=117052]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Mar 10 19:36:03.424787357 (104h36m1.021558384s) [upload=117052]: Done uploading segments...
Mar 10 19:36:03.424860036 (104h36m1.021631063s) [upload=117052]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 2.50554505s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [..........X.......X.........................................] duration:
2.556231425s trace:
https://jaeger.dev.storj.io/trace/78c3ad63aac6b527
FAIL (2026-03-10 20:06:47 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
:03.719970769 (104h35m1.316741801s) [noise=true , address=185.167.204.205:4033 , piece_id=6JIW66HORXSEPSDWIUONOUZNA2UNXKBA4A6VZ7PBWW3IN2SCJSPQ, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.720006314 (104h35m1.316777347s) [noise=true , address=185.167.204.205:4024 , piece_id=3CKPLBO33LOMNYBF47K2KGQFPO53ZXLMKRKWPVSR4FKQKXKMZMCA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.721245847 (104h35m1.318016883s) [noise=true , address=185.167.204.206:4046 , piece_id=5WVC4R6VY25QUMGSOAORUK3WPGIADBXLKN7ZSLJ4GQEVKMSHZPKA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.722131967 (104h35m1.318902999s) [noise=true , address=89.149.201.147:4004 , piece_id=TMKNRIR6RWRW3TUSA2MQL4LCNTJ3LKEFIT6BAVRX2V54QHNF26FA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.722154312 (104h35m1.318925349s) [noise=true , address=185.167.204.205:4028 , piece_id=IQ4CMHEIUNHIUBQV2PVFQ6BURDMWU2S55VGFP5Y7HPFSDVBY6PYA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.723422869 (104h35m1.320193903s) [noise=true , address=81.171.14.70:4047 , piece_id=DLCUNYJFEKIU3MWFXK24QGJ4GW2H5XE4Y2ARNL2W55EZUMSC4Z7Q, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.724205203 (104h35m1.320976237s) [noise=true , address=185.167.204.205:4022 , piece_id=F5UFUTAFPRBWMSU3QKV7ZSYQR6UUC7SBUDOC7AAYGDXTC5VYUBHA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.727803141 (104h35m1.324574176s) [noise=true , address=185.167.204.206:4067 , piece_id=RH2DWZ3FL7JX6EJ2WTJBPBSWDYVSHLYSLZCMMGRQXUDR6QXVZZ5A, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:04.167055423 (104h35m1.763826466s) [upload=117052]: Next returned nil segment
Mar 10 19:35:04.167072399 (104h35m1.763843424s) [upload=117052]: Waiting for error group managing segments...
Mar 10 19:35:04.350286979 (104h35m1.947058004s) [noise=true , address=81.171.14.70:4035 , piece_id=RNJYIOH4ETYQBSZFYTV3FX73J3PNFR2KBYNZEDXCVBOYMUAN5FCA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.351883208 (104h35m1.948654233s) [noise=true , address=81.171.14.70:4043 , piece_id=LSOHRZMEV5TEQUKZIJPYHVP43CIZKMXBTRVKIZIQNSSFNMAA5FLQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.356149085 (104h35m1.952920098s) [noise=true , address=185.167.204.205:4036 , piece_id=BBGCJ7U7PQDNB5NZTJWHUWXR6TFYF6D7LOR2UUBNV647YQX5JD2Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.369094269 (104h35m1.965865391s) [noise=true , address=89.149.201.147:4011 , piece_id=YHPLXK3ADD6VMYDWQ55PLROBCY2IOLOR2ZWEZ22FRRA4TXKSSWIQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.382982597 (104h35m1.979753611s) [noise=true , address=81.171.14.70:4034 , piece_id=BPXMDI54RTQVH5QKY6JCN5H3AKCPH62V76VBES4GQ6SLIZDIMMOA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.384672279 (104h35m1.981443307s) [noise=true , address=81.171.14.70:4047 , piece_id=DLCUNYJFEKIU3MWFXK24QGJ4GW2H5XE4Y2ARNL2W55EZUMSC4Z7Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.386337288 (104h35m1.98310832s) [noise=true , address=81.171.14.70:4025 , piece_id=R62A3AXM4OKUZFZJ2LB55UPB6VRGHBQLMXXVBWDTHUDWPGPIWEVQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.387583761 (104h35m1.984354793s) [noise=true , address=89.149.201.147:4009 , piece_id=XFSD3W6JWUXGXTHYE57SS5E5322JTRG7RKBI7MBF25GU7SEAKZJQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.388110220 (104h35m1.984881248s) [noise=true , address=81.171.14.70:4026 , piece_id=4CBQ4RZ44USXNYAGA3CIGNC3LBQXR675JVJX6CVTV3X7KVYPRHQA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.389451350 (104h35m1.986222362s) [noise=true , address=185.167.204.205:4012 , piece_id=7UFJIUKCHPKYZF42HTP6Q5D2FMHRSCRUQ7MH2MYJSEOZHZAJRZXA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.404093624 (104h35m2.00086464s) [noise=true , address=81.171.14.70:4048 , piece_id=4F2F4X4ZJ4FXUYQJQKB7YB3JZY53QNVRUSM3N4HQKGEYHMFYQB2A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.413111656 (104h35m2.009882687s) [noise=true , address=185.167.204.206:4071 , piece_id=ZQJLN55G37453GZS54ZDTPM7CEU5GH6LJR3BIKSJDC3C4QOKW5YQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.416183369 (104h35m2.012954404s) [noise=true , address=89.149.201.147:4014 , piece_id=WZOPRPDLFVOO6O2JYZZB75CLIAA4SVM7OEZXBX6WOJV3PUL2YSAQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.449375785 (104h35m2.046146815s) [noise=true , address=185.167.204.206:4039 , piece_id=DKKIZZMA3RAKMLUJTU6OANGBA3F6XWRD5ELVNXAGBIOZMFBVQWZQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.457716093 (104h35m2.054487108s) [noise=true , address=185.167.204.205:4024 , piece_id=3CKPLBO33LOMNYBF47K2KGQFPO53ZXLMKRKWPVSR4FKQKXKMZMCA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.491116161 (104h35m2.087887192s) [noise=true , address=185.167.204.205:4005 , piece_id=E7RO77MCAMFEAIE7ETPALCBV5S4X3DG5ZUD27ZE55RXYICIIXNEQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.507099744 (104h35m2.103870782s) [noise=true , address=185.167.204.205:4029 , piece_id=R33ONZELVLYL76YTJKOZASDJKFBACNNMC3PDUIITQRDGBWWZUNKA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.537139239 (104h35m2.133910269s) [noise=true , address=185.167.204.206:4067 , piece_id=RH2DWZ3FL7JX6EJ2WTJBPBSWDYVSHLYSLZCMMGRQXUDR6QXVZZ5A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.787100189 (104h35m2.383871224s) [noise=true , address=81.171.14.70:4033 , piece_id=ZFV477QXRX42OJIG4D5JLOYU3WERTD65PMHI5MFQLRTQJG3ENUTA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.962846444 (104h35m2.559617469s) [noise=true , address=89.149.201.147:4004 , piece_id=TMKNRIR6RWRW3TUSA2MQL4LCNTJ3LKEFIT6BAVRX2V54QHNF26FA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:08.299132133 (104h35m5.895903147s) [noise=true , address=185.167.204.205:4028 , piece_id=IQ4CMHEIUNHIUBQV2PVFQ6BURDMWU2S55VGFP5Y7HPFSDVBY6PYA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:08.481127955 (104h35m6.077898968s) [noise=true , address=185.167.204.206:4051 , piece_id=GO4OTIL5WFOS7V77FAT2NRFG2HRE2HXKMU2C3XEGDBGPQL5U5HUA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:10.314952727 (104h35m7.91172374s) [noise=true , address=185.167.204.206:4046 , piece_id=5WVC4R6VY25QUMGSOAORUK3WPGIADBXLKN7ZSLJ4GQEVKMSHZPKA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:36:03.322697363 (104h36m0.919468398s) [noise=true , address=185.167.204.206:4062 , piece_id=7NP2ZIMW6ZDXW3YHZSBO5PRTAZPLYLDWXDA4QPN5XH2C6KAGYLEA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:12BL6XLqsoUGm2FxrTgYTxLZakt2u2CAhZJG82oJ31eaE4vTtMr, address:185.167.204.206:4062): protocol: write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection; write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection
Mar 10 19:36:03.322771647 (104h36m0.919542676s) [noise=true , address=89.149.201.147:4007 , piece_id=AAJY5QBN55ENHTODH25WBVCFDT2SAQ2F27TKHFCIURCZSBMGOY6A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1Q7d4vj1KoWhDr6xd9gEfFrZ1sV3Yro2AFke2f3Ke1rXsGnvew, address:89.149.201.147:4007): protocol: read tcp 159.223.143.39:54460->89.149.201.147:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.322958851 (104h36m0.919729884s) [noise=true , address=185.167.204.205:4033 , piece_id=6JIW66HORXSEPSDWIUONOUZNA2UNXKBA4A6VZ7PBWW3IN2SCJSPQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:12Q73mXHLEfuX4VUbJ8PTjhMoXCrYSKjJbFxqjDdqF3U99N8FXe, address:185.167.204.205:4033): protocol: write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection; write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection
Mar 10 19:36:03.323134221 (104h36m0.919905244s) [noise=true , address=89.149.201.147:4018 , piece_id=V3KLJ75HJ3NRMBP2FIM33GC45QW4U5XMN2R4IAHSX32COGBJF47Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1Ku7o12bSmJ1YR3dGetGFcWqs1wJEh53Zkg3H7ELJrt4w1hkua, address:89.149.201.147:4018): protocol: read tcp 159.223.143.39:47122->89.149.201.147:4018: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.323461923 (104h36m0.920232956s) [noise=true , address=185.167.204.205:4022 , piece_id=F5UFUTAFPRBWMSU3QKV7ZSYQR6UUC7SBUDOC7AAYGDXTC5VYUBHA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1fT9qH1igPUNfrzpwa2VkBX7heh475sNqoN4Nsf9FYSwi1VmHj, address:185.167.204.205:4022): protocol: read tcp 159.223.143.39:51226->185.167.204.205:4022: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.323503666 (104h36m0.920274689s) [seg_pos={0 0}, upload=117052]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Mar 10 19:36:03.424787357 (104h36m1.021558384s) [upload=117052]: Done uploading segments...
Mar 10 19:36:03.424860036 (104h36m1.021631063s) [upload=117052]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 2.556324021s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [...........X.......X........................................] duration:
2.163478332s trace:
https://jaeger.dev.storj.io/trace/78c3ad63aac6b527
FAIL (2026-03-10 20:05:36 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
:03.719970769 (104h35m1.316741801s) [noise=true , address=185.167.204.205:4033 , piece_id=6JIW66HORXSEPSDWIUONOUZNA2UNXKBA4A6VZ7PBWW3IN2SCJSPQ, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.720006314 (104h35m1.316777347s) [noise=true , address=185.167.204.205:4024 , piece_id=3CKPLBO33LOMNYBF47K2KGQFPO53ZXLMKRKWPVSR4FKQKXKMZMCA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.721245847 (104h35m1.318016883s) [noise=true , address=185.167.204.206:4046 , piece_id=5WVC4R6VY25QUMGSOAORUK3WPGIADBXLKN7ZSLJ4GQEVKMSHZPKA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.722131967 (104h35m1.318902999s) [noise=true , address=89.149.201.147:4004 , piece_id=TMKNRIR6RWRW3TUSA2MQL4LCNTJ3LKEFIT6BAVRX2V54QHNF26FA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.722154312 (104h35m1.318925349s) [noise=true , address=185.167.204.205:4028 , piece_id=IQ4CMHEIUNHIUBQV2PVFQ6BURDMWU2S55VGFP5Y7HPFSDVBY6PYA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.723422869 (104h35m1.320193903s) [noise=true , address=81.171.14.70:4047 , piece_id=DLCUNYJFEKIU3MWFXK24QGJ4GW2H5XE4Y2ARNL2W55EZUMSC4Z7Q, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.724205203 (104h35m1.320976237s) [noise=true , address=185.167.204.205:4022 , piece_id=F5UFUTAFPRBWMSU3QKV7ZSYQR6UUC7SBUDOC7AAYGDXTC5VYUBHA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.727803141 (104h35m1.324574176s) [noise=true , address=185.167.204.206:4067 , piece_id=RH2DWZ3FL7JX6EJ2WTJBPBSWDYVSHLYSLZCMMGRQXUDR6QXVZZ5A, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:04.167055423 (104h35m1.763826466s) [upload=117052]: Next returned nil segment
Mar 10 19:35:04.167072399 (104h35m1.763843424s) [upload=117052]: Waiting for error group managing segments...
Mar 10 19:35:04.350286979 (104h35m1.947058004s) [noise=true , address=81.171.14.70:4035 , piece_id=RNJYIOH4ETYQBSZFYTV3FX73J3PNFR2KBYNZEDXCVBOYMUAN5FCA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.351883208 (104h35m1.948654233s) [noise=true , address=81.171.14.70:4043 , piece_id=LSOHRZMEV5TEQUKZIJPYHVP43CIZKMXBTRVKIZIQNSSFNMAA5FLQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.356149085 (104h35m1.952920098s) [noise=true , address=185.167.204.205:4036 , piece_id=BBGCJ7U7PQDNB5NZTJWHUWXR6TFYF6D7LOR2UUBNV647YQX5JD2Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.369094269 (104h35m1.965865391s) [noise=true , address=89.149.201.147:4011 , piece_id=YHPLXK3ADD6VMYDWQ55PLROBCY2IOLOR2ZWEZ22FRRA4TXKSSWIQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.382982597 (104h35m1.979753611s) [noise=true , address=81.171.14.70:4034 , piece_id=BPXMDI54RTQVH5QKY6JCN5H3AKCPH62V76VBES4GQ6SLIZDIMMOA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.384672279 (104h35m1.981443307s) [noise=true , address=81.171.14.70:4047 , piece_id=DLCUNYJFEKIU3MWFXK24QGJ4GW2H5XE4Y2ARNL2W55EZUMSC4Z7Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.386337288 (104h35m1.98310832s) [noise=true , address=81.171.14.70:4025 , piece_id=R62A3AXM4OKUZFZJ2LB55UPB6VRGHBQLMXXVBWDTHUDWPGPIWEVQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.387583761 (104h35m1.984354793s) [noise=true , address=89.149.201.147:4009 , piece_id=XFSD3W6JWUXGXTHYE57SS5E5322JTRG7RKBI7MBF25GU7SEAKZJQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.388110220 (104h35m1.984881248s) [noise=true , address=81.171.14.70:4026 , piece_id=4CBQ4RZ44USXNYAGA3CIGNC3LBQXR675JVJX6CVTV3X7KVYPRHQA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.389451350 (104h35m1.986222362s) [noise=true , address=185.167.204.205:4012 , piece_id=7UFJIUKCHPKYZF42HTP6Q5D2FMHRSCRUQ7MH2MYJSEOZHZAJRZXA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.404093624 (104h35m2.00086464s) [noise=true , address=81.171.14.70:4048 , piece_id=4F2F4X4ZJ4FXUYQJQKB7YB3JZY53QNVRUSM3N4HQKGEYHMFYQB2A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.413111656 (104h35m2.009882687s) [noise=true , address=185.167.204.206:4071 , piece_id=ZQJLN55G37453GZS54ZDTPM7CEU5GH6LJR3BIKSJDC3C4QOKW5YQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.416183369 (104h35m2.012954404s) [noise=true , address=89.149.201.147:4014 , piece_id=WZOPRPDLFVOO6O2JYZZB75CLIAA4SVM7OEZXBX6WOJV3PUL2YSAQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.449375785 (104h35m2.046146815s) [noise=true , address=185.167.204.206:4039 , piece_id=DKKIZZMA3RAKMLUJTU6OANGBA3F6XWRD5ELVNXAGBIOZMFBVQWZQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.457716093 (104h35m2.054487108s) [noise=true , address=185.167.204.205:4024 , piece_id=3CKPLBO33LOMNYBF47K2KGQFPO53ZXLMKRKWPVSR4FKQKXKMZMCA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.491116161 (104h35m2.087887192s) [noise=true , address=185.167.204.205:4005 , piece_id=E7RO77MCAMFEAIE7ETPALCBV5S4X3DG5ZUD27ZE55RXYICIIXNEQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.507099744 (104h35m2.103870782s) [noise=true , address=185.167.204.205:4029 , piece_id=R33ONZELVLYL76YTJKOZASDJKFBACNNMC3PDUIITQRDGBWWZUNKA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.537139239 (104h35m2.133910269s) [noise=true , address=185.167.204.206:4067 , piece_id=RH2DWZ3FL7JX6EJ2WTJBPBSWDYVSHLYSLZCMMGRQXUDR6QXVZZ5A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.787100189 (104h35m2.383871224s) [noise=true , address=81.171.14.70:4033 , piece_id=ZFV477QXRX42OJIG4D5JLOYU3WERTD65PMHI5MFQLRTQJG3ENUTA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.962846444 (104h35m2.559617469s) [noise=true , address=89.149.201.147:4004 , piece_id=TMKNRIR6RWRW3TUSA2MQL4LCNTJ3LKEFIT6BAVRX2V54QHNF26FA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:08.299132133 (104h35m5.895903147s) [noise=true , address=185.167.204.205:4028 , piece_id=IQ4CMHEIUNHIUBQV2PVFQ6BURDMWU2S55VGFP5Y7HPFSDVBY6PYA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:08.481127955 (104h35m6.077898968s) [noise=true , address=185.167.204.206:4051 , piece_id=GO4OTIL5WFOS7V77FAT2NRFG2HRE2HXKMU2C3XEGDBGPQL5U5HUA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:10.314952727 (104h35m7.91172374s) [noise=true , address=185.167.204.206:4046 , piece_id=5WVC4R6VY25QUMGSOAORUK3WPGIADBXLKN7ZSLJ4GQEVKMSHZPKA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:36:03.322697363 (104h36m0.919468398s) [noise=true , address=185.167.204.206:4062 , piece_id=7NP2ZIMW6ZDXW3YHZSBO5PRTAZPLYLDWXDA4QPN5XH2C6KAGYLEA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:12BL6XLqsoUGm2FxrTgYTxLZakt2u2CAhZJG82oJ31eaE4vTtMr, address:185.167.204.206:4062): protocol: write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection; write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection
Mar 10 19:36:03.322771647 (104h36m0.919542676s) [noise=true , address=89.149.201.147:4007 , piece_id=AAJY5QBN55ENHTODH25WBVCFDT2SAQ2F27TKHFCIURCZSBMGOY6A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1Q7d4vj1KoWhDr6xd9gEfFrZ1sV3Yro2AFke2f3Ke1rXsGnvew, address:89.149.201.147:4007): protocol: read tcp 159.223.143.39:54460->89.149.201.147:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.322958851 (104h36m0.919729884s) [noise=true , address=185.167.204.205:4033 , piece_id=6JIW66HORXSEPSDWIUONOUZNA2UNXKBA4A6VZ7PBWW3IN2SCJSPQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:12Q73mXHLEfuX4VUbJ8PTjhMoXCrYSKjJbFxqjDdqF3U99N8FXe, address:185.167.204.205:4033): protocol: write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection; write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection
Mar 10 19:36:03.323134221 (104h36m0.919905244s) [noise=true , address=89.149.201.147:4018 , piece_id=V3KLJ75HJ3NRMBP2FIM33GC45QW4U5XMN2R4IAHSX32COGBJF47Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1Ku7o12bSmJ1YR3dGetGFcWqs1wJEh53Zkg3H7ELJrt4w1hkua, address:89.149.201.147:4018): protocol: read tcp 159.223.143.39:47122->89.149.201.147:4018: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.323461923 (104h36m0.920232956s) [noise=true , address=185.167.204.205:4022 , piece_id=F5UFUTAFPRBWMSU3QKV7ZSYQR6UUC7SBUDOC7AAYGDXTC5VYUBHA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1fT9qH1igPUNfrzpwa2VkBX7heh475sNqoN4Nsf9FYSwi1VmHj, address:185.167.204.205:4022): protocol: read tcp 159.223.143.39:51226->185.167.204.205:4022: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.323503666 (104h36m0.920274689s) [seg_pos={0 0}, upload=117052]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Mar 10 19:36:03.424787357 (104h36m1.021558384s) [upload=117052]: Done uploading segments...
Mar 10 19:36:03.424860036 (104h36m1.021631063s) [upload=117052]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 2.163554414s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
3 out of 60 is failed (threshold=3) [X...........X.......X.......................................] duration:
2.617566234s trace:
https://jaeger.dev.storj.io/trace/78c3ad63aac6b527
FAIL (2026-03-10 20:05:05 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
:03.719970769 (104h35m1.316741801s) [noise=true , address=185.167.204.205:4033 , piece_id=6JIW66HORXSEPSDWIUONOUZNA2UNXKBA4A6VZ7PBWW3IN2SCJSPQ, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.720006314 (104h35m1.316777347s) [noise=true , address=185.167.204.205:4024 , piece_id=3CKPLBO33LOMNYBF47K2KGQFPO53ZXLMKRKWPVSR4FKQKXKMZMCA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.721245847 (104h35m1.318016883s) [noise=true , address=185.167.204.206:4046 , piece_id=5WVC4R6VY25QUMGSOAORUK3WPGIADBXLKN7ZSLJ4GQEVKMSHZPKA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.722131967 (104h35m1.318902999s) [noise=true , address=89.149.201.147:4004 , piece_id=TMKNRIR6RWRW3TUSA2MQL4LCNTJ3LKEFIT6BAVRX2V54QHNF26FA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.722154312 (104h35m1.318925349s) [noise=true , address=185.167.204.205:4028 , piece_id=IQ4CMHEIUNHIUBQV2PVFQ6BURDMWU2S55VGFP5Y7HPFSDVBY6PYA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.723422869 (104h35m1.320193903s) [noise=true , address=81.171.14.70:4047 , piece_id=DLCUNYJFEKIU3MWFXK24QGJ4GW2H5XE4Y2ARNL2W55EZUMSC4Z7Q, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.724205203 (104h35m1.320976237s) [noise=true , address=185.167.204.205:4022 , piece_id=F5UFUTAFPRBWMSU3QKV7ZSYQR6UUC7SBUDOC7AAYGDXTC5VYUBHA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.727803141 (104h35m1.324574176s) [noise=true , address=185.167.204.206:4067 , piece_id=RH2DWZ3FL7JX6EJ2WTJBPBSWDYVSHLYSLZCMMGRQXUDR6QXVZZ5A, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:04.167055423 (104h35m1.763826466s) [upload=117052]: Next returned nil segment
Mar 10 19:35:04.167072399 (104h35m1.763843424s) [upload=117052]: Waiting for error group managing segments...
Mar 10 19:35:04.350286979 (104h35m1.947058004s) [noise=true , address=81.171.14.70:4035 , piece_id=RNJYIOH4ETYQBSZFYTV3FX73J3PNFR2KBYNZEDXCVBOYMUAN5FCA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.351883208 (104h35m1.948654233s) [noise=true , address=81.171.14.70:4043 , piece_id=LSOHRZMEV5TEQUKZIJPYHVP43CIZKMXBTRVKIZIQNSSFNMAA5FLQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.356149085 (104h35m1.952920098s) [noise=true , address=185.167.204.205:4036 , piece_id=BBGCJ7U7PQDNB5NZTJWHUWXR6TFYF6D7LOR2UUBNV647YQX5JD2Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.369094269 (104h35m1.965865391s) [noise=true , address=89.149.201.147:4011 , piece_id=YHPLXK3ADD6VMYDWQ55PLROBCY2IOLOR2ZWEZ22FRRA4TXKSSWIQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.382982597 (104h35m1.979753611s) [noise=true , address=81.171.14.70:4034 , piece_id=BPXMDI54RTQVH5QKY6JCN5H3AKCPH62V76VBES4GQ6SLIZDIMMOA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.384672279 (104h35m1.981443307s) [noise=true , address=81.171.14.70:4047 , piece_id=DLCUNYJFEKIU3MWFXK24QGJ4GW2H5XE4Y2ARNL2W55EZUMSC4Z7Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.386337288 (104h35m1.98310832s) [noise=true , address=81.171.14.70:4025 , piece_id=R62A3AXM4OKUZFZJ2LB55UPB6VRGHBQLMXXVBWDTHUDWPGPIWEVQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.387583761 (104h35m1.984354793s) [noise=true , address=89.149.201.147:4009 , piece_id=XFSD3W6JWUXGXTHYE57SS5E5322JTRG7RKBI7MBF25GU7SEAKZJQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.388110220 (104h35m1.984881248s) [noise=true , address=81.171.14.70:4026 , piece_id=4CBQ4RZ44USXNYAGA3CIGNC3LBQXR675JVJX6CVTV3X7KVYPRHQA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.389451350 (104h35m1.986222362s) [noise=true , address=185.167.204.205:4012 , piece_id=7UFJIUKCHPKYZF42HTP6Q5D2FMHRSCRUQ7MH2MYJSEOZHZAJRZXA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.404093624 (104h35m2.00086464s) [noise=true , address=81.171.14.70:4048 , piece_id=4F2F4X4ZJ4FXUYQJQKB7YB3JZY53QNVRUSM3N4HQKGEYHMFYQB2A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.413111656 (104h35m2.009882687s) [noise=true , address=185.167.204.206:4071 , piece_id=ZQJLN55G37453GZS54ZDTPM7CEU5GH6LJR3BIKSJDC3C4QOKW5YQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.416183369 (104h35m2.012954404s) [noise=true , address=89.149.201.147:4014 , piece_id=WZOPRPDLFVOO6O2JYZZB75CLIAA4SVM7OEZXBX6WOJV3PUL2YSAQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.449375785 (104h35m2.046146815s) [noise=true , address=185.167.204.206:4039 , piece_id=DKKIZZMA3RAKMLUJTU6OANGBA3F6XWRD5ELVNXAGBIOZMFBVQWZQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.457716093 (104h35m2.054487108s) [noise=true , address=185.167.204.205:4024 , piece_id=3CKPLBO33LOMNYBF47K2KGQFPO53ZXLMKRKWPVSR4FKQKXKMZMCA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.491116161 (104h35m2.087887192s) [noise=true , address=185.167.204.205:4005 , piece_id=E7RO77MCAMFEAIE7ETPALCBV5S4X3DG5ZUD27ZE55RXYICIIXNEQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.507099744 (104h35m2.103870782s) [noise=true , address=185.167.204.205:4029 , piece_id=R33ONZELVLYL76YTJKOZASDJKFBACNNMC3PDUIITQRDGBWWZUNKA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.537139239 (104h35m2.133910269s) [noise=true , address=185.167.204.206:4067 , piece_id=RH2DWZ3FL7JX6EJ2WTJBPBSWDYVSHLYSLZCMMGRQXUDR6QXVZZ5A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.787100189 (104h35m2.383871224s) [noise=true , address=81.171.14.70:4033 , piece_id=ZFV477QXRX42OJIG4D5JLOYU3WERTD65PMHI5MFQLRTQJG3ENUTA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.962846444 (104h35m2.559617469s) [noise=true , address=89.149.201.147:4004 , piece_id=TMKNRIR6RWRW3TUSA2MQL4LCNTJ3LKEFIT6BAVRX2V54QHNF26FA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:08.299132133 (104h35m5.895903147s) [noise=true , address=185.167.204.205:4028 , piece_id=IQ4CMHEIUNHIUBQV2PVFQ6BURDMWU2S55VGFP5Y7HPFSDVBY6PYA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:08.481127955 (104h35m6.077898968s) [noise=true , address=185.167.204.206:4051 , piece_id=GO4OTIL5WFOS7V77FAT2NRFG2HRE2HXKMU2C3XEGDBGPQL5U5HUA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:10.314952727 (104h35m7.91172374s) [noise=true , address=185.167.204.206:4046 , piece_id=5WVC4R6VY25QUMGSOAORUK3WPGIADBXLKN7ZSLJ4GQEVKMSHZPKA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:36:03.322697363 (104h36m0.919468398s) [noise=true , address=185.167.204.206:4062 , piece_id=7NP2ZIMW6ZDXW3YHZSBO5PRTAZPLYLDWXDA4QPN5XH2C6KAGYLEA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:12BL6XLqsoUGm2FxrTgYTxLZakt2u2CAhZJG82oJ31eaE4vTtMr, address:185.167.204.206:4062): protocol: write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection; write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection
Mar 10 19:36:03.322771647 (104h36m0.919542676s) [noise=true , address=89.149.201.147:4007 , piece_id=AAJY5QBN55ENHTODH25WBVCFDT2SAQ2F27TKHFCIURCZSBMGOY6A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1Q7d4vj1KoWhDr6xd9gEfFrZ1sV3Yro2AFke2f3Ke1rXsGnvew, address:89.149.201.147:4007): protocol: read tcp 159.223.143.39:54460->89.149.201.147:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.322958851 (104h36m0.919729884s) [noise=true , address=185.167.204.205:4033 , piece_id=6JIW66HORXSEPSDWIUONOUZNA2UNXKBA4A6VZ7PBWW3IN2SCJSPQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:12Q73mXHLEfuX4VUbJ8PTjhMoXCrYSKjJbFxqjDdqF3U99N8FXe, address:185.167.204.205:4033): protocol: write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection; write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection
Mar 10 19:36:03.323134221 (104h36m0.919905244s) [noise=true , address=89.149.201.147:4018 , piece_id=V3KLJ75HJ3NRMBP2FIM33GC45QW4U5XMN2R4IAHSX32COGBJF47Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1Ku7o12bSmJ1YR3dGetGFcWqs1wJEh53Zkg3H7ELJrt4w1hkua, address:89.149.201.147:4018): protocol: read tcp 159.223.143.39:47122->89.149.201.147:4018: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.323461923 (104h36m0.920232956s) [noise=true , address=185.167.204.205:4022 , piece_id=F5UFUTAFPRBWMSU3QKV7ZSYQR6UUC7SBUDOC7AAYGDXTC5VYUBHA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1fT9qH1igPUNfrzpwa2VkBX7heh475sNqoN4Nsf9FYSwi1VmHj, address:185.167.204.205:4022): protocol: read tcp 159.223.143.39:51226->185.167.204.205:4022: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.323503666 (104h36m0.920274689s) [seg_pos={0 0}, upload=117052]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Mar 10 19:36:03.424787357 (104h36m1.021558384s) [upload=117052]: Done uploading segments...
Mar 10 19:36:03.424860036 (104h36m1.021631063s) [upload=117052]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 2.617643445s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
3 out of 60 is failed (threshold=3) [.X...........X.......X......................................] duration:
2.168837219s trace:
https://jaeger.dev.storj.io/trace/78c3ad63aac6b527
FAIL (2026-03-10 20:04:06 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
:03.719970769 (104h35m1.316741801s) [noise=true , address=185.167.204.205:4033 , piece_id=6JIW66HORXSEPSDWIUONOUZNA2UNXKBA4A6VZ7PBWW3IN2SCJSPQ, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.720006314 (104h35m1.316777347s) [noise=true , address=185.167.204.205:4024 , piece_id=3CKPLBO33LOMNYBF47K2KGQFPO53ZXLMKRKWPVSR4FKQKXKMZMCA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.721245847 (104h35m1.318016883s) [noise=true , address=185.167.204.206:4046 , piece_id=5WVC4R6VY25QUMGSOAORUK3WPGIADBXLKN7ZSLJ4GQEVKMSHZPKA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.722131967 (104h35m1.318902999s) [noise=true , address=89.149.201.147:4004 , piece_id=TMKNRIR6RWRW3TUSA2MQL4LCNTJ3LKEFIT6BAVRX2V54QHNF26FA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.722154312 (104h35m1.318925349s) [noise=true , address=185.167.204.205:4028 , piece_id=IQ4CMHEIUNHIUBQV2PVFQ6BURDMWU2S55VGFP5Y7HPFSDVBY6PYA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.723422869 (104h35m1.320193903s) [noise=true , address=81.171.14.70:4047 , piece_id=DLCUNYJFEKIU3MWFXK24QGJ4GW2H5XE4Y2ARNL2W55EZUMSC4Z7Q, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.724205203 (104h35m1.320976237s) [noise=true , address=185.167.204.205:4022 , piece_id=F5UFUTAFPRBWMSU3QKV7ZSYQR6UUC7SBUDOC7AAYGDXTC5VYUBHA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.727803141 (104h35m1.324574176s) [noise=true , address=185.167.204.206:4067 , piece_id=RH2DWZ3FL7JX6EJ2WTJBPBSWDYVSHLYSLZCMMGRQXUDR6QXVZZ5A, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:04.167055423 (104h35m1.763826466s) [upload=117052]: Next returned nil segment
Mar 10 19:35:04.167072399 (104h35m1.763843424s) [upload=117052]: Waiting for error group managing segments...
Mar 10 19:35:04.350286979 (104h35m1.947058004s) [noise=true , address=81.171.14.70:4035 , piece_id=RNJYIOH4ETYQBSZFYTV3FX73J3PNFR2KBYNZEDXCVBOYMUAN5FCA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.351883208 (104h35m1.948654233s) [noise=true , address=81.171.14.70:4043 , piece_id=LSOHRZMEV5TEQUKZIJPYHVP43CIZKMXBTRVKIZIQNSSFNMAA5FLQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.356149085 (104h35m1.952920098s) [noise=true , address=185.167.204.205:4036 , piece_id=BBGCJ7U7PQDNB5NZTJWHUWXR6TFYF6D7LOR2UUBNV647YQX5JD2Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.369094269 (104h35m1.965865391s) [noise=true , address=89.149.201.147:4011 , piece_id=YHPLXK3ADD6VMYDWQ55PLROBCY2IOLOR2ZWEZ22FRRA4TXKSSWIQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.382982597 (104h35m1.979753611s) [noise=true , address=81.171.14.70:4034 , piece_id=BPXMDI54RTQVH5QKY6JCN5H3AKCPH62V76VBES4GQ6SLIZDIMMOA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.384672279 (104h35m1.981443307s) [noise=true , address=81.171.14.70:4047 , piece_id=DLCUNYJFEKIU3MWFXK24QGJ4GW2H5XE4Y2ARNL2W55EZUMSC4Z7Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.386337288 (104h35m1.98310832s) [noise=true , address=81.171.14.70:4025 , piece_id=R62A3AXM4OKUZFZJ2LB55UPB6VRGHBQLMXXVBWDTHUDWPGPIWEVQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.387583761 (104h35m1.984354793s) [noise=true , address=89.149.201.147:4009 , piece_id=XFSD3W6JWUXGXTHYE57SS5E5322JTRG7RKBI7MBF25GU7SEAKZJQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.388110220 (104h35m1.984881248s) [noise=true , address=81.171.14.70:4026 , piece_id=4CBQ4RZ44USXNYAGA3CIGNC3LBQXR675JVJX6CVTV3X7KVYPRHQA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.389451350 (104h35m1.986222362s) [noise=true , address=185.167.204.205:4012 , piece_id=7UFJIUKCHPKYZF42HTP6Q5D2FMHRSCRUQ7MH2MYJSEOZHZAJRZXA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.404093624 (104h35m2.00086464s) [noise=true , address=81.171.14.70:4048 , piece_id=4F2F4X4ZJ4FXUYQJQKB7YB3JZY53QNVRUSM3N4HQKGEYHMFYQB2A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.413111656 (104h35m2.009882687s) [noise=true , address=185.167.204.206:4071 , piece_id=ZQJLN55G37453GZS54ZDTPM7CEU5GH6LJR3BIKSJDC3C4QOKW5YQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.416183369 (104h35m2.012954404s) [noise=true , address=89.149.201.147:4014 , piece_id=WZOPRPDLFVOO6O2JYZZB75CLIAA4SVM7OEZXBX6WOJV3PUL2YSAQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.449375785 (104h35m2.046146815s) [noise=true , address=185.167.204.206:4039 , piece_id=DKKIZZMA3RAKMLUJTU6OANGBA3F6XWRD5ELVNXAGBIOZMFBVQWZQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.457716093 (104h35m2.054487108s) [noise=true , address=185.167.204.205:4024 , piece_id=3CKPLBO33LOMNYBF47K2KGQFPO53ZXLMKRKWPVSR4FKQKXKMZMCA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.491116161 (104h35m2.087887192s) [noise=true , address=185.167.204.205:4005 , piece_id=E7RO77MCAMFEAIE7ETPALCBV5S4X3DG5ZUD27ZE55RXYICIIXNEQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.507099744 (104h35m2.103870782s) [noise=true , address=185.167.204.205:4029 , piece_id=R33ONZELVLYL76YTJKOZASDJKFBACNNMC3PDUIITQRDGBWWZUNKA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.537139239 (104h35m2.133910269s) [noise=true , address=185.167.204.206:4067 , piece_id=RH2DWZ3FL7JX6EJ2WTJBPBSWDYVSHLYSLZCMMGRQXUDR6QXVZZ5A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.787100189 (104h35m2.383871224s) [noise=true , address=81.171.14.70:4033 , piece_id=ZFV477QXRX42OJIG4D5JLOYU3WERTD65PMHI5MFQLRTQJG3ENUTA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.962846444 (104h35m2.559617469s) [noise=true , address=89.149.201.147:4004 , piece_id=TMKNRIR6RWRW3TUSA2MQL4LCNTJ3LKEFIT6BAVRX2V54QHNF26FA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:08.299132133 (104h35m5.895903147s) [noise=true , address=185.167.204.205:4028 , piece_id=IQ4CMHEIUNHIUBQV2PVFQ6BURDMWU2S55VGFP5Y7HPFSDVBY6PYA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:08.481127955 (104h35m6.077898968s) [noise=true , address=185.167.204.206:4051 , piece_id=GO4OTIL5WFOS7V77FAT2NRFG2HRE2HXKMU2C3XEGDBGPQL5U5HUA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:10.314952727 (104h35m7.91172374s) [noise=true , address=185.167.204.206:4046 , piece_id=5WVC4R6VY25QUMGSOAORUK3WPGIADBXLKN7ZSLJ4GQEVKMSHZPKA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:36:03.322697363 (104h36m0.919468398s) [noise=true , address=185.167.204.206:4062 , piece_id=7NP2ZIMW6ZDXW3YHZSBO5PRTAZPLYLDWXDA4QPN5XH2C6KAGYLEA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:12BL6XLqsoUGm2FxrTgYTxLZakt2u2CAhZJG82oJ31eaE4vTtMr, address:185.167.204.206:4062): protocol: write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection; write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection
Mar 10 19:36:03.322771647 (104h36m0.919542676s) [noise=true , address=89.149.201.147:4007 , piece_id=AAJY5QBN55ENHTODH25WBVCFDT2SAQ2F27TKHFCIURCZSBMGOY6A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1Q7d4vj1KoWhDr6xd9gEfFrZ1sV3Yro2AFke2f3Ke1rXsGnvew, address:89.149.201.147:4007): protocol: read tcp 159.223.143.39:54460->89.149.201.147:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.322958851 (104h36m0.919729884s) [noise=true , address=185.167.204.205:4033 , piece_id=6JIW66HORXSEPSDWIUONOUZNA2UNXKBA4A6VZ7PBWW3IN2SCJSPQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:12Q73mXHLEfuX4VUbJ8PTjhMoXCrYSKjJbFxqjDdqF3U99N8FXe, address:185.167.204.205:4033): protocol: write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection; write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection
Mar 10 19:36:03.323134221 (104h36m0.919905244s) [noise=true , address=89.149.201.147:4018 , piece_id=V3KLJ75HJ3NRMBP2FIM33GC45QW4U5XMN2R4IAHSX32COGBJF47Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1Ku7o12bSmJ1YR3dGetGFcWqs1wJEh53Zkg3H7ELJrt4w1hkua, address:89.149.201.147:4018): protocol: read tcp 159.223.143.39:47122->89.149.201.147:4018: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.323461923 (104h36m0.920232956s) [noise=true , address=185.167.204.205:4022 , piece_id=F5UFUTAFPRBWMSU3QKV7ZSYQR6UUC7SBUDOC7AAYGDXTC5VYUBHA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1fT9qH1igPUNfrzpwa2VkBX7heh475sNqoN4Nsf9FYSwi1VmHj, address:185.167.204.205:4022): protocol: read tcp 159.223.143.39:51226->185.167.204.205:4022: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.323503666 (104h36m0.920274689s) [seg_pos={0 0}, upload=117052]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Mar 10 19:36:03.424787357 (104h36m1.021558384s) [upload=117052]: Done uploading segments...
Mar 10 19:36:03.424860036 (104h36m1.021631063s) [upload=117052]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 2.168864924s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
4 out of 60 is failed (threshold=3) [X.X...........X.......X.....................................] duration:
2.550598704s trace:
https://jaeger.dev.storj.io/trace/78c3ad63aac6b527
FAIL (2026-03-10 20:02:55 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
:03.719970769 (104h35m1.316741801s) [noise=true , address=185.167.204.205:4033 , piece_id=6JIW66HORXSEPSDWIUONOUZNA2UNXKBA4A6VZ7PBWW3IN2SCJSPQ, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.720006314 (104h35m1.316777347s) [noise=true , address=185.167.204.205:4024 , piece_id=3CKPLBO33LOMNYBF47K2KGQFPO53ZXLMKRKWPVSR4FKQKXKMZMCA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.721245847 (104h35m1.318016883s) [noise=true , address=185.167.204.206:4046 , piece_id=5WVC4R6VY25QUMGSOAORUK3WPGIADBXLKN7ZSLJ4GQEVKMSHZPKA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.722131967 (104h35m1.318902999s) [noise=true , address=89.149.201.147:4004 , piece_id=TMKNRIR6RWRW3TUSA2MQL4LCNTJ3LKEFIT6BAVRX2V54QHNF26FA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.722154312 (104h35m1.318925349s) [noise=true , address=185.167.204.205:4028 , piece_id=IQ4CMHEIUNHIUBQV2PVFQ6BURDMWU2S55VGFP5Y7HPFSDVBY6PYA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.723422869 (104h35m1.320193903s) [noise=true , address=81.171.14.70:4047 , piece_id=DLCUNYJFEKIU3MWFXK24QGJ4GW2H5XE4Y2ARNL2W55EZUMSC4Z7Q, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.724205203 (104h35m1.320976237s) [noise=true , address=185.167.204.205:4022 , piece_id=F5UFUTAFPRBWMSU3QKV7ZSYQR6UUC7SBUDOC7AAYGDXTC5VYUBHA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.727803141 (104h35m1.324574176s) [noise=true , address=185.167.204.206:4067 , piece_id=RH2DWZ3FL7JX6EJ2WTJBPBSWDYVSHLYSLZCMMGRQXUDR6QXVZZ5A, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:04.167055423 (104h35m1.763826466s) [upload=117052]: Next returned nil segment
Mar 10 19:35:04.167072399 (104h35m1.763843424s) [upload=117052]: Waiting for error group managing segments...
Mar 10 19:35:04.350286979 (104h35m1.947058004s) [noise=true , address=81.171.14.70:4035 , piece_id=RNJYIOH4ETYQBSZFYTV3FX73J3PNFR2KBYNZEDXCVBOYMUAN5FCA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.351883208 (104h35m1.948654233s) [noise=true , address=81.171.14.70:4043 , piece_id=LSOHRZMEV5TEQUKZIJPYHVP43CIZKMXBTRVKIZIQNSSFNMAA5FLQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.356149085 (104h35m1.952920098s) [noise=true , address=185.167.204.205:4036 , piece_id=BBGCJ7U7PQDNB5NZTJWHUWXR6TFYF6D7LOR2UUBNV647YQX5JD2Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.369094269 (104h35m1.965865391s) [noise=true , address=89.149.201.147:4011 , piece_id=YHPLXK3ADD6VMYDWQ55PLROBCY2IOLOR2ZWEZ22FRRA4TXKSSWIQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.382982597 (104h35m1.979753611s) [noise=true , address=81.171.14.70:4034 , piece_id=BPXMDI54RTQVH5QKY6JCN5H3AKCPH62V76VBES4GQ6SLIZDIMMOA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.384672279 (104h35m1.981443307s) [noise=true , address=81.171.14.70:4047 , piece_id=DLCUNYJFEKIU3MWFXK24QGJ4GW2H5XE4Y2ARNL2W55EZUMSC4Z7Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.386337288 (104h35m1.98310832s) [noise=true , address=81.171.14.70:4025 , piece_id=R62A3AXM4OKUZFZJ2LB55UPB6VRGHBQLMXXVBWDTHUDWPGPIWEVQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.387583761 (104h35m1.984354793s) [noise=true , address=89.149.201.147:4009 , piece_id=XFSD3W6JWUXGXTHYE57SS5E5322JTRG7RKBI7MBF25GU7SEAKZJQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.388110220 (104h35m1.984881248s) [noise=true , address=81.171.14.70:4026 , piece_id=4CBQ4RZ44USXNYAGA3CIGNC3LBQXR675JVJX6CVTV3X7KVYPRHQA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.389451350 (104h35m1.986222362s) [noise=true , address=185.167.204.205:4012 , piece_id=7UFJIUKCHPKYZF42HTP6Q5D2FMHRSCRUQ7MH2MYJSEOZHZAJRZXA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.404093624 (104h35m2.00086464s) [noise=true , address=81.171.14.70:4048 , piece_id=4F2F4X4ZJ4FXUYQJQKB7YB3JZY53QNVRUSM3N4HQKGEYHMFYQB2A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.413111656 (104h35m2.009882687s) [noise=true , address=185.167.204.206:4071 , piece_id=ZQJLN55G37453GZS54ZDTPM7CEU5GH6LJR3BIKSJDC3C4QOKW5YQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.416183369 (104h35m2.012954404s) [noise=true , address=89.149.201.147:4014 , piece_id=WZOPRPDLFVOO6O2JYZZB75CLIAA4SVM7OEZXBX6WOJV3PUL2YSAQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.449375785 (104h35m2.046146815s) [noise=true , address=185.167.204.206:4039 , piece_id=DKKIZZMA3RAKMLUJTU6OANGBA3F6XWRD5ELVNXAGBIOZMFBVQWZQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.457716093 (104h35m2.054487108s) [noise=true , address=185.167.204.205:4024 , piece_id=3CKPLBO33LOMNYBF47K2KGQFPO53ZXLMKRKWPVSR4FKQKXKMZMCA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.491116161 (104h35m2.087887192s) [noise=true , address=185.167.204.205:4005 , piece_id=E7RO77MCAMFEAIE7ETPALCBV5S4X3DG5ZUD27ZE55RXYICIIXNEQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.507099744 (104h35m2.103870782s) [noise=true , address=185.167.204.205:4029 , piece_id=R33ONZELVLYL76YTJKOZASDJKFBACNNMC3PDUIITQRDGBWWZUNKA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.537139239 (104h35m2.133910269s) [noise=true , address=185.167.204.206:4067 , piece_id=RH2DWZ3FL7JX6EJ2WTJBPBSWDYVSHLYSLZCMMGRQXUDR6QXVZZ5A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.787100189 (104h35m2.383871224s) [noise=true , address=81.171.14.70:4033 , piece_id=ZFV477QXRX42OJIG4D5JLOYU3WERTD65PMHI5MFQLRTQJG3ENUTA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.962846444 (104h35m2.559617469s) [noise=true , address=89.149.201.147:4004 , piece_id=TMKNRIR6RWRW3TUSA2MQL4LCNTJ3LKEFIT6BAVRX2V54QHNF26FA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:08.299132133 (104h35m5.895903147s) [noise=true , address=185.167.204.205:4028 , piece_id=IQ4CMHEIUNHIUBQV2PVFQ6BURDMWU2S55VGFP5Y7HPFSDVBY6PYA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:08.481127955 (104h35m6.077898968s) [noise=true , address=185.167.204.206:4051 , piece_id=GO4OTIL5WFOS7V77FAT2NRFG2HRE2HXKMU2C3XEGDBGPQL5U5HUA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:10.314952727 (104h35m7.91172374s) [noise=true , address=185.167.204.206:4046 , piece_id=5WVC4R6VY25QUMGSOAORUK3WPGIADBXLKN7ZSLJ4GQEVKMSHZPKA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:36:03.322697363 (104h36m0.919468398s) [noise=true , address=185.167.204.206:4062 , piece_id=7NP2ZIMW6ZDXW3YHZSBO5PRTAZPLYLDWXDA4QPN5XH2C6KAGYLEA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:12BL6XLqsoUGm2FxrTgYTxLZakt2u2CAhZJG82oJ31eaE4vTtMr, address:185.167.204.206:4062): protocol: write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection; write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection
Mar 10 19:36:03.322771647 (104h36m0.919542676s) [noise=true , address=89.149.201.147:4007 , piece_id=AAJY5QBN55ENHTODH25WBVCFDT2SAQ2F27TKHFCIURCZSBMGOY6A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1Q7d4vj1KoWhDr6xd9gEfFrZ1sV3Yro2AFke2f3Ke1rXsGnvew, address:89.149.201.147:4007): protocol: read tcp 159.223.143.39:54460->89.149.201.147:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.322958851 (104h36m0.919729884s) [noise=true , address=185.167.204.205:4033 , piece_id=6JIW66HORXSEPSDWIUONOUZNA2UNXKBA4A6VZ7PBWW3IN2SCJSPQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:12Q73mXHLEfuX4VUbJ8PTjhMoXCrYSKjJbFxqjDdqF3U99N8FXe, address:185.167.204.205:4033): protocol: write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection; write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection
Mar 10 19:36:03.323134221 (104h36m0.919905244s) [noise=true , address=89.149.201.147:4018 , piece_id=V3KLJ75HJ3NRMBP2FIM33GC45QW4U5XMN2R4IAHSX32COGBJF47Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1Ku7o12bSmJ1YR3dGetGFcWqs1wJEh53Zkg3H7ELJrt4w1hkua, address:89.149.201.147:4018): protocol: read tcp 159.223.143.39:47122->89.149.201.147:4018: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.323461923 (104h36m0.920232956s) [noise=true , address=185.167.204.205:4022 , piece_id=F5UFUTAFPRBWMSU3QKV7ZSYQR6UUC7SBUDOC7AAYGDXTC5VYUBHA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1fT9qH1igPUNfrzpwa2VkBX7heh475sNqoN4Nsf9FYSwi1VmHj, address:185.167.204.205:4022): protocol: read tcp 159.223.143.39:51226->185.167.204.205:4022: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.323503666 (104h36m0.920274689s) [seg_pos={0 0}, upload=117052]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Mar 10 19:36:03.424787357 (104h36m1.021558384s) [upload=117052]: Done uploading segments...
Mar 10 19:36:03.424860036 (104h36m1.021631063s) [upload=117052]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 2.550630457s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
5 out of 60 is failed (threshold=3) [XX.X...........X.......X....................................] duration:
2.448944316s trace:
https://jaeger.dev.storj.io/trace/78c3ad63aac6b527
FAIL (2026-03-10 20:02:15 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
:03.719970769 (104h35m1.316741801s) [noise=true , address=185.167.204.205:4033 , piece_id=6JIW66HORXSEPSDWIUONOUZNA2UNXKBA4A6VZ7PBWW3IN2SCJSPQ, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.720006314 (104h35m1.316777347s) [noise=true , address=185.167.204.205:4024 , piece_id=3CKPLBO33LOMNYBF47K2KGQFPO53ZXLMKRKWPVSR4FKQKXKMZMCA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.721245847 (104h35m1.318016883s) [noise=true , address=185.167.204.206:4046 , piece_id=5WVC4R6VY25QUMGSOAORUK3WPGIADBXLKN7ZSLJ4GQEVKMSHZPKA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.722131967 (104h35m1.318902999s) [noise=true , address=89.149.201.147:4004 , piece_id=TMKNRIR6RWRW3TUSA2MQL4LCNTJ3LKEFIT6BAVRX2V54QHNF26FA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.722154312 (104h35m1.318925349s) [noise=true , address=185.167.204.205:4028 , piece_id=IQ4CMHEIUNHIUBQV2PVFQ6BURDMWU2S55VGFP5Y7HPFSDVBY6PYA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.723422869 (104h35m1.320193903s) [noise=true , address=81.171.14.70:4047 , piece_id=DLCUNYJFEKIU3MWFXK24QGJ4GW2H5XE4Y2ARNL2W55EZUMSC4Z7Q, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.724205203 (104h35m1.320976237s) [noise=true , address=185.167.204.205:4022 , piece_id=F5UFUTAFPRBWMSU3QKV7ZSYQR6UUC7SBUDOC7AAYGDXTC5VYUBHA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.727803141 (104h35m1.324574176s) [noise=true , address=185.167.204.206:4067 , piece_id=RH2DWZ3FL7JX6EJ2WTJBPBSWDYVSHLYSLZCMMGRQXUDR6QXVZZ5A, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:04.167055423 (104h35m1.763826466s) [upload=117052]: Next returned nil segment
Mar 10 19:35:04.167072399 (104h35m1.763843424s) [upload=117052]: Waiting for error group managing segments...
Mar 10 19:35:04.350286979 (104h35m1.947058004s) [noise=true , address=81.171.14.70:4035 , piece_id=RNJYIOH4ETYQBSZFYTV3FX73J3PNFR2KBYNZEDXCVBOYMUAN5FCA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.351883208 (104h35m1.948654233s) [noise=true , address=81.171.14.70:4043 , piece_id=LSOHRZMEV5TEQUKZIJPYHVP43CIZKMXBTRVKIZIQNSSFNMAA5FLQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.356149085 (104h35m1.952920098s) [noise=true , address=185.167.204.205:4036 , piece_id=BBGCJ7U7PQDNB5NZTJWHUWXR6TFYF6D7LOR2UUBNV647YQX5JD2Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.369094269 (104h35m1.965865391s) [noise=true , address=89.149.201.147:4011 , piece_id=YHPLXK3ADD6VMYDWQ55PLROBCY2IOLOR2ZWEZ22FRRA4TXKSSWIQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.382982597 (104h35m1.979753611s) [noise=true , address=81.171.14.70:4034 , piece_id=BPXMDI54RTQVH5QKY6JCN5H3AKCPH62V76VBES4GQ6SLIZDIMMOA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.384672279 (104h35m1.981443307s) [noise=true , address=81.171.14.70:4047 , piece_id=DLCUNYJFEKIU3MWFXK24QGJ4GW2H5XE4Y2ARNL2W55EZUMSC4Z7Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.386337288 (104h35m1.98310832s) [noise=true , address=81.171.14.70:4025 , piece_id=R62A3AXM4OKUZFZJ2LB55UPB6VRGHBQLMXXVBWDTHUDWPGPIWEVQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.387583761 (104h35m1.984354793s) [noise=true , address=89.149.201.147:4009 , piece_id=XFSD3W6JWUXGXTHYE57SS5E5322JTRG7RKBI7MBF25GU7SEAKZJQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.388110220 (104h35m1.984881248s) [noise=true , address=81.171.14.70:4026 , piece_id=4CBQ4RZ44USXNYAGA3CIGNC3LBQXR675JVJX6CVTV3X7KVYPRHQA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.389451350 (104h35m1.986222362s) [noise=true , address=185.167.204.205:4012 , piece_id=7UFJIUKCHPKYZF42HTP6Q5D2FMHRSCRUQ7MH2MYJSEOZHZAJRZXA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.404093624 (104h35m2.00086464s) [noise=true , address=81.171.14.70:4048 , piece_id=4F2F4X4ZJ4FXUYQJQKB7YB3JZY53QNVRUSM3N4HQKGEYHMFYQB2A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.413111656 (104h35m2.009882687s) [noise=true , address=185.167.204.206:4071 , piece_id=ZQJLN55G37453GZS54ZDTPM7CEU5GH6LJR3BIKSJDC3C4QOKW5YQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.416183369 (104h35m2.012954404s) [noise=true , address=89.149.201.147:4014 , piece_id=WZOPRPDLFVOO6O2JYZZB75CLIAA4SVM7OEZXBX6WOJV3PUL2YSAQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.449375785 (104h35m2.046146815s) [noise=true , address=185.167.204.206:4039 , piece_id=DKKIZZMA3RAKMLUJTU6OANGBA3F6XWRD5ELVNXAGBIOZMFBVQWZQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.457716093 (104h35m2.054487108s) [noise=true , address=185.167.204.205:4024 , piece_id=3CKPLBO33LOMNYBF47K2KGQFPO53ZXLMKRKWPVSR4FKQKXKMZMCA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.491116161 (104h35m2.087887192s) [noise=true , address=185.167.204.205:4005 , piece_id=E7RO77MCAMFEAIE7ETPALCBV5S4X3DG5ZUD27ZE55RXYICIIXNEQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.507099744 (104h35m2.103870782s) [noise=true , address=185.167.204.205:4029 , piece_id=R33ONZELVLYL76YTJKOZASDJKFBACNNMC3PDUIITQRDGBWWZUNKA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.537139239 (104h35m2.133910269s) [noise=true , address=185.167.204.206:4067 , piece_id=RH2DWZ3FL7JX6EJ2WTJBPBSWDYVSHLYSLZCMMGRQXUDR6QXVZZ5A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.787100189 (104h35m2.383871224s) [noise=true , address=81.171.14.70:4033 , piece_id=ZFV477QXRX42OJIG4D5JLOYU3WERTD65PMHI5MFQLRTQJG3ENUTA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.962846444 (104h35m2.559617469s) [noise=true , address=89.149.201.147:4004 , piece_id=TMKNRIR6RWRW3TUSA2MQL4LCNTJ3LKEFIT6BAVRX2V54QHNF26FA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:08.299132133 (104h35m5.895903147s) [noise=true , address=185.167.204.205:4028 , piece_id=IQ4CMHEIUNHIUBQV2PVFQ6BURDMWU2S55VGFP5Y7HPFSDVBY6PYA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:08.481127955 (104h35m6.077898968s) [noise=true , address=185.167.204.206:4051 , piece_id=GO4OTIL5WFOS7V77FAT2NRFG2HRE2HXKMU2C3XEGDBGPQL5U5HUA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:10.314952727 (104h35m7.91172374s) [noise=true , address=185.167.204.206:4046 , piece_id=5WVC4R6VY25QUMGSOAORUK3WPGIADBXLKN7ZSLJ4GQEVKMSHZPKA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:36:03.322697363 (104h36m0.919468398s) [noise=true , address=185.167.204.206:4062 , piece_id=7NP2ZIMW6ZDXW3YHZSBO5PRTAZPLYLDWXDA4QPN5XH2C6KAGYLEA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:12BL6XLqsoUGm2FxrTgYTxLZakt2u2CAhZJG82oJ31eaE4vTtMr, address:185.167.204.206:4062): protocol: write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection; write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection
Mar 10 19:36:03.322771647 (104h36m0.919542676s) [noise=true , address=89.149.201.147:4007 , piece_id=AAJY5QBN55ENHTODH25WBVCFDT2SAQ2F27TKHFCIURCZSBMGOY6A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1Q7d4vj1KoWhDr6xd9gEfFrZ1sV3Yro2AFke2f3Ke1rXsGnvew, address:89.149.201.147:4007): protocol: read tcp 159.223.143.39:54460->89.149.201.147:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.322958851 (104h36m0.919729884s) [noise=true , address=185.167.204.205:4033 , piece_id=6JIW66HORXSEPSDWIUONOUZNA2UNXKBA4A6VZ7PBWW3IN2SCJSPQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:12Q73mXHLEfuX4VUbJ8PTjhMoXCrYSKjJbFxqjDdqF3U99N8FXe, address:185.167.204.205:4033): protocol: write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection; write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection
Mar 10 19:36:03.323134221 (104h36m0.919905244s) [noise=true , address=89.149.201.147:4018 , piece_id=V3KLJ75HJ3NRMBP2FIM33GC45QW4U5XMN2R4IAHSX32COGBJF47Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1Ku7o12bSmJ1YR3dGetGFcWqs1wJEh53Zkg3H7ELJrt4w1hkua, address:89.149.201.147:4018): protocol: read tcp 159.223.143.39:47122->89.149.201.147:4018: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.323461923 (104h36m0.920232956s) [noise=true , address=185.167.204.205:4022 , piece_id=F5UFUTAFPRBWMSU3QKV7ZSYQR6UUC7SBUDOC7AAYGDXTC5VYUBHA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1fT9qH1igPUNfrzpwa2VkBX7heh475sNqoN4Nsf9FYSwi1VmHj, address:185.167.204.205:4022): protocol: read tcp 159.223.143.39:51226->185.167.204.205:4022: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.323503666 (104h36m0.920274689s) [seg_pos={0 0}, upload=117052]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Mar 10 19:36:03.424787357 (104h36m1.021558384s) [upload=117052]: Done uploading segments...
Mar 10 19:36:03.424860036 (104h36m1.021631063s) [upload=117052]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 2.448978968s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
5 out of 60 is failed (threshold=3) [.XX.X...........X.......X...................................] duration:
2.190639172s trace:
https://jaeger.dev.storj.io/trace/78c3ad63aac6b527
FAIL (2026-03-10 20:01:03 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
:03.719970769 (104h35m1.316741801s) [noise=true , address=185.167.204.205:4033 , piece_id=6JIW66HORXSEPSDWIUONOUZNA2UNXKBA4A6VZ7PBWW3IN2SCJSPQ, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.720006314 (104h35m1.316777347s) [noise=true , address=185.167.204.205:4024 , piece_id=3CKPLBO33LOMNYBF47K2KGQFPO53ZXLMKRKWPVSR4FKQKXKMZMCA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.721245847 (104h35m1.318016883s) [noise=true , address=185.167.204.206:4046 , piece_id=5WVC4R6VY25QUMGSOAORUK3WPGIADBXLKN7ZSLJ4GQEVKMSHZPKA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.722131967 (104h35m1.318902999s) [noise=true , address=89.149.201.147:4004 , piece_id=TMKNRIR6RWRW3TUSA2MQL4LCNTJ3LKEFIT6BAVRX2V54QHNF26FA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.722154312 (104h35m1.318925349s) [noise=true , address=185.167.204.205:4028 , piece_id=IQ4CMHEIUNHIUBQV2PVFQ6BURDMWU2S55VGFP5Y7HPFSDVBY6PYA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.723422869 (104h35m1.320193903s) [noise=true , address=81.171.14.70:4047 , piece_id=DLCUNYJFEKIU3MWFXK24QGJ4GW2H5XE4Y2ARNL2W55EZUMSC4Z7Q, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.724205203 (104h35m1.320976237s) [noise=true , address=185.167.204.205:4022 , piece_id=F5UFUTAFPRBWMSU3QKV7ZSYQR6UUC7SBUDOC7AAYGDXTC5VYUBHA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.727803141 (104h35m1.324574176s) [noise=true , address=185.167.204.206:4067 , piece_id=RH2DWZ3FL7JX6EJ2WTJBPBSWDYVSHLYSLZCMMGRQXUDR6QXVZZ5A, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:04.167055423 (104h35m1.763826466s) [upload=117052]: Next returned nil segment
Mar 10 19:35:04.167072399 (104h35m1.763843424s) [upload=117052]: Waiting for error group managing segments...
Mar 10 19:35:04.350286979 (104h35m1.947058004s) [noise=true , address=81.171.14.70:4035 , piece_id=RNJYIOH4ETYQBSZFYTV3FX73J3PNFR2KBYNZEDXCVBOYMUAN5FCA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.351883208 (104h35m1.948654233s) [noise=true , address=81.171.14.70:4043 , piece_id=LSOHRZMEV5TEQUKZIJPYHVP43CIZKMXBTRVKIZIQNSSFNMAA5FLQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.356149085 (104h35m1.952920098s) [noise=true , address=185.167.204.205:4036 , piece_id=BBGCJ7U7PQDNB5NZTJWHUWXR6TFYF6D7LOR2UUBNV647YQX5JD2Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.369094269 (104h35m1.965865391s) [noise=true , address=89.149.201.147:4011 , piece_id=YHPLXK3ADD6VMYDWQ55PLROBCY2IOLOR2ZWEZ22FRRA4TXKSSWIQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.382982597 (104h35m1.979753611s) [noise=true , address=81.171.14.70:4034 , piece_id=BPXMDI54RTQVH5QKY6JCN5H3AKCPH62V76VBES4GQ6SLIZDIMMOA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.384672279 (104h35m1.981443307s) [noise=true , address=81.171.14.70:4047 , piece_id=DLCUNYJFEKIU3MWFXK24QGJ4GW2H5XE4Y2ARNL2W55EZUMSC4Z7Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.386337288 (104h35m1.98310832s) [noise=true , address=81.171.14.70:4025 , piece_id=R62A3AXM4OKUZFZJ2LB55UPB6VRGHBQLMXXVBWDTHUDWPGPIWEVQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.387583761 (104h35m1.984354793s) [noise=true , address=89.149.201.147:4009 , piece_id=XFSD3W6JWUXGXTHYE57SS5E5322JTRG7RKBI7MBF25GU7SEAKZJQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.388110220 (104h35m1.984881248s) [noise=true , address=81.171.14.70:4026 , piece_id=4CBQ4RZ44USXNYAGA3CIGNC3LBQXR675JVJX6CVTV3X7KVYPRHQA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.389451350 (104h35m1.986222362s) [noise=true , address=185.167.204.205:4012 , piece_id=7UFJIUKCHPKYZF42HTP6Q5D2FMHRSCRUQ7MH2MYJSEOZHZAJRZXA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.404093624 (104h35m2.00086464s) [noise=true , address=81.171.14.70:4048 , piece_id=4F2F4X4ZJ4FXUYQJQKB7YB3JZY53QNVRUSM3N4HQKGEYHMFYQB2A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.413111656 (104h35m2.009882687s) [noise=true , address=185.167.204.206:4071 , piece_id=ZQJLN55G37453GZS54ZDTPM7CEU5GH6LJR3BIKSJDC3C4QOKW5YQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.416183369 (104h35m2.012954404s) [noise=true , address=89.149.201.147:4014 , piece_id=WZOPRPDLFVOO6O2JYZZB75CLIAA4SVM7OEZXBX6WOJV3PUL2YSAQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.449375785 (104h35m2.046146815s) [noise=true , address=185.167.204.206:4039 , piece_id=DKKIZZMA3RAKMLUJTU6OANGBA3F6XWRD5ELVNXAGBIOZMFBVQWZQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.457716093 (104h35m2.054487108s) [noise=true , address=185.167.204.205:4024 , piece_id=3CKPLBO33LOMNYBF47K2KGQFPO53ZXLMKRKWPVSR4FKQKXKMZMCA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.491116161 (104h35m2.087887192s) [noise=true , address=185.167.204.205:4005 , piece_id=E7RO77MCAMFEAIE7ETPALCBV5S4X3DG5ZUD27ZE55RXYICIIXNEQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.507099744 (104h35m2.103870782s) [noise=true , address=185.167.204.205:4029 , piece_id=R33ONZELVLYL76YTJKOZASDJKFBACNNMC3PDUIITQRDGBWWZUNKA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.537139239 (104h35m2.133910269s) [noise=true , address=185.167.204.206:4067 , piece_id=RH2DWZ3FL7JX6EJ2WTJBPBSWDYVSHLYSLZCMMGRQXUDR6QXVZZ5A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.787100189 (104h35m2.383871224s) [noise=true , address=81.171.14.70:4033 , piece_id=ZFV477QXRX42OJIG4D5JLOYU3WERTD65PMHI5MFQLRTQJG3ENUTA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.962846444 (104h35m2.559617469s) [noise=true , address=89.149.201.147:4004 , piece_id=TMKNRIR6RWRW3TUSA2MQL4LCNTJ3LKEFIT6BAVRX2V54QHNF26FA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:08.299132133 (104h35m5.895903147s) [noise=true , address=185.167.204.205:4028 , piece_id=IQ4CMHEIUNHIUBQV2PVFQ6BURDMWU2S55VGFP5Y7HPFSDVBY6PYA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:08.481127955 (104h35m6.077898968s) [noise=true , address=185.167.204.206:4051 , piece_id=GO4OTIL5WFOS7V77FAT2NRFG2HRE2HXKMU2C3XEGDBGPQL5U5HUA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:10.314952727 (104h35m7.91172374s) [noise=true , address=185.167.204.206:4046 , piece_id=5WVC4R6VY25QUMGSOAORUK3WPGIADBXLKN7ZSLJ4GQEVKMSHZPKA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:36:03.322697363 (104h36m0.919468398s) [noise=true , address=185.167.204.206:4062 , piece_id=7NP2ZIMW6ZDXW3YHZSBO5PRTAZPLYLDWXDA4QPN5XH2C6KAGYLEA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:12BL6XLqsoUGm2FxrTgYTxLZakt2u2CAhZJG82oJ31eaE4vTtMr, address:185.167.204.206:4062): protocol: write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection; write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection
Mar 10 19:36:03.322771647 (104h36m0.919542676s) [noise=true , address=89.149.201.147:4007 , piece_id=AAJY5QBN55ENHTODH25WBVCFDT2SAQ2F27TKHFCIURCZSBMGOY6A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1Q7d4vj1KoWhDr6xd9gEfFrZ1sV3Yro2AFke2f3Ke1rXsGnvew, address:89.149.201.147:4007): protocol: read tcp 159.223.143.39:54460->89.149.201.147:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.322958851 (104h36m0.919729884s) [noise=true , address=185.167.204.205:4033 , piece_id=6JIW66HORXSEPSDWIUONOUZNA2UNXKBA4A6VZ7PBWW3IN2SCJSPQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:12Q73mXHLEfuX4VUbJ8PTjhMoXCrYSKjJbFxqjDdqF3U99N8FXe, address:185.167.204.205:4033): protocol: write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection; write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection
Mar 10 19:36:03.323134221 (104h36m0.919905244s) [noise=true , address=89.149.201.147:4018 , piece_id=V3KLJ75HJ3NRMBP2FIM33GC45QW4U5XMN2R4IAHSX32COGBJF47Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1Ku7o12bSmJ1YR3dGetGFcWqs1wJEh53Zkg3H7ELJrt4w1hkua, address:89.149.201.147:4018): protocol: read tcp 159.223.143.39:47122->89.149.201.147:4018: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.323461923 (104h36m0.920232956s) [noise=true , address=185.167.204.205:4022 , piece_id=F5UFUTAFPRBWMSU3QKV7ZSYQR6UUC7SBUDOC7AAYGDXTC5VYUBHA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1fT9qH1igPUNfrzpwa2VkBX7heh475sNqoN4Nsf9FYSwi1VmHj, address:185.167.204.205:4022): protocol: read tcp 159.223.143.39:51226->185.167.204.205:4022: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.323503666 (104h36m0.920274689s) [seg_pos={0 0}, upload=117052]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Mar 10 19:36:03.424787357 (104h36m1.021558384s) [upload=117052]: Done uploading segments...
Mar 10 19:36:03.424860036 (104h36m1.021631063s) [upload=117052]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 2.190697343s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
5 out of 60 is failed (threshold=3) [..XX.X...........X.......X..................................] duration:
2.525882691s trace:
https://jaeger.dev.storj.io/trace/78c3ad63aac6b527
FAIL (2026-03-10 20:00:36 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
:03.719970769 (104h35m1.316741801s) [noise=true , address=185.167.204.205:4033 , piece_id=6JIW66HORXSEPSDWIUONOUZNA2UNXKBA4A6VZ7PBWW3IN2SCJSPQ, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.720006314 (104h35m1.316777347s) [noise=true , address=185.167.204.205:4024 , piece_id=3CKPLBO33LOMNYBF47K2KGQFPO53ZXLMKRKWPVSR4FKQKXKMZMCA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.721245847 (104h35m1.318016883s) [noise=true , address=185.167.204.206:4046 , piece_id=5WVC4R6VY25QUMGSOAORUK3WPGIADBXLKN7ZSLJ4GQEVKMSHZPKA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.722131967 (104h35m1.318902999s) [noise=true , address=89.149.201.147:4004 , piece_id=TMKNRIR6RWRW3TUSA2MQL4LCNTJ3LKEFIT6BAVRX2V54QHNF26FA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.722154312 (104h35m1.318925349s) [noise=true , address=185.167.204.205:4028 , piece_id=IQ4CMHEIUNHIUBQV2PVFQ6BURDMWU2S55VGFP5Y7HPFSDVBY6PYA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.723422869 (104h35m1.320193903s) [noise=true , address=81.171.14.70:4047 , piece_id=DLCUNYJFEKIU3MWFXK24QGJ4GW2H5XE4Y2ARNL2W55EZUMSC4Z7Q, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.724205203 (104h35m1.320976237s) [noise=true , address=185.167.204.205:4022 , piece_id=F5UFUTAFPRBWMSU3QKV7ZSYQR6UUC7SBUDOC7AAYGDXTC5VYUBHA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.727803141 (104h35m1.324574176s) [noise=true , address=185.167.204.206:4067 , piece_id=RH2DWZ3FL7JX6EJ2WTJBPBSWDYVSHLYSLZCMMGRQXUDR6QXVZZ5A, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:04.167055423 (104h35m1.763826466s) [upload=117052]: Next returned nil segment
Mar 10 19:35:04.167072399 (104h35m1.763843424s) [upload=117052]: Waiting for error group managing segments...
Mar 10 19:35:04.350286979 (104h35m1.947058004s) [noise=true , address=81.171.14.70:4035 , piece_id=RNJYIOH4ETYQBSZFYTV3FX73J3PNFR2KBYNZEDXCVBOYMUAN5FCA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.351883208 (104h35m1.948654233s) [noise=true , address=81.171.14.70:4043 , piece_id=LSOHRZMEV5TEQUKZIJPYHVP43CIZKMXBTRVKIZIQNSSFNMAA5FLQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.356149085 (104h35m1.952920098s) [noise=true , address=185.167.204.205:4036 , piece_id=BBGCJ7U7PQDNB5NZTJWHUWXR6TFYF6D7LOR2UUBNV647YQX5JD2Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.369094269 (104h35m1.965865391s) [noise=true , address=89.149.201.147:4011 , piece_id=YHPLXK3ADD6VMYDWQ55PLROBCY2IOLOR2ZWEZ22FRRA4TXKSSWIQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.382982597 (104h35m1.979753611s) [noise=true , address=81.171.14.70:4034 , piece_id=BPXMDI54RTQVH5QKY6JCN5H3AKCPH62V76VBES4GQ6SLIZDIMMOA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.384672279 (104h35m1.981443307s) [noise=true , address=81.171.14.70:4047 , piece_id=DLCUNYJFEKIU3MWFXK24QGJ4GW2H5XE4Y2ARNL2W55EZUMSC4Z7Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.386337288 (104h35m1.98310832s) [noise=true , address=81.171.14.70:4025 , piece_id=R62A3AXM4OKUZFZJ2LB55UPB6VRGHBQLMXXVBWDTHUDWPGPIWEVQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.387583761 (104h35m1.984354793s) [noise=true , address=89.149.201.147:4009 , piece_id=XFSD3W6JWUXGXTHYE57SS5E5322JTRG7RKBI7MBF25GU7SEAKZJQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.388110220 (104h35m1.984881248s) [noise=true , address=81.171.14.70:4026 , piece_id=4CBQ4RZ44USXNYAGA3CIGNC3LBQXR675JVJX6CVTV3X7KVYPRHQA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.389451350 (104h35m1.986222362s) [noise=true , address=185.167.204.205:4012 , piece_id=7UFJIUKCHPKYZF42HTP6Q5D2FMHRSCRUQ7MH2MYJSEOZHZAJRZXA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.404093624 (104h35m2.00086464s) [noise=true , address=81.171.14.70:4048 , piece_id=4F2F4X4ZJ4FXUYQJQKB7YB3JZY53QNVRUSM3N4HQKGEYHMFYQB2A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.413111656 (104h35m2.009882687s) [noise=true , address=185.167.204.206:4071 , piece_id=ZQJLN55G37453GZS54ZDTPM7CEU5GH6LJR3BIKSJDC3C4QOKW5YQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.416183369 (104h35m2.012954404s) [noise=true , address=89.149.201.147:4014 , piece_id=WZOPRPDLFVOO6O2JYZZB75CLIAA4SVM7OEZXBX6WOJV3PUL2YSAQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.449375785 (104h35m2.046146815s) [noise=true , address=185.167.204.206:4039 , piece_id=DKKIZZMA3RAKMLUJTU6OANGBA3F6XWRD5ELVNXAGBIOZMFBVQWZQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.457716093 (104h35m2.054487108s) [noise=true , address=185.167.204.205:4024 , piece_id=3CKPLBO33LOMNYBF47K2KGQFPO53ZXLMKRKWPVSR4FKQKXKMZMCA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.491116161 (104h35m2.087887192s) [noise=true , address=185.167.204.205:4005 , piece_id=E7RO77MCAMFEAIE7ETPALCBV5S4X3DG5ZUD27ZE55RXYICIIXNEQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.507099744 (104h35m2.103870782s) [noise=true , address=185.167.204.205:4029 , piece_id=R33ONZELVLYL76YTJKOZASDJKFBACNNMC3PDUIITQRDGBWWZUNKA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.537139239 (104h35m2.133910269s) [noise=true , address=185.167.204.206:4067 , piece_id=RH2DWZ3FL7JX6EJ2WTJBPBSWDYVSHLYSLZCMMGRQXUDR6QXVZZ5A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.787100189 (104h35m2.383871224s) [noise=true , address=81.171.14.70:4033 , piece_id=ZFV477QXRX42OJIG4D5JLOYU3WERTD65PMHI5MFQLRTQJG3ENUTA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.962846444 (104h35m2.559617469s) [noise=true , address=89.149.201.147:4004 , piece_id=TMKNRIR6RWRW3TUSA2MQL4LCNTJ3LKEFIT6BAVRX2V54QHNF26FA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:08.299132133 (104h35m5.895903147s) [noise=true , address=185.167.204.205:4028 , piece_id=IQ4CMHEIUNHIUBQV2PVFQ6BURDMWU2S55VGFP5Y7HPFSDVBY6PYA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:08.481127955 (104h35m6.077898968s) [noise=true , address=185.167.204.206:4051 , piece_id=GO4OTIL5WFOS7V77FAT2NRFG2HRE2HXKMU2C3XEGDBGPQL5U5HUA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:10.314952727 (104h35m7.91172374s) [noise=true , address=185.167.204.206:4046 , piece_id=5WVC4R6VY25QUMGSOAORUK3WPGIADBXLKN7ZSLJ4GQEVKMSHZPKA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:36:03.322697363 (104h36m0.919468398s) [noise=true , address=185.167.204.206:4062 , piece_id=7NP2ZIMW6ZDXW3YHZSBO5PRTAZPLYLDWXDA4QPN5XH2C6KAGYLEA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:12BL6XLqsoUGm2FxrTgYTxLZakt2u2CAhZJG82oJ31eaE4vTtMr, address:185.167.204.206:4062): protocol: write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection; write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection
Mar 10 19:36:03.322771647 (104h36m0.919542676s) [noise=true , address=89.149.201.147:4007 , piece_id=AAJY5QBN55ENHTODH25WBVCFDT2SAQ2F27TKHFCIURCZSBMGOY6A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1Q7d4vj1KoWhDr6xd9gEfFrZ1sV3Yro2AFke2f3Ke1rXsGnvew, address:89.149.201.147:4007): protocol: read tcp 159.223.143.39:54460->89.149.201.147:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.322958851 (104h36m0.919729884s) [noise=true , address=185.167.204.205:4033 , piece_id=6JIW66HORXSEPSDWIUONOUZNA2UNXKBA4A6VZ7PBWW3IN2SCJSPQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:12Q73mXHLEfuX4VUbJ8PTjhMoXCrYSKjJbFxqjDdqF3U99N8FXe, address:185.167.204.205:4033): protocol: write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection; write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection
Mar 10 19:36:03.323134221 (104h36m0.919905244s) [noise=true , address=89.149.201.147:4018 , piece_id=V3KLJ75HJ3NRMBP2FIM33GC45QW4U5XMN2R4IAHSX32COGBJF47Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1Ku7o12bSmJ1YR3dGetGFcWqs1wJEh53Zkg3H7ELJrt4w1hkua, address:89.149.201.147:4018): protocol: read tcp 159.223.143.39:47122->89.149.201.147:4018: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.323461923 (104h36m0.920232956s) [noise=true , address=185.167.204.205:4022 , piece_id=F5UFUTAFPRBWMSU3QKV7ZSYQR6UUC7SBUDOC7AAYGDXTC5VYUBHA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1fT9qH1igPUNfrzpwa2VkBX7heh475sNqoN4Nsf9FYSwi1VmHj, address:185.167.204.205:4022): protocol: read tcp 159.223.143.39:51226->185.167.204.205:4022: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.323503666 (104h36m0.920274689s) [seg_pos={0 0}, upload=117052]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Mar 10 19:36:03.424787357 (104h36m1.021558384s) [upload=117052]: Done uploading segments...
Mar 10 19:36:03.424860036 (104h36m1.021631063s) [upload=117052]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 2.525913231s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
5 out of 60 is failed (threshold=3) [...XX.X...........X.......X.................................] duration:
2.468213562s trace:
https://jaeger.dev.storj.io/trace/78c3ad63aac6b527
FAIL (2026-03-10 19:59:43 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
:03.719970769 (104h35m1.316741801s) [noise=true , address=185.167.204.205:4033 , piece_id=6JIW66HORXSEPSDWIUONOUZNA2UNXKBA4A6VZ7PBWW3IN2SCJSPQ, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.720006314 (104h35m1.316777347s) [noise=true , address=185.167.204.205:4024 , piece_id=3CKPLBO33LOMNYBF47K2KGQFPO53ZXLMKRKWPVSR4FKQKXKMZMCA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.721245847 (104h35m1.318016883s) [noise=true , address=185.167.204.206:4046 , piece_id=5WVC4R6VY25QUMGSOAORUK3WPGIADBXLKN7ZSLJ4GQEVKMSHZPKA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.722131967 (104h35m1.318902999s) [noise=true , address=89.149.201.147:4004 , piece_id=TMKNRIR6RWRW3TUSA2MQL4LCNTJ3LKEFIT6BAVRX2V54QHNF26FA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.722154312 (104h35m1.318925349s) [noise=true , address=185.167.204.205:4028 , piece_id=IQ4CMHEIUNHIUBQV2PVFQ6BURDMWU2S55VGFP5Y7HPFSDVBY6PYA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.723422869 (104h35m1.320193903s) [noise=true , address=81.171.14.70:4047 , piece_id=DLCUNYJFEKIU3MWFXK24QGJ4GW2H5XE4Y2ARNL2W55EZUMSC4Z7Q, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.724205203 (104h35m1.320976237s) [noise=true , address=185.167.204.205:4022 , piece_id=F5UFUTAFPRBWMSU3QKV7ZSYQR6UUC7SBUDOC7AAYGDXTC5VYUBHA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.727803141 (104h35m1.324574176s) [noise=true , address=185.167.204.206:4067 , piece_id=RH2DWZ3FL7JX6EJ2WTJBPBSWDYVSHLYSLZCMMGRQXUDR6QXVZZ5A, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:04.167055423 (104h35m1.763826466s) [upload=117052]: Next returned nil segment
Mar 10 19:35:04.167072399 (104h35m1.763843424s) [upload=117052]: Waiting for error group managing segments...
Mar 10 19:35:04.350286979 (104h35m1.947058004s) [noise=true , address=81.171.14.70:4035 , piece_id=RNJYIOH4ETYQBSZFYTV3FX73J3PNFR2KBYNZEDXCVBOYMUAN5FCA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.351883208 (104h35m1.948654233s) [noise=true , address=81.171.14.70:4043 , piece_id=LSOHRZMEV5TEQUKZIJPYHVP43CIZKMXBTRVKIZIQNSSFNMAA5FLQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.356149085 (104h35m1.952920098s) [noise=true , address=185.167.204.205:4036 , piece_id=BBGCJ7U7PQDNB5NZTJWHUWXR6TFYF6D7LOR2UUBNV647YQX5JD2Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.369094269 (104h35m1.965865391s) [noise=true , address=89.149.201.147:4011 , piece_id=YHPLXK3ADD6VMYDWQ55PLROBCY2IOLOR2ZWEZ22FRRA4TXKSSWIQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.382982597 (104h35m1.979753611s) [noise=true , address=81.171.14.70:4034 , piece_id=BPXMDI54RTQVH5QKY6JCN5H3AKCPH62V76VBES4GQ6SLIZDIMMOA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.384672279 (104h35m1.981443307s) [noise=true , address=81.171.14.70:4047 , piece_id=DLCUNYJFEKIU3MWFXK24QGJ4GW2H5XE4Y2ARNL2W55EZUMSC4Z7Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.386337288 (104h35m1.98310832s) [noise=true , address=81.171.14.70:4025 , piece_id=R62A3AXM4OKUZFZJ2LB55UPB6VRGHBQLMXXVBWDTHUDWPGPIWEVQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.387583761 (104h35m1.984354793s) [noise=true , address=89.149.201.147:4009 , piece_id=XFSD3W6JWUXGXTHYE57SS5E5322JTRG7RKBI7MBF25GU7SEAKZJQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.388110220 (104h35m1.984881248s) [noise=true , address=81.171.14.70:4026 , piece_id=4CBQ4RZ44USXNYAGA3CIGNC3LBQXR675JVJX6CVTV3X7KVYPRHQA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.389451350 (104h35m1.986222362s) [noise=true , address=185.167.204.205:4012 , piece_id=7UFJIUKCHPKYZF42HTP6Q5D2FMHRSCRUQ7MH2MYJSEOZHZAJRZXA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.404093624 (104h35m2.00086464s) [noise=true , address=81.171.14.70:4048 , piece_id=4F2F4X4ZJ4FXUYQJQKB7YB3JZY53QNVRUSM3N4HQKGEYHMFYQB2A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.413111656 (104h35m2.009882687s) [noise=true , address=185.167.204.206:4071 , piece_id=ZQJLN55G37453GZS54ZDTPM7CEU5GH6LJR3BIKSJDC3C4QOKW5YQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.416183369 (104h35m2.012954404s) [noise=true , address=89.149.201.147:4014 , piece_id=WZOPRPDLFVOO6O2JYZZB75CLIAA4SVM7OEZXBX6WOJV3PUL2YSAQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.449375785 (104h35m2.046146815s) [noise=true , address=185.167.204.206:4039 , piece_id=DKKIZZMA3RAKMLUJTU6OANGBA3F6XWRD5ELVNXAGBIOZMFBVQWZQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.457716093 (104h35m2.054487108s) [noise=true , address=185.167.204.205:4024 , piece_id=3CKPLBO33LOMNYBF47K2KGQFPO53ZXLMKRKWPVSR4FKQKXKMZMCA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.491116161 (104h35m2.087887192s) [noise=true , address=185.167.204.205:4005 , piece_id=E7RO77MCAMFEAIE7ETPALCBV5S4X3DG5ZUD27ZE55RXYICIIXNEQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.507099744 (104h35m2.103870782s) [noise=true , address=185.167.204.205:4029 , piece_id=R33ONZELVLYL76YTJKOZASDJKFBACNNMC3PDUIITQRDGBWWZUNKA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.537139239 (104h35m2.133910269s) [noise=true , address=185.167.204.206:4067 , piece_id=RH2DWZ3FL7JX6EJ2WTJBPBSWDYVSHLYSLZCMMGRQXUDR6QXVZZ5A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.787100189 (104h35m2.383871224s) [noise=true , address=81.171.14.70:4033 , piece_id=ZFV477QXRX42OJIG4D5JLOYU3WERTD65PMHI5MFQLRTQJG3ENUTA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.962846444 (104h35m2.559617469s) [noise=true , address=89.149.201.147:4004 , piece_id=TMKNRIR6RWRW3TUSA2MQL4LCNTJ3LKEFIT6BAVRX2V54QHNF26FA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:08.299132133 (104h35m5.895903147s) [noise=true , address=185.167.204.205:4028 , piece_id=IQ4CMHEIUNHIUBQV2PVFQ6BURDMWU2S55VGFP5Y7HPFSDVBY6PYA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:08.481127955 (104h35m6.077898968s) [noise=true , address=185.167.204.206:4051 , piece_id=GO4OTIL5WFOS7V77FAT2NRFG2HRE2HXKMU2C3XEGDBGPQL5U5HUA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:10.314952727 (104h35m7.91172374s) [noise=true , address=185.167.204.206:4046 , piece_id=5WVC4R6VY25QUMGSOAORUK3WPGIADBXLKN7ZSLJ4GQEVKMSHZPKA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:36:03.322697363 (104h36m0.919468398s) [noise=true , address=185.167.204.206:4062 , piece_id=7NP2ZIMW6ZDXW3YHZSBO5PRTAZPLYLDWXDA4QPN5XH2C6KAGYLEA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:12BL6XLqsoUGm2FxrTgYTxLZakt2u2CAhZJG82oJ31eaE4vTtMr, address:185.167.204.206:4062): protocol: write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection; write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection
Mar 10 19:36:03.322771647 (104h36m0.919542676s) [noise=true , address=89.149.201.147:4007 , piece_id=AAJY5QBN55ENHTODH25WBVCFDT2SAQ2F27TKHFCIURCZSBMGOY6A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1Q7d4vj1KoWhDr6xd9gEfFrZ1sV3Yro2AFke2f3Ke1rXsGnvew, address:89.149.201.147:4007): protocol: read tcp 159.223.143.39:54460->89.149.201.147:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.322958851 (104h36m0.919729884s) [noise=true , address=185.167.204.205:4033 , piece_id=6JIW66HORXSEPSDWIUONOUZNA2UNXKBA4A6VZ7PBWW3IN2SCJSPQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:12Q73mXHLEfuX4VUbJ8PTjhMoXCrYSKjJbFxqjDdqF3U99N8FXe, address:185.167.204.205:4033): protocol: write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection; write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection
Mar 10 19:36:03.323134221 (104h36m0.919905244s) [noise=true , address=89.149.201.147:4018 , piece_id=V3KLJ75HJ3NRMBP2FIM33GC45QW4U5XMN2R4IAHSX32COGBJF47Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1Ku7o12bSmJ1YR3dGetGFcWqs1wJEh53Zkg3H7ELJrt4w1hkua, address:89.149.201.147:4018): protocol: read tcp 159.223.143.39:47122->89.149.201.147:4018: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.323461923 (104h36m0.920232956s) [noise=true , address=185.167.204.205:4022 , piece_id=F5UFUTAFPRBWMSU3QKV7ZSYQR6UUC7SBUDOC7AAYGDXTC5VYUBHA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1fT9qH1igPUNfrzpwa2VkBX7heh475sNqoN4Nsf9FYSwi1VmHj, address:185.167.204.205:4022): protocol: read tcp 159.223.143.39:51226->185.167.204.205:4022: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.323503666 (104h36m0.920274689s) [seg_pos={0 0}, upload=117052]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Mar 10 19:36:03.424787357 (104h36m1.021558384s) [upload=117052]: Done uploading segments...
Mar 10 19:36:03.424860036 (104h36m1.021631063s) [upload=117052]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 2.468244327s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
5 out of 60 is failed (threshold=3) [....XX.X...........X.......X................................] duration:
2.604067018s trace:
https://jaeger.dev.storj.io/trace/78c3ad63aac6b527
FAIL (2026-03-10 19:59:13 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
:03.719970769 (104h35m1.316741801s) [noise=true , address=185.167.204.205:4033 , piece_id=6JIW66HORXSEPSDWIUONOUZNA2UNXKBA4A6VZ7PBWW3IN2SCJSPQ, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.720006314 (104h35m1.316777347s) [noise=true , address=185.167.204.205:4024 , piece_id=3CKPLBO33LOMNYBF47K2KGQFPO53ZXLMKRKWPVSR4FKQKXKMZMCA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.721245847 (104h35m1.318016883s) [noise=true , address=185.167.204.206:4046 , piece_id=5WVC4R6VY25QUMGSOAORUK3WPGIADBXLKN7ZSLJ4GQEVKMSHZPKA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.722131967 (104h35m1.318902999s) [noise=true , address=89.149.201.147:4004 , piece_id=TMKNRIR6RWRW3TUSA2MQL4LCNTJ3LKEFIT6BAVRX2V54QHNF26FA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.722154312 (104h35m1.318925349s) [noise=true , address=185.167.204.205:4028 , piece_id=IQ4CMHEIUNHIUBQV2PVFQ6BURDMWU2S55VGFP5Y7HPFSDVBY6PYA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.723422869 (104h35m1.320193903s) [noise=true , address=81.171.14.70:4047 , piece_id=DLCUNYJFEKIU3MWFXK24QGJ4GW2H5XE4Y2ARNL2W55EZUMSC4Z7Q, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.724205203 (104h35m1.320976237s) [noise=true , address=185.167.204.205:4022 , piece_id=F5UFUTAFPRBWMSU3QKV7ZSYQR6UUC7SBUDOC7AAYGDXTC5VYUBHA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.727803141 (104h35m1.324574176s) [noise=true , address=185.167.204.206:4067 , piece_id=RH2DWZ3FL7JX6EJ2WTJBPBSWDYVSHLYSLZCMMGRQXUDR6QXVZZ5A, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:04.167055423 (104h35m1.763826466s) [upload=117052]: Next returned nil segment
Mar 10 19:35:04.167072399 (104h35m1.763843424s) [upload=117052]: Waiting for error group managing segments...
Mar 10 19:35:04.350286979 (104h35m1.947058004s) [noise=true , address=81.171.14.70:4035 , piece_id=RNJYIOH4ETYQBSZFYTV3FX73J3PNFR2KBYNZEDXCVBOYMUAN5FCA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.351883208 (104h35m1.948654233s) [noise=true , address=81.171.14.70:4043 , piece_id=LSOHRZMEV5TEQUKZIJPYHVP43CIZKMXBTRVKIZIQNSSFNMAA5FLQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.356149085 (104h35m1.952920098s) [noise=true , address=185.167.204.205:4036 , piece_id=BBGCJ7U7PQDNB5NZTJWHUWXR6TFYF6D7LOR2UUBNV647YQX5JD2Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.369094269 (104h35m1.965865391s) [noise=true , address=89.149.201.147:4011 , piece_id=YHPLXK3ADD6VMYDWQ55PLROBCY2IOLOR2ZWEZ22FRRA4TXKSSWIQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.382982597 (104h35m1.979753611s) [noise=true , address=81.171.14.70:4034 , piece_id=BPXMDI54RTQVH5QKY6JCN5H3AKCPH62V76VBES4GQ6SLIZDIMMOA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.384672279 (104h35m1.981443307s) [noise=true , address=81.171.14.70:4047 , piece_id=DLCUNYJFEKIU3MWFXK24QGJ4GW2H5XE4Y2ARNL2W55EZUMSC4Z7Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.386337288 (104h35m1.98310832s) [noise=true , address=81.171.14.70:4025 , piece_id=R62A3AXM4OKUZFZJ2LB55UPB6VRGHBQLMXXVBWDTHUDWPGPIWEVQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.387583761 (104h35m1.984354793s) [noise=true , address=89.149.201.147:4009 , piece_id=XFSD3W6JWUXGXTHYE57SS5E5322JTRG7RKBI7MBF25GU7SEAKZJQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.388110220 (104h35m1.984881248s) [noise=true , address=81.171.14.70:4026 , piece_id=4CBQ4RZ44USXNYAGA3CIGNC3LBQXR675JVJX6CVTV3X7KVYPRHQA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.389451350 (104h35m1.986222362s) [noise=true , address=185.167.204.205:4012 , piece_id=7UFJIUKCHPKYZF42HTP6Q5D2FMHRSCRUQ7MH2MYJSEOZHZAJRZXA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.404093624 (104h35m2.00086464s) [noise=true , address=81.171.14.70:4048 , piece_id=4F2F4X4ZJ4FXUYQJQKB7YB3JZY53QNVRUSM3N4HQKGEYHMFYQB2A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.413111656 (104h35m2.009882687s) [noise=true , address=185.167.204.206:4071 , piece_id=ZQJLN55G37453GZS54ZDTPM7CEU5GH6LJR3BIKSJDC3C4QOKW5YQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.416183369 (104h35m2.012954404s) [noise=true , address=89.149.201.147:4014 , piece_id=WZOPRPDLFVOO6O2JYZZB75CLIAA4SVM7OEZXBX6WOJV3PUL2YSAQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.449375785 (104h35m2.046146815s) [noise=true , address=185.167.204.206:4039 , piece_id=DKKIZZMA3RAKMLUJTU6OANGBA3F6XWRD5ELVNXAGBIOZMFBVQWZQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.457716093 (104h35m2.054487108s) [noise=true , address=185.167.204.205:4024 , piece_id=3CKPLBO33LOMNYBF47K2KGQFPO53ZXLMKRKWPVSR4FKQKXKMZMCA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.491116161 (104h35m2.087887192s) [noise=true , address=185.167.204.205:4005 , piece_id=E7RO77MCAMFEAIE7ETPALCBV5S4X3DG5ZUD27ZE55RXYICIIXNEQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.507099744 (104h35m2.103870782s) [noise=true , address=185.167.204.205:4029 , piece_id=R33ONZELVLYL76YTJKOZASDJKFBACNNMC3PDUIITQRDGBWWZUNKA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.537139239 (104h35m2.133910269s) [noise=true , address=185.167.204.206:4067 , piece_id=RH2DWZ3FL7JX6EJ2WTJBPBSWDYVSHLYSLZCMMGRQXUDR6QXVZZ5A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.787100189 (104h35m2.383871224s) [noise=true , address=81.171.14.70:4033 , piece_id=ZFV477QXRX42OJIG4D5JLOYU3WERTD65PMHI5MFQLRTQJG3ENUTA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.962846444 (104h35m2.559617469s) [noise=true , address=89.149.201.147:4004 , piece_id=TMKNRIR6RWRW3TUSA2MQL4LCNTJ3LKEFIT6BAVRX2V54QHNF26FA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:08.299132133 (104h35m5.895903147s) [noise=true , address=185.167.204.205:4028 , piece_id=IQ4CMHEIUNHIUBQV2PVFQ6BURDMWU2S55VGFP5Y7HPFSDVBY6PYA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:08.481127955 (104h35m6.077898968s) [noise=true , address=185.167.204.206:4051 , piece_id=GO4OTIL5WFOS7V77FAT2NRFG2HRE2HXKMU2C3XEGDBGPQL5U5HUA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:10.314952727 (104h35m7.91172374s) [noise=true , address=185.167.204.206:4046 , piece_id=5WVC4R6VY25QUMGSOAORUK3WPGIADBXLKN7ZSLJ4GQEVKMSHZPKA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:36:03.322697363 (104h36m0.919468398s) [noise=true , address=185.167.204.206:4062 , piece_id=7NP2ZIMW6ZDXW3YHZSBO5PRTAZPLYLDWXDA4QPN5XH2C6KAGYLEA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:12BL6XLqsoUGm2FxrTgYTxLZakt2u2CAhZJG82oJ31eaE4vTtMr, address:185.167.204.206:4062): protocol: write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection; write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection
Mar 10 19:36:03.322771647 (104h36m0.919542676s) [noise=true , address=89.149.201.147:4007 , piece_id=AAJY5QBN55ENHTODH25WBVCFDT2SAQ2F27TKHFCIURCZSBMGOY6A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1Q7d4vj1KoWhDr6xd9gEfFrZ1sV3Yro2AFke2f3Ke1rXsGnvew, address:89.149.201.147:4007): protocol: read tcp 159.223.143.39:54460->89.149.201.147:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.322958851 (104h36m0.919729884s) [noise=true , address=185.167.204.205:4033 , piece_id=6JIW66HORXSEPSDWIUONOUZNA2UNXKBA4A6VZ7PBWW3IN2SCJSPQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:12Q73mXHLEfuX4VUbJ8PTjhMoXCrYSKjJbFxqjDdqF3U99N8FXe, address:185.167.204.205:4033): protocol: write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection; write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection
Mar 10 19:36:03.323134221 (104h36m0.919905244s) [noise=true , address=89.149.201.147:4018 , piece_id=V3KLJ75HJ3NRMBP2FIM33GC45QW4U5XMN2R4IAHSX32COGBJF47Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1Ku7o12bSmJ1YR3dGetGFcWqs1wJEh53Zkg3H7ELJrt4w1hkua, address:89.149.201.147:4018): protocol: read tcp 159.223.143.39:47122->89.149.201.147:4018: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.323461923 (104h36m0.920232956s) [noise=true , address=185.167.204.205:4022 , piece_id=F5UFUTAFPRBWMSU3QKV7ZSYQR6UUC7SBUDOC7AAYGDXTC5VYUBHA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1fT9qH1igPUNfrzpwa2VkBX7heh475sNqoN4Nsf9FYSwi1VmHj, address:185.167.204.205:4022): protocol: read tcp 159.223.143.39:51226->185.167.204.205:4022: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.323503666 (104h36m0.920274689s) [seg_pos={0 0}, upload=117052]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Mar 10 19:36:03.424787357 (104h36m1.021558384s) [upload=117052]: Done uploading segments...
Mar 10 19:36:03.424860036 (104h36m1.021631063s) [upload=117052]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 2.604118844s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
5 out of 60 is failed (threshold=3) [.....XX.X...........X.......X...............................] duration:
2.580477901s trace:
https://jaeger.dev.storj.io/trace/78c3ad63aac6b527
FAIL (2026-03-10 19:57:59 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
:03.719970769 (104h35m1.316741801s) [noise=true , address=185.167.204.205:4033 , piece_id=6JIW66HORXSEPSDWIUONOUZNA2UNXKBA4A6VZ7PBWW3IN2SCJSPQ, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.720006314 (104h35m1.316777347s) [noise=true , address=185.167.204.205:4024 , piece_id=3CKPLBO33LOMNYBF47K2KGQFPO53ZXLMKRKWPVSR4FKQKXKMZMCA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.721245847 (104h35m1.318016883s) [noise=true , address=185.167.204.206:4046 , piece_id=5WVC4R6VY25QUMGSOAORUK3WPGIADBXLKN7ZSLJ4GQEVKMSHZPKA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.722131967 (104h35m1.318902999s) [noise=true , address=89.149.201.147:4004 , piece_id=TMKNRIR6RWRW3TUSA2MQL4LCNTJ3LKEFIT6BAVRX2V54QHNF26FA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.722154312 (104h35m1.318925349s) [noise=true , address=185.167.204.205:4028 , piece_id=IQ4CMHEIUNHIUBQV2PVFQ6BURDMWU2S55VGFP5Y7HPFSDVBY6PYA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.723422869 (104h35m1.320193903s) [noise=true , address=81.171.14.70:4047 , piece_id=DLCUNYJFEKIU3MWFXK24QGJ4GW2H5XE4Y2ARNL2W55EZUMSC4Z7Q, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.724205203 (104h35m1.320976237s) [noise=true , address=185.167.204.205:4022 , piece_id=F5UFUTAFPRBWMSU3QKV7ZSYQR6UUC7SBUDOC7AAYGDXTC5VYUBHA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.727803141 (104h35m1.324574176s) [noise=true , address=185.167.204.206:4067 , piece_id=RH2DWZ3FL7JX6EJ2WTJBPBSWDYVSHLYSLZCMMGRQXUDR6QXVZZ5A, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:04.167055423 (104h35m1.763826466s) [upload=117052]: Next returned nil segment
Mar 10 19:35:04.167072399 (104h35m1.763843424s) [upload=117052]: Waiting for error group managing segments...
Mar 10 19:35:04.350286979 (104h35m1.947058004s) [noise=true , address=81.171.14.70:4035 , piece_id=RNJYIOH4ETYQBSZFYTV3FX73J3PNFR2KBYNZEDXCVBOYMUAN5FCA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.351883208 (104h35m1.948654233s) [noise=true , address=81.171.14.70:4043 , piece_id=LSOHRZMEV5TEQUKZIJPYHVP43CIZKMXBTRVKIZIQNSSFNMAA5FLQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.356149085 (104h35m1.952920098s) [noise=true , address=185.167.204.205:4036 , piece_id=BBGCJ7U7PQDNB5NZTJWHUWXR6TFYF6D7LOR2UUBNV647YQX5JD2Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.369094269 (104h35m1.965865391s) [noise=true , address=89.149.201.147:4011 , piece_id=YHPLXK3ADD6VMYDWQ55PLROBCY2IOLOR2ZWEZ22FRRA4TXKSSWIQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.382982597 (104h35m1.979753611s) [noise=true , address=81.171.14.70:4034 , piece_id=BPXMDI54RTQVH5QKY6JCN5H3AKCPH62V76VBES4GQ6SLIZDIMMOA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.384672279 (104h35m1.981443307s) [noise=true , address=81.171.14.70:4047 , piece_id=DLCUNYJFEKIU3MWFXK24QGJ4GW2H5XE4Y2ARNL2W55EZUMSC4Z7Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.386337288 (104h35m1.98310832s) [noise=true , address=81.171.14.70:4025 , piece_id=R62A3AXM4OKUZFZJ2LB55UPB6VRGHBQLMXXVBWDTHUDWPGPIWEVQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.387583761 (104h35m1.984354793s) [noise=true , address=89.149.201.147:4009 , piece_id=XFSD3W6JWUXGXTHYE57SS5E5322JTRG7RKBI7MBF25GU7SEAKZJQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.388110220 (104h35m1.984881248s) [noise=true , address=81.171.14.70:4026 , piece_id=4CBQ4RZ44USXNYAGA3CIGNC3LBQXR675JVJX6CVTV3X7KVYPRHQA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.389451350 (104h35m1.986222362s) [noise=true , address=185.167.204.205:4012 , piece_id=7UFJIUKCHPKYZF42HTP6Q5D2FMHRSCRUQ7MH2MYJSEOZHZAJRZXA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.404093624 (104h35m2.00086464s) [noise=true , address=81.171.14.70:4048 , piece_id=4F2F4X4ZJ4FXUYQJQKB7YB3JZY53QNVRUSM3N4HQKGEYHMFYQB2A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.413111656 (104h35m2.009882687s) [noise=true , address=185.167.204.206:4071 , piece_id=ZQJLN55G37453GZS54ZDTPM7CEU5GH6LJR3BIKSJDC3C4QOKW5YQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.416183369 (104h35m2.012954404s) [noise=true , address=89.149.201.147:4014 , piece_id=WZOPRPDLFVOO6O2JYZZB75CLIAA4SVM7OEZXBX6WOJV3PUL2YSAQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.449375785 (104h35m2.046146815s) [noise=true , address=185.167.204.206:4039 , piece_id=DKKIZZMA3RAKMLUJTU6OANGBA3F6XWRD5ELVNXAGBIOZMFBVQWZQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.457716093 (104h35m2.054487108s) [noise=true , address=185.167.204.205:4024 , piece_id=3CKPLBO33LOMNYBF47K2KGQFPO53ZXLMKRKWPVSR4FKQKXKMZMCA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.491116161 (104h35m2.087887192s) [noise=true , address=185.167.204.205:4005 , piece_id=E7RO77MCAMFEAIE7ETPALCBV5S4X3DG5ZUD27ZE55RXYICIIXNEQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.507099744 (104h35m2.103870782s) [noise=true , address=185.167.204.205:4029 , piece_id=R33ONZELVLYL76YTJKOZASDJKFBACNNMC3PDUIITQRDGBWWZUNKA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.537139239 (104h35m2.133910269s) [noise=true , address=185.167.204.206:4067 , piece_id=RH2DWZ3FL7JX6EJ2WTJBPBSWDYVSHLYSLZCMMGRQXUDR6QXVZZ5A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.787100189 (104h35m2.383871224s) [noise=true , address=81.171.14.70:4033 , piece_id=ZFV477QXRX42OJIG4D5JLOYU3WERTD65PMHI5MFQLRTQJG3ENUTA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.962846444 (104h35m2.559617469s) [noise=true , address=89.149.201.147:4004 , piece_id=TMKNRIR6RWRW3TUSA2MQL4LCNTJ3LKEFIT6BAVRX2V54QHNF26FA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:08.299132133 (104h35m5.895903147s) [noise=true , address=185.167.204.205:4028 , piece_id=IQ4CMHEIUNHIUBQV2PVFQ6BURDMWU2S55VGFP5Y7HPFSDVBY6PYA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:08.481127955 (104h35m6.077898968s) [noise=true , address=185.167.204.206:4051 , piece_id=GO4OTIL5WFOS7V77FAT2NRFG2HRE2HXKMU2C3XEGDBGPQL5U5HUA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:10.314952727 (104h35m7.91172374s) [noise=true , address=185.167.204.206:4046 , piece_id=5WVC4R6VY25QUMGSOAORUK3WPGIADBXLKN7ZSLJ4GQEVKMSHZPKA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:36:03.322697363 (104h36m0.919468398s) [noise=true , address=185.167.204.206:4062 , piece_id=7NP2ZIMW6ZDXW3YHZSBO5PRTAZPLYLDWXDA4QPN5XH2C6KAGYLEA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:12BL6XLqsoUGm2FxrTgYTxLZakt2u2CAhZJG82oJ31eaE4vTtMr, address:185.167.204.206:4062): protocol: write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection; write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection
Mar 10 19:36:03.322771647 (104h36m0.919542676s) [noise=true , address=89.149.201.147:4007 , piece_id=AAJY5QBN55ENHTODH25WBVCFDT2SAQ2F27TKHFCIURCZSBMGOY6A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1Q7d4vj1KoWhDr6xd9gEfFrZ1sV3Yro2AFke2f3Ke1rXsGnvew, address:89.149.201.147:4007): protocol: read tcp 159.223.143.39:54460->89.149.201.147:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.322958851 (104h36m0.919729884s) [noise=true , address=185.167.204.205:4033 , piece_id=6JIW66HORXSEPSDWIUONOUZNA2UNXKBA4A6VZ7PBWW3IN2SCJSPQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:12Q73mXHLEfuX4VUbJ8PTjhMoXCrYSKjJbFxqjDdqF3U99N8FXe, address:185.167.204.205:4033): protocol: write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection; write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection
Mar 10 19:36:03.323134221 (104h36m0.919905244s) [noise=true , address=89.149.201.147:4018 , piece_id=V3KLJ75HJ3NRMBP2FIM33GC45QW4U5XMN2R4IAHSX32COGBJF47Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1Ku7o12bSmJ1YR3dGetGFcWqs1wJEh53Zkg3H7ELJrt4w1hkua, address:89.149.201.147:4018): protocol: read tcp 159.223.143.39:47122->89.149.201.147:4018: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.323461923 (104h36m0.920232956s) [noise=true , address=185.167.204.205:4022 , piece_id=F5UFUTAFPRBWMSU3QKV7ZSYQR6UUC7SBUDOC7AAYGDXTC5VYUBHA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1fT9qH1igPUNfrzpwa2VkBX7heh475sNqoN4Nsf9FYSwi1VmHj, address:185.167.204.205:4022): protocol: read tcp 159.223.143.39:51226->185.167.204.205:4022: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.323503666 (104h36m0.920274689s) [seg_pos={0 0}, upload=117052]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Mar 10 19:36:03.424787357 (104h36m1.021558384s) [upload=117052]: Done uploading segments...
Mar 10 19:36:03.424860036 (104h36m1.021631063s) [upload=117052]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 2.580501247s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
5 out of 60 is failed (threshold=3) [......XX.X...........X.......X..............................] duration:
2.557860629s trace:
https://jaeger.dev.storj.io/trace/78c3ad63aac6b527
FAIL (2026-03-10 19:56:54 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
:03.719970769 (104h35m1.316741801s) [noise=true , address=185.167.204.205:4033 , piece_id=6JIW66HORXSEPSDWIUONOUZNA2UNXKBA4A6VZ7PBWW3IN2SCJSPQ, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.720006314 (104h35m1.316777347s) [noise=true , address=185.167.204.205:4024 , piece_id=3CKPLBO33LOMNYBF47K2KGQFPO53ZXLMKRKWPVSR4FKQKXKMZMCA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.721245847 (104h35m1.318016883s) [noise=true , address=185.167.204.206:4046 , piece_id=5WVC4R6VY25QUMGSOAORUK3WPGIADBXLKN7ZSLJ4GQEVKMSHZPKA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.722131967 (104h35m1.318902999s) [noise=true , address=89.149.201.147:4004 , piece_id=TMKNRIR6RWRW3TUSA2MQL4LCNTJ3LKEFIT6BAVRX2V54QHNF26FA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.722154312 (104h35m1.318925349s) [noise=true , address=185.167.204.205:4028 , piece_id=IQ4CMHEIUNHIUBQV2PVFQ6BURDMWU2S55VGFP5Y7HPFSDVBY6PYA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.723422869 (104h35m1.320193903s) [noise=true , address=81.171.14.70:4047 , piece_id=DLCUNYJFEKIU3MWFXK24QGJ4GW2H5XE4Y2ARNL2W55EZUMSC4Z7Q, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.724205203 (104h35m1.320976237s) [noise=true , address=185.167.204.205:4022 , piece_id=F5UFUTAFPRBWMSU3QKV7ZSYQR6UUC7SBUDOC7AAYGDXTC5VYUBHA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.727803141 (104h35m1.324574176s) [noise=true , address=185.167.204.206:4067 , piece_id=RH2DWZ3FL7JX6EJ2WTJBPBSWDYVSHLYSLZCMMGRQXUDR6QXVZZ5A, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:04.167055423 (104h35m1.763826466s) [upload=117052]: Next returned nil segment
Mar 10 19:35:04.167072399 (104h35m1.763843424s) [upload=117052]: Waiting for error group managing segments...
Mar 10 19:35:04.350286979 (104h35m1.947058004s) [noise=true , address=81.171.14.70:4035 , piece_id=RNJYIOH4ETYQBSZFYTV3FX73J3PNFR2KBYNZEDXCVBOYMUAN5FCA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.351883208 (104h35m1.948654233s) [noise=true , address=81.171.14.70:4043 , piece_id=LSOHRZMEV5TEQUKZIJPYHVP43CIZKMXBTRVKIZIQNSSFNMAA5FLQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.356149085 (104h35m1.952920098s) [noise=true , address=185.167.204.205:4036 , piece_id=BBGCJ7U7PQDNB5NZTJWHUWXR6TFYF6D7LOR2UUBNV647YQX5JD2Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.369094269 (104h35m1.965865391s) [noise=true , address=89.149.201.147:4011 , piece_id=YHPLXK3ADD6VMYDWQ55PLROBCY2IOLOR2ZWEZ22FRRA4TXKSSWIQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.382982597 (104h35m1.979753611s) [noise=true , address=81.171.14.70:4034 , piece_id=BPXMDI54RTQVH5QKY6JCN5H3AKCPH62V76VBES4GQ6SLIZDIMMOA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.384672279 (104h35m1.981443307s) [noise=true , address=81.171.14.70:4047 , piece_id=DLCUNYJFEKIU3MWFXK24QGJ4GW2H5XE4Y2ARNL2W55EZUMSC4Z7Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.386337288 (104h35m1.98310832s) [noise=true , address=81.171.14.70:4025 , piece_id=R62A3AXM4OKUZFZJ2LB55UPB6VRGHBQLMXXVBWDTHUDWPGPIWEVQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.387583761 (104h35m1.984354793s) [noise=true , address=89.149.201.147:4009 , piece_id=XFSD3W6JWUXGXTHYE57SS5E5322JTRG7RKBI7MBF25GU7SEAKZJQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.388110220 (104h35m1.984881248s) [noise=true , address=81.171.14.70:4026 , piece_id=4CBQ4RZ44USXNYAGA3CIGNC3LBQXR675JVJX6CVTV3X7KVYPRHQA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.389451350 (104h35m1.986222362s) [noise=true , address=185.167.204.205:4012 , piece_id=7UFJIUKCHPKYZF42HTP6Q5D2FMHRSCRUQ7MH2MYJSEOZHZAJRZXA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.404093624 (104h35m2.00086464s) [noise=true , address=81.171.14.70:4048 , piece_id=4F2F4X4ZJ4FXUYQJQKB7YB3JZY53QNVRUSM3N4HQKGEYHMFYQB2A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.413111656 (104h35m2.009882687s) [noise=true , address=185.167.204.206:4071 , piece_id=ZQJLN55G37453GZS54ZDTPM7CEU5GH6LJR3BIKSJDC3C4QOKW5YQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.416183369 (104h35m2.012954404s) [noise=true , address=89.149.201.147:4014 , piece_id=WZOPRPDLFVOO6O2JYZZB75CLIAA4SVM7OEZXBX6WOJV3PUL2YSAQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.449375785 (104h35m2.046146815s) [noise=true , address=185.167.204.206:4039 , piece_id=DKKIZZMA3RAKMLUJTU6OANGBA3F6XWRD5ELVNXAGBIOZMFBVQWZQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.457716093 (104h35m2.054487108s) [noise=true , address=185.167.204.205:4024 , piece_id=3CKPLBO33LOMNYBF47K2KGQFPO53ZXLMKRKWPVSR4FKQKXKMZMCA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.491116161 (104h35m2.087887192s) [noise=true , address=185.167.204.205:4005 , piece_id=E7RO77MCAMFEAIE7ETPALCBV5S4X3DG5ZUD27ZE55RXYICIIXNEQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.507099744 (104h35m2.103870782s) [noise=true , address=185.167.204.205:4029 , piece_id=R33ONZELVLYL76YTJKOZASDJKFBACNNMC3PDUIITQRDGBWWZUNKA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.537139239 (104h35m2.133910269s) [noise=true , address=185.167.204.206:4067 , piece_id=RH2DWZ3FL7JX6EJ2WTJBPBSWDYVSHLYSLZCMMGRQXUDR6QXVZZ5A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.787100189 (104h35m2.383871224s) [noise=true , address=81.171.14.70:4033 , piece_id=ZFV477QXRX42OJIG4D5JLOYU3WERTD65PMHI5MFQLRTQJG3ENUTA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.962846444 (104h35m2.559617469s) [noise=true , address=89.149.201.147:4004 , piece_id=TMKNRIR6RWRW3TUSA2MQL4LCNTJ3LKEFIT6BAVRX2V54QHNF26FA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:08.299132133 (104h35m5.895903147s) [noise=true , address=185.167.204.205:4028 , piece_id=IQ4CMHEIUNHIUBQV2PVFQ6BURDMWU2S55VGFP5Y7HPFSDVBY6PYA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:08.481127955 (104h35m6.077898968s) [noise=true , address=185.167.204.206:4051 , piece_id=GO4OTIL5WFOS7V77FAT2NRFG2HRE2HXKMU2C3XEGDBGPQL5U5HUA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:10.314952727 (104h35m7.91172374s) [noise=true , address=185.167.204.206:4046 , piece_id=5WVC4R6VY25QUMGSOAORUK3WPGIADBXLKN7ZSLJ4GQEVKMSHZPKA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:36:03.322697363 (104h36m0.919468398s) [noise=true , address=185.167.204.206:4062 , piece_id=7NP2ZIMW6ZDXW3YHZSBO5PRTAZPLYLDWXDA4QPN5XH2C6KAGYLEA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:12BL6XLqsoUGm2FxrTgYTxLZakt2u2CAhZJG82oJ31eaE4vTtMr, address:185.167.204.206:4062): protocol: write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection; write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection
Mar 10 19:36:03.322771647 (104h36m0.919542676s) [noise=true , address=89.149.201.147:4007 , piece_id=AAJY5QBN55ENHTODH25WBVCFDT2SAQ2F27TKHFCIURCZSBMGOY6A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1Q7d4vj1KoWhDr6xd9gEfFrZ1sV3Yro2AFke2f3Ke1rXsGnvew, address:89.149.201.147:4007): protocol: read tcp 159.223.143.39:54460->89.149.201.147:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.322958851 (104h36m0.919729884s) [noise=true , address=185.167.204.205:4033 , piece_id=6JIW66HORXSEPSDWIUONOUZNA2UNXKBA4A6VZ7PBWW3IN2SCJSPQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:12Q73mXHLEfuX4VUbJ8PTjhMoXCrYSKjJbFxqjDdqF3U99N8FXe, address:185.167.204.205:4033): protocol: write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection; write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection
Mar 10 19:36:03.323134221 (104h36m0.919905244s) [noise=true , address=89.149.201.147:4018 , piece_id=V3KLJ75HJ3NRMBP2FIM33GC45QW4U5XMN2R4IAHSX32COGBJF47Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1Ku7o12bSmJ1YR3dGetGFcWqs1wJEh53Zkg3H7ELJrt4w1hkua, address:89.149.201.147:4018): protocol: read tcp 159.223.143.39:47122->89.149.201.147:4018: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.323461923 (104h36m0.920232956s) [noise=true , address=185.167.204.205:4022 , piece_id=F5UFUTAFPRBWMSU3QKV7ZSYQR6UUC7SBUDOC7AAYGDXTC5VYUBHA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1fT9qH1igPUNfrzpwa2VkBX7heh475sNqoN4Nsf9FYSwi1VmHj, address:185.167.204.205:4022): protocol: read tcp 159.223.143.39:51226->185.167.204.205:4022: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.323503666 (104h36m0.920274689s) [seg_pos={0 0}, upload=117052]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Mar 10 19:36:03.424787357 (104h36m1.021558384s) [upload=117052]: Done uploading segments...
Mar 10 19:36:03.424860036 (104h36m1.021631063s) [upload=117052]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 2.557889192s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
6 out of 60 is failed (threshold=3) [X......XX.X...........X.......X.............................] duration:
2.14686418s trace:
https://jaeger.dev.storj.io/trace/78c3ad63aac6b527
FAIL (2026-03-10 19:56:04 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
:03.719970769 (104h35m1.316741801s) [noise=true , address=185.167.204.205:4033 , piece_id=6JIW66HORXSEPSDWIUONOUZNA2UNXKBA4A6VZ7PBWW3IN2SCJSPQ, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.720006314 (104h35m1.316777347s) [noise=true , address=185.167.204.205:4024 , piece_id=3CKPLBO33LOMNYBF47K2KGQFPO53ZXLMKRKWPVSR4FKQKXKMZMCA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.721245847 (104h35m1.318016883s) [noise=true , address=185.167.204.206:4046 , piece_id=5WVC4R6VY25QUMGSOAORUK3WPGIADBXLKN7ZSLJ4GQEVKMSHZPKA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.722131967 (104h35m1.318902999s) [noise=true , address=89.149.201.147:4004 , piece_id=TMKNRIR6RWRW3TUSA2MQL4LCNTJ3LKEFIT6BAVRX2V54QHNF26FA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.722154312 (104h35m1.318925349s) [noise=true , address=185.167.204.205:4028 , piece_id=IQ4CMHEIUNHIUBQV2PVFQ6BURDMWU2S55VGFP5Y7HPFSDVBY6PYA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.723422869 (104h35m1.320193903s) [noise=true , address=81.171.14.70:4047 , piece_id=DLCUNYJFEKIU3MWFXK24QGJ4GW2H5XE4Y2ARNL2W55EZUMSC4Z7Q, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.724205203 (104h35m1.320976237s) [noise=true , address=185.167.204.205:4022 , piece_id=F5UFUTAFPRBWMSU3QKV7ZSYQR6UUC7SBUDOC7AAYGDXTC5VYUBHA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.727803141 (104h35m1.324574176s) [noise=true , address=185.167.204.206:4067 , piece_id=RH2DWZ3FL7JX6EJ2WTJBPBSWDYVSHLYSLZCMMGRQXUDR6QXVZZ5A, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:04.167055423 (104h35m1.763826466s) [upload=117052]: Next returned nil segment
Mar 10 19:35:04.167072399 (104h35m1.763843424s) [upload=117052]: Waiting for error group managing segments...
Mar 10 19:35:04.350286979 (104h35m1.947058004s) [noise=true , address=81.171.14.70:4035 , piece_id=RNJYIOH4ETYQBSZFYTV3FX73J3PNFR2KBYNZEDXCVBOYMUAN5FCA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.351883208 (104h35m1.948654233s) [noise=true , address=81.171.14.70:4043 , piece_id=LSOHRZMEV5TEQUKZIJPYHVP43CIZKMXBTRVKIZIQNSSFNMAA5FLQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.356149085 (104h35m1.952920098s) [noise=true , address=185.167.204.205:4036 , piece_id=BBGCJ7U7PQDNB5NZTJWHUWXR6TFYF6D7LOR2UUBNV647YQX5JD2Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.369094269 (104h35m1.965865391s) [noise=true , address=89.149.201.147:4011 , piece_id=YHPLXK3ADD6VMYDWQ55PLROBCY2IOLOR2ZWEZ22FRRA4TXKSSWIQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.382982597 (104h35m1.979753611s) [noise=true , address=81.171.14.70:4034 , piece_id=BPXMDI54RTQVH5QKY6JCN5H3AKCPH62V76VBES4GQ6SLIZDIMMOA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.384672279 (104h35m1.981443307s) [noise=true , address=81.171.14.70:4047 , piece_id=DLCUNYJFEKIU3MWFXK24QGJ4GW2H5XE4Y2ARNL2W55EZUMSC4Z7Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.386337288 (104h35m1.98310832s) [noise=true , address=81.171.14.70:4025 , piece_id=R62A3AXM4OKUZFZJ2LB55UPB6VRGHBQLMXXVBWDTHUDWPGPIWEVQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.387583761 (104h35m1.984354793s) [noise=true , address=89.149.201.147:4009 , piece_id=XFSD3W6JWUXGXTHYE57SS5E5322JTRG7RKBI7MBF25GU7SEAKZJQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.388110220 (104h35m1.984881248s) [noise=true , address=81.171.14.70:4026 , piece_id=4CBQ4RZ44USXNYAGA3CIGNC3LBQXR675JVJX6CVTV3X7KVYPRHQA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.389451350 (104h35m1.986222362s) [noise=true , address=185.167.204.205:4012 , piece_id=7UFJIUKCHPKYZF42HTP6Q5D2FMHRSCRUQ7MH2MYJSEOZHZAJRZXA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.404093624 (104h35m2.00086464s) [noise=true , address=81.171.14.70:4048 , piece_id=4F2F4X4ZJ4FXUYQJQKB7YB3JZY53QNVRUSM3N4HQKGEYHMFYQB2A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.413111656 (104h35m2.009882687s) [noise=true , address=185.167.204.206:4071 , piece_id=ZQJLN55G37453GZS54ZDTPM7CEU5GH6LJR3BIKSJDC3C4QOKW5YQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.416183369 (104h35m2.012954404s) [noise=true , address=89.149.201.147:4014 , piece_id=WZOPRPDLFVOO6O2JYZZB75CLIAA4SVM7OEZXBX6WOJV3PUL2YSAQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.449375785 (104h35m2.046146815s) [noise=true , address=185.167.204.206:4039 , piece_id=DKKIZZMA3RAKMLUJTU6OANGBA3F6XWRD5ELVNXAGBIOZMFBVQWZQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.457716093 (104h35m2.054487108s) [noise=true , address=185.167.204.205:4024 , piece_id=3CKPLBO33LOMNYBF47K2KGQFPO53ZXLMKRKWPVSR4FKQKXKMZMCA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.491116161 (104h35m2.087887192s) [noise=true , address=185.167.204.205:4005 , piece_id=E7RO77MCAMFEAIE7ETPALCBV5S4X3DG5ZUD27ZE55RXYICIIXNEQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.507099744 (104h35m2.103870782s) [noise=true , address=185.167.204.205:4029 , piece_id=R33ONZELVLYL76YTJKOZASDJKFBACNNMC3PDUIITQRDGBWWZUNKA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.537139239 (104h35m2.133910269s) [noise=true , address=185.167.204.206:4067 , piece_id=RH2DWZ3FL7JX6EJ2WTJBPBSWDYVSHLYSLZCMMGRQXUDR6QXVZZ5A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.787100189 (104h35m2.383871224s) [noise=true , address=81.171.14.70:4033 , piece_id=ZFV477QXRX42OJIG4D5JLOYU3WERTD65PMHI5MFQLRTQJG3ENUTA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.962846444 (104h35m2.559617469s) [noise=true , address=89.149.201.147:4004 , piece_id=TMKNRIR6RWRW3TUSA2MQL4LCNTJ3LKEFIT6BAVRX2V54QHNF26FA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:08.299132133 (104h35m5.895903147s) [noise=true , address=185.167.204.205:4028 , piece_id=IQ4CMHEIUNHIUBQV2PVFQ6BURDMWU2S55VGFP5Y7HPFSDVBY6PYA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:08.481127955 (104h35m6.077898968s) [noise=true , address=185.167.204.206:4051 , piece_id=GO4OTIL5WFOS7V77FAT2NRFG2HRE2HXKMU2C3XEGDBGPQL5U5HUA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:10.314952727 (104h35m7.91172374s) [noise=true , address=185.167.204.206:4046 , piece_id=5WVC4R6VY25QUMGSOAORUK3WPGIADBXLKN7ZSLJ4GQEVKMSHZPKA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:36:03.322697363 (104h36m0.919468398s) [noise=true , address=185.167.204.206:4062 , piece_id=7NP2ZIMW6ZDXW3YHZSBO5PRTAZPLYLDWXDA4QPN5XH2C6KAGYLEA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:12BL6XLqsoUGm2FxrTgYTxLZakt2u2CAhZJG82oJ31eaE4vTtMr, address:185.167.204.206:4062): protocol: write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection; write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection
Mar 10 19:36:03.322771647 (104h36m0.919542676s) [noise=true , address=89.149.201.147:4007 , piece_id=AAJY5QBN55ENHTODH25WBVCFDT2SAQ2F27TKHFCIURCZSBMGOY6A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1Q7d4vj1KoWhDr6xd9gEfFrZ1sV3Yro2AFke2f3Ke1rXsGnvew, address:89.149.201.147:4007): protocol: read tcp 159.223.143.39:54460->89.149.201.147:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.322958851 (104h36m0.919729884s) [noise=true , address=185.167.204.205:4033 , piece_id=6JIW66HORXSEPSDWIUONOUZNA2UNXKBA4A6VZ7PBWW3IN2SCJSPQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:12Q73mXHLEfuX4VUbJ8PTjhMoXCrYSKjJbFxqjDdqF3U99N8FXe, address:185.167.204.205:4033): protocol: write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection; write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection
Mar 10 19:36:03.323134221 (104h36m0.919905244s) [noise=true , address=89.149.201.147:4018 , piece_id=V3KLJ75HJ3NRMBP2FIM33GC45QW4U5XMN2R4IAHSX32COGBJF47Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1Ku7o12bSmJ1YR3dGetGFcWqs1wJEh53Zkg3H7ELJrt4w1hkua, address:89.149.201.147:4018): protocol: read tcp 159.223.143.39:47122->89.149.201.147:4018: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.323461923 (104h36m0.920232956s) [noise=true , address=185.167.204.205:4022 , piece_id=F5UFUTAFPRBWMSU3QKV7ZSYQR6UUC7SBUDOC7AAYGDXTC5VYUBHA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1fT9qH1igPUNfrzpwa2VkBX7heh475sNqoN4Nsf9FYSwi1VmHj, address:185.167.204.205:4022): protocol: read tcp 159.223.143.39:51226->185.167.204.205:4022: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.323503666 (104h36m0.920274689s) [seg_pos={0 0}, upload=117052]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Mar 10 19:36:03.424787357 (104h36m1.021558384s) [upload=117052]: Done uploading segments...
Mar 10 19:36:03.424860036 (104h36m1.021631063s) [upload=117052]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 2.146895164s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
1.06349957s trace:
https://jaeger.dev.storj.io/trace/69062766ac2178dd
FAIL (2026-03-10 19:55:47 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 12)
======LOGS======
Mar 10 19:10:26.813115721 (104h10m24.409886745s) [upload=116599]: Starting upload
Mar 10 19:10:26.813211715 (104h10m24.409982738s) [upload=116599]: Done starting upload
Mar 10 19:10:26.813505664 (104h10m24.410276708s) [upload=116599]: Uploading segments...
Mar 10 19:10:26.813650528 (104h10m24.410421539s) [upload=116599]: Got next segment. Inline: false
Mar 10 19:10:26.813743195 (104h10m24.410514218s) [upload=116599]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Mar 10 19:10:26.813750388 (104h10m24.410521411s) [upload=116599]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Mar 10 19:10:26.846063672 (104h10m24.442834685s) [seg_pos={0 0}, upload=116599]: Begin upload segment...
Mar 10 19:10:26.846148290 (104h10m24.442919315s) [seg_pos={0 0}, upload=116599]: Done begin upload segment.
Mar 10 19:10:26.846228010 (104h10m24.442999069s) [noise=true , address=77.90.31.16:4754 , piece_id=PE4KKNG6B7T5FTCJOYIKG2ML2XOAD6SYAKAIRTKSKQM66GVTFASQ, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.846301186 (104h10m24.44307221s) [noise=true , address=79.127.205.233:4154 , piece_id=LDMVPR6RMS2F3YNIMVH2ELREQMP6WMEXRW5UDBHTZIPZEPWRD4BQ, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.846940319 (104h10m24.443711359s) [noise=true , address=193.149.164.70:4116 , piece_id=FVAICHAZFMXEXK5CQ3SXYQ46T54PQBGYVPPBSKZ7WOHSGOKPHWRQ, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.849741507 (104h10m24.446512533s) [noise=true , address=167.17.72.68:4654 , piece_id=WHKUMMAPP7ZSKFMAGKFHINHWIICOCMREK75F3EON723TKLAWDDAQ, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.850583955 (104h10m24.447354987s) [noise=true , address=167.17.72.70:4758 , piece_id=3OZLHHR5C6OX4EJK23YRLEGJOJ3IXWPR35CZ6MQARXGX5NNGXUGQ, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.851500001 (104h10m24.448271012s) [noise=true , address=77.90.31.17:4793 , piece_id=GB6MI5XNLXQL4KFADPIKGKDKIQ7LNZ65EOTKY7LLSQMYY7WS56FQ, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.851893521 (104h10m24.448664555s) [noise=true , address=5.231.73.4:4012 , piece_id=5RZOPQ6PFL42RJPI4ZR4GS756AU332JPC34SVWK4M2FELGI3O7CA, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.853333272 (104h10m24.450104308s) [noise=true , address=77.90.31.14:4690 , piece_id=XUIROGKLTQGEBWNIYKM5LA6U4DZG3ZYFMCHMNYCZJ2R4Y5WB7Z5A, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.853969417 (104h10m24.450740435s) [noise=true , address=77.90.31.12:4604 , piece_id=JUIZARJIP27O4PO3IZ54Y734TUIG2I3DUCQNFWI5TH5ZMCBDGJMA, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.854283058 (104h10m24.451054099s) [noise=true , address=145.79.227.2:4133 , piece_id=AXG6E4SMSMKGVOINK4HMHTXL6GRKQWU6NJOP633KYOKQ3HQHMFOA, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.854903526 (104h10m24.45167454s) [noise=true , address=77.90.31.11:4585 , piece_id=VX7QB2HCS64PW5BIRIJ3RUY7V5FMDVTO423ENNPOTSQUINRR6R5A, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.855071429 (104h10m24.451842462s) [noise=true , address=64.49.13.139:4474 , piece_id=HFVZT3V5KD42X64HCTKPEG2LWZJZAVOH5MA5CFCIMPB7WK7MWQNQ, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.855704995 (104h10m24.452476028s) [noise=true , address=193.149.164.69:4092 , piece_id=T63NKUCMVMLADZYIMCW3FMJLYURBF7YVSMQ7AFLCYOU2K3C5RYNQ, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.855767508 (104h10m24.45253852s) [noise=true , address=77.90.31.18:4814 , piece_id=OUCWOBYCYXTPTVHHOOILTZTLUB6PJ42HOV3YURYC3LTKMTDDGPSA, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.856577407 (104h10m24.453348417s) [noise=true , address=193.149.164.71:4306 , piece_id=SJ5TPY5AZVF55AK3MRAT6K5ZPA4V6N2JXBYILVM3NHTCTRZI6MLQ, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.857565751 (104h10m24.454336779s) [noise=true , address=64.49.13.140:4507 , piece_id=6PNJPCU3KVM4ZFBTXQVZG26IEG57G3HHDRLBXEAB3VTU6IEPCAWQ, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.858400866 (104h10m24.455171879s) [noise=true , address=64.49.13.170:4725 , piece_id=DEFHEGRC3SUXYVLYV2YR3KPBS4GLCPQBMV2GWR7KRY4ZBWDGAJJA, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.859258131 (104h10m24.456029145s) [noise=true , address=79.127.205.237:4245 , piece_id=ZO6RJQU6Z5XIV6NXYBV7PWACGAEP7PWJMA7VISBKQS7IRWOXNRZQ, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.934798292 (104h10m24.531569311s) [upload=116599]: Next returned nil segment
Mar 10 19:10:26.934817790 (104h10m24.531588824s) [upload=116599]: Waiting for error group managing segments...
Mar 10 19:10:26.999719507 (104h10m24.59649054s) [noise=true , address=145.79.227.2:4133 , piece_id=AXG6E4SMSMKGVOINK4HMHTXL6GRKQWU6NJOP633KYOKQ3HQHMFOA, seg_pos={0 0}, upload=116599]: Done uploading piece. err: <nil>
Mar 10 19:10:27.030173609 (104h10m24.626944622s) [noise=true , address=5.231.73.4:4012 , piece_id=5RZOPQ6PFL42RJPI4ZR4GS756AU332JPC34SVWK4M2FELGI3O7CA, seg_pos={0 0}, upload=116599]: Done uploading piece. err: <nil>
Mar 10 19:10:27.238694294 (104h10m24.83546533s) [noise=true , address=167.17.72.70:4758 , piece_id=3OZLHHR5C6OX4EJK23YRLEGJOJ3IXWPR35CZ6MQARXGX5NNGXUGQ, seg_pos={0 0}, upload=116599]: Done uploading piece. err: <nil>
Mar 10 19:10:27.294399709 (104h10m24.891170737s) [noise=true , address=167.17.72.68:4654 , piece_id=WHKUMMAPP7ZSKFMAGKFHINHWIICOCMREK75F3EON723TKLAWDDAQ, seg_pos={0 0}, upload=116599]: Done uploading piece. err: <nil>
Mar 10 19:10:27.393989371 (104h10m24.990760384s) [noise=true , address=79.127.205.233:4154 , piece_id=LDMVPR6RMS2F3YNIMVH2ELREQMP6WMEXRW5UDBHTZIPZEPWRD4BQ, seg_pos={0 0}, upload=116599]: Done uploading piece. err: <nil>
Mar 10 19:10:27.451000332 (104h10m25.047771355s) [noise=true , address=193.149.164.69:4092 , piece_id=T63NKUCMVMLADZYIMCW3FMJLYURBF7YVSMQ7AFLCYOU2K3C5RYNQ, seg_pos={0 0}, upload=116599]: Done uploading piece. err: <nil>
Mar 10 19:10:27.453000338 (104h10m25.04977136s) [noise=true , address=193.149.164.70:4116 , piece_id=FVAICHAZFMXEXK5CQ3SXYQ46T54PQBGYVPPBSKZ7WOHSGOKPHWRQ, seg_pos={0 0}, upload=116599]: Done uploading piece. err: <nil>
Mar 10 19:10:27.472395366 (104h10m25.069166393s) [noise=true , address=193.149.164.71:4306 , piece_id=SJ5TPY5AZVF55AK3MRAT6K5ZPA4V6N2JXBYILVM3NHTCTRZI6MLQ, seg_pos={0 0}, upload=116599]: Done uploading piece. err: <nil>
Mar 10 19:10:27.486423055 (104h10m25.083194088s) [noise=true , address=64.49.13.140:4507 , piece_id=6PNJPCU3KVM4ZFBTXQVZG26IEG57G3HHDRLBXEAB3VTU6IEPCAWQ, seg_pos={0 0}, upload=116599]: Done uploading piece. err: <nil>
Mar 10 19:10:27.487859041 (104h10m25.084630054s) [noise=true , address=64.49.13.139:4474 , piece_id=HFVZT3V5KD42X64HCTKPEG2LWZJZAVOH5MA5CFCIMPB7WK7MWQNQ, seg_pos={0 0}, upload=116599]: Done uploading piece. err: <nil>
Mar 10 19:10:27.489472801 (104h10m25.086243832s) [noise=true , address=64.49.13.170:4725 , piece_id=DEFHEGRC3SUXYVLYV2YR3KPBS4GLCPQBMV2GWR7KRY4ZBWDGAJJA, seg_pos={0 0}, upload=116599]: Done uploading piece. err: <nil>
Mar 10 19:10:28.942749218 (104h10m26.539520243s) [noise=true , address=79.127.205.237:4245 , piece_id=ZO6RJQU6Z5XIV6NXYBV7PWACGAEP7PWJMA7VISBKQS7IRWOXNRZQ, seg_pos={0 0}, upload=116599]: Done uploading piece. err: <nil>
Mar 10 19:11:26.775681128 (104h11m24.37245216s) [noise=true , address=77.90.31.17:4793 , piece_id=GB6MI5XNLXQL4KFADPIKGKDKIQ7LNZ65EOTKY7LLSQMYY7WS56FQ, seg_pos={0 0}, upload=116599]: Done uploading piece. err: ecclient: upload failed (node:1rxrH2F4LTfrK9SMsNNEuqVr4AjV1VbEYDc3veWbQf88rj2Es4, address:77.90.31.17:4793): protocol: rpc: dial tcp 77.90.31.17:4793: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:11:26.775748621 (104h11m24.37251965s) [noise=true , address=77.90.31.12:4604 , piece_id=JUIZARJIP27O4PO3IZ54Y734TUIG2I3DUCQNFWI5TH5ZMCBDGJMA, seg_pos={0 0}, upload=116599]: Done uploading piece. err: ecclient: upload failed (node:1vK9QCCj1rszVCR72YBc8BdKCPRno49RrcH9uGQb9rBJkZY4F7, address:77.90.31.12:4604): protocol: rpc: dial tcp 77.90.31.12:4604: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:11:26.775972253 (104h11m24.372743279s) [noise=true , address=77.90.31.16:4754 , piece_id=PE4KKNG6B7T5FTCJOYIKG2ML2XOAD6SYAKAIRTKSKQM66GVTFASQ, seg_pos={0 0}, upload=116599]: Done uploading piece. err: ecclient: upload failed (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8, address:77.90.31.16:4754): protocol: write tcp 159.223.143.39:58400->77.90.31.16:4754: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:11:26.776001817 (104h11m24.37277285s) [noise=true , address=77.90.31.14:4690 , piece_id=XUIROGKLTQGEBWNIYKM5LA6U4DZG3ZYFMCHMNYCZJ2R4Y5WB7Z5A, seg_pos={0 0}, upload=116599]: Done uploading piece. err: ecclient: upload failed (node:1WPKeCcN1uEhGjZHAJgCoJMaHxAWouGCdaU1vZwAS7kX22rxcv, address:77.90.31.14:4690): protocol: rpc: dial tcp 77.90.31.14:4690: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:11:26.776244713 (104h11m24.373015806s) [noise=true , address=77.90.31.18:4814 , piece_id=OUCWOBYCYXTPTVHHOOILTZTLUB6PJ42HOV3YURYC3LTKMTDDGPSA, seg_pos={0 0}, upload=116599]: Done uploading piece. err: ecclient: upload failed (node:1FCikLXgPYS2qM9zLTthSNHKsQNckzg9oNkKGxfvwJZZjhKkVf, address:77.90.31.18:4814): protocol: write tcp 159.223.143.39:36246->77.90.31.18:4814: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:11:26.776280556 (104h11m24.373051587s) [noise=true , address=77.90.31.11:4585 , piece_id=VX7QB2HCS64PW5BIRIJ3RUY7V5FMDVTO423ENNPOTSQUINRR6R5A, seg_pos={0 0}, upload=116599]: Done uploading piece. err: ecclient: upload failed (node:1Tve8WUWhJCK2cVWLbxxnz3WVzR2bi2wLx5XrF6QbhSN4HcsQG, address:77.90.31.11:4585): protocol: context deadline exceeded; context deadline exceeded
Mar 10 19:11:26.776313694 (104h11m24.373084705s) [seg_pos={0 0}, upload=116599]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil>
Mar 10 19:11:26.804828285 (104h11m24.401599298s) [upload=116599]: Done uploading segments...
Mar 10 19:11:26.805017989 (104h11m24.401789013s) [upload=116599]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12)
Duration: 1.063529101s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
6 out of 60 is failed (threshold=3) [.X......XX.X...........X.......X............................] duration:
4.14048482s trace:
https://jaeger.dev.storj.io/trace/78c3ad63aac6b527
FAIL (2026-03-10 19:54:53 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
:03.719970769 (104h35m1.316741801s) [noise=true , address=185.167.204.205:4033 , piece_id=6JIW66HORXSEPSDWIUONOUZNA2UNXKBA4A6VZ7PBWW3IN2SCJSPQ, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.720006314 (104h35m1.316777347s) [noise=true , address=185.167.204.205:4024 , piece_id=3CKPLBO33LOMNYBF47K2KGQFPO53ZXLMKRKWPVSR4FKQKXKMZMCA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.721245847 (104h35m1.318016883s) [noise=true , address=185.167.204.206:4046 , piece_id=5WVC4R6VY25QUMGSOAORUK3WPGIADBXLKN7ZSLJ4GQEVKMSHZPKA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.722131967 (104h35m1.318902999s) [noise=true , address=89.149.201.147:4004 , piece_id=TMKNRIR6RWRW3TUSA2MQL4LCNTJ3LKEFIT6BAVRX2V54QHNF26FA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.722154312 (104h35m1.318925349s) [noise=true , address=185.167.204.205:4028 , piece_id=IQ4CMHEIUNHIUBQV2PVFQ6BURDMWU2S55VGFP5Y7HPFSDVBY6PYA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.723422869 (104h35m1.320193903s) [noise=true , address=81.171.14.70:4047 , piece_id=DLCUNYJFEKIU3MWFXK24QGJ4GW2H5XE4Y2ARNL2W55EZUMSC4Z7Q, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.724205203 (104h35m1.320976237s) [noise=true , address=185.167.204.205:4022 , piece_id=F5UFUTAFPRBWMSU3QKV7ZSYQR6UUC7SBUDOC7AAYGDXTC5VYUBHA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.727803141 (104h35m1.324574176s) [noise=true , address=185.167.204.206:4067 , piece_id=RH2DWZ3FL7JX6EJ2WTJBPBSWDYVSHLYSLZCMMGRQXUDR6QXVZZ5A, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:04.167055423 (104h35m1.763826466s) [upload=117052]: Next returned nil segment
Mar 10 19:35:04.167072399 (104h35m1.763843424s) [upload=117052]: Waiting for error group managing segments...
Mar 10 19:35:04.350286979 (104h35m1.947058004s) [noise=true , address=81.171.14.70:4035 , piece_id=RNJYIOH4ETYQBSZFYTV3FX73J3PNFR2KBYNZEDXCVBOYMUAN5FCA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.351883208 (104h35m1.948654233s) [noise=true , address=81.171.14.70:4043 , piece_id=LSOHRZMEV5TEQUKZIJPYHVP43CIZKMXBTRVKIZIQNSSFNMAA5FLQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.356149085 (104h35m1.952920098s) [noise=true , address=185.167.204.205:4036 , piece_id=BBGCJ7U7PQDNB5NZTJWHUWXR6TFYF6D7LOR2UUBNV647YQX5JD2Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.369094269 (104h35m1.965865391s) [noise=true , address=89.149.201.147:4011 , piece_id=YHPLXK3ADD6VMYDWQ55PLROBCY2IOLOR2ZWEZ22FRRA4TXKSSWIQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.382982597 (104h35m1.979753611s) [noise=true , address=81.171.14.70:4034 , piece_id=BPXMDI54RTQVH5QKY6JCN5H3AKCPH62V76VBES4GQ6SLIZDIMMOA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.384672279 (104h35m1.981443307s) [noise=true , address=81.171.14.70:4047 , piece_id=DLCUNYJFEKIU3MWFXK24QGJ4GW2H5XE4Y2ARNL2W55EZUMSC4Z7Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.386337288 (104h35m1.98310832s) [noise=true , address=81.171.14.70:4025 , piece_id=R62A3AXM4OKUZFZJ2LB55UPB6VRGHBQLMXXVBWDTHUDWPGPIWEVQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.387583761 (104h35m1.984354793s) [noise=true , address=89.149.201.147:4009 , piece_id=XFSD3W6JWUXGXTHYE57SS5E5322JTRG7RKBI7MBF25GU7SEAKZJQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.388110220 (104h35m1.984881248s) [noise=true , address=81.171.14.70:4026 , piece_id=4CBQ4RZ44USXNYAGA3CIGNC3LBQXR675JVJX6CVTV3X7KVYPRHQA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.389451350 (104h35m1.986222362s) [noise=true , address=185.167.204.205:4012 , piece_id=7UFJIUKCHPKYZF42HTP6Q5D2FMHRSCRUQ7MH2MYJSEOZHZAJRZXA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.404093624 (104h35m2.00086464s) [noise=true , address=81.171.14.70:4048 , piece_id=4F2F4X4ZJ4FXUYQJQKB7YB3JZY53QNVRUSM3N4HQKGEYHMFYQB2A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.413111656 (104h35m2.009882687s) [noise=true , address=185.167.204.206:4071 , piece_id=ZQJLN55G37453GZS54ZDTPM7CEU5GH6LJR3BIKSJDC3C4QOKW5YQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.416183369 (104h35m2.012954404s) [noise=true , address=89.149.201.147:4014 , piece_id=WZOPRPDLFVOO6O2JYZZB75CLIAA4SVM7OEZXBX6WOJV3PUL2YSAQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.449375785 (104h35m2.046146815s) [noise=true , address=185.167.204.206:4039 , piece_id=DKKIZZMA3RAKMLUJTU6OANGBA3F6XWRD5ELVNXAGBIOZMFBVQWZQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.457716093 (104h35m2.054487108s) [noise=true , address=185.167.204.205:4024 , piece_id=3CKPLBO33LOMNYBF47K2KGQFPO53ZXLMKRKWPVSR4FKQKXKMZMCA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.491116161 (104h35m2.087887192s) [noise=true , address=185.167.204.205:4005 , piece_id=E7RO77MCAMFEAIE7ETPALCBV5S4X3DG5ZUD27ZE55RXYICIIXNEQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.507099744 (104h35m2.103870782s) [noise=true , address=185.167.204.205:4029 , piece_id=R33ONZELVLYL76YTJKOZASDJKFBACNNMC3PDUIITQRDGBWWZUNKA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.537139239 (104h35m2.133910269s) [noise=true , address=185.167.204.206:4067 , piece_id=RH2DWZ3FL7JX6EJ2WTJBPBSWDYVSHLYSLZCMMGRQXUDR6QXVZZ5A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.787100189 (104h35m2.383871224s) [noise=true , address=81.171.14.70:4033 , piece_id=ZFV477QXRX42OJIG4D5JLOYU3WERTD65PMHI5MFQLRTQJG3ENUTA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.962846444 (104h35m2.559617469s) [noise=true , address=89.149.201.147:4004 , piece_id=TMKNRIR6RWRW3TUSA2MQL4LCNTJ3LKEFIT6BAVRX2V54QHNF26FA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:08.299132133 (104h35m5.895903147s) [noise=true , address=185.167.204.205:4028 , piece_id=IQ4CMHEIUNHIUBQV2PVFQ6BURDMWU2S55VGFP5Y7HPFSDVBY6PYA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:08.481127955 (104h35m6.077898968s) [noise=true , address=185.167.204.206:4051 , piece_id=GO4OTIL5WFOS7V77FAT2NRFG2HRE2HXKMU2C3XEGDBGPQL5U5HUA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:10.314952727 (104h35m7.91172374s) [noise=true , address=185.167.204.206:4046 , piece_id=5WVC4R6VY25QUMGSOAORUK3WPGIADBXLKN7ZSLJ4GQEVKMSHZPKA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:36:03.322697363 (104h36m0.919468398s) [noise=true , address=185.167.204.206:4062 , piece_id=7NP2ZIMW6ZDXW3YHZSBO5PRTAZPLYLDWXDA4QPN5XH2C6KAGYLEA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:12BL6XLqsoUGm2FxrTgYTxLZakt2u2CAhZJG82oJ31eaE4vTtMr, address:185.167.204.206:4062): protocol: write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection; write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection
Mar 10 19:36:03.322771647 (104h36m0.919542676s) [noise=true , address=89.149.201.147:4007 , piece_id=AAJY5QBN55ENHTODH25WBVCFDT2SAQ2F27TKHFCIURCZSBMGOY6A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1Q7d4vj1KoWhDr6xd9gEfFrZ1sV3Yro2AFke2f3Ke1rXsGnvew, address:89.149.201.147:4007): protocol: read tcp 159.223.143.39:54460->89.149.201.147:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.322958851 (104h36m0.919729884s) [noise=true , address=185.167.204.205:4033 , piece_id=6JIW66HORXSEPSDWIUONOUZNA2UNXKBA4A6VZ7PBWW3IN2SCJSPQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:12Q73mXHLEfuX4VUbJ8PTjhMoXCrYSKjJbFxqjDdqF3U99N8FXe, address:185.167.204.205:4033): protocol: write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection; write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection
Mar 10 19:36:03.323134221 (104h36m0.919905244s) [noise=true , address=89.149.201.147:4018 , piece_id=V3KLJ75HJ3NRMBP2FIM33GC45QW4U5XMN2R4IAHSX32COGBJF47Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1Ku7o12bSmJ1YR3dGetGFcWqs1wJEh53Zkg3H7ELJrt4w1hkua, address:89.149.201.147:4018): protocol: read tcp 159.223.143.39:47122->89.149.201.147:4018: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.323461923 (104h36m0.920232956s) [noise=true , address=185.167.204.205:4022 , piece_id=F5UFUTAFPRBWMSU3QKV7ZSYQR6UUC7SBUDOC7AAYGDXTC5VYUBHA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1fT9qH1igPUNfrzpwa2VkBX7heh475sNqoN4Nsf9FYSwi1VmHj, address:185.167.204.205:4022): protocol: read tcp 159.223.143.39:51226->185.167.204.205:4022: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.323503666 (104h36m0.920274689s) [seg_pos={0 0}, upload=117052]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Mar 10 19:36:03.424787357 (104h36m1.021558384s) [upload=117052]: Done uploading segments...
Mar 10 19:36:03.424860036 (104h36m1.021631063s) [upload=117052]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 4.140512626s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
1.060067982s trace:
https://jaeger.dev.storj.io/trace/69062766ac2178dd
FAIL (2026-03-10 19:54:35 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 12)
======LOGS======
Mar 10 19:10:26.813115721 (104h10m24.409886745s) [upload=116599]: Starting upload
Mar 10 19:10:26.813211715 (104h10m24.409982738s) [upload=116599]: Done starting upload
Mar 10 19:10:26.813505664 (104h10m24.410276708s) [upload=116599]: Uploading segments...
Mar 10 19:10:26.813650528 (104h10m24.410421539s) [upload=116599]: Got next segment. Inline: false
Mar 10 19:10:26.813743195 (104h10m24.410514218s) [upload=116599]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Mar 10 19:10:26.813750388 (104h10m24.410521411s) [upload=116599]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Mar 10 19:10:26.846063672 (104h10m24.442834685s) [seg_pos={0 0}, upload=116599]: Begin upload segment...
Mar 10 19:10:26.846148290 (104h10m24.442919315s) [seg_pos={0 0}, upload=116599]: Done begin upload segment.
Mar 10 19:10:26.846228010 (104h10m24.442999069s) [noise=true , address=77.90.31.16:4754 , piece_id=PE4KKNG6B7T5FTCJOYIKG2ML2XOAD6SYAKAIRTKSKQM66GVTFASQ, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.846301186 (104h10m24.44307221s) [noise=true , address=79.127.205.233:4154 , piece_id=LDMVPR6RMS2F3YNIMVH2ELREQMP6WMEXRW5UDBHTZIPZEPWRD4BQ, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.846940319 (104h10m24.443711359s) [noise=true , address=193.149.164.70:4116 , piece_id=FVAICHAZFMXEXK5CQ3SXYQ46T54PQBGYVPPBSKZ7WOHSGOKPHWRQ, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.849741507 (104h10m24.446512533s) [noise=true , address=167.17.72.68:4654 , piece_id=WHKUMMAPP7ZSKFMAGKFHINHWIICOCMREK75F3EON723TKLAWDDAQ, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.850583955 (104h10m24.447354987s) [noise=true , address=167.17.72.70:4758 , piece_id=3OZLHHR5C6OX4EJK23YRLEGJOJ3IXWPR35CZ6MQARXGX5NNGXUGQ, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.851500001 (104h10m24.448271012s) [noise=true , address=77.90.31.17:4793 , piece_id=GB6MI5XNLXQL4KFADPIKGKDKIQ7LNZ65EOTKY7LLSQMYY7WS56FQ, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.851893521 (104h10m24.448664555s) [noise=true , address=5.231.73.4:4012 , piece_id=5RZOPQ6PFL42RJPI4ZR4GS756AU332JPC34SVWK4M2FELGI3O7CA, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.853333272 (104h10m24.450104308s) [noise=true , address=77.90.31.14:4690 , piece_id=XUIROGKLTQGEBWNIYKM5LA6U4DZG3ZYFMCHMNYCZJ2R4Y5WB7Z5A, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.853969417 (104h10m24.450740435s) [noise=true , address=77.90.31.12:4604 , piece_id=JUIZARJIP27O4PO3IZ54Y734TUIG2I3DUCQNFWI5TH5ZMCBDGJMA, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.854283058 (104h10m24.451054099s) [noise=true , address=145.79.227.2:4133 , piece_id=AXG6E4SMSMKGVOINK4HMHTXL6GRKQWU6NJOP633KYOKQ3HQHMFOA, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.854903526 (104h10m24.45167454s) [noise=true , address=77.90.31.11:4585 , piece_id=VX7QB2HCS64PW5BIRIJ3RUY7V5FMDVTO423ENNPOTSQUINRR6R5A, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.855071429 (104h10m24.451842462s) [noise=true , address=64.49.13.139:4474 , piece_id=HFVZT3V5KD42X64HCTKPEG2LWZJZAVOH5MA5CFCIMPB7WK7MWQNQ, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.855704995 (104h10m24.452476028s) [noise=true , address=193.149.164.69:4092 , piece_id=T63NKUCMVMLADZYIMCW3FMJLYURBF7YVSMQ7AFLCYOU2K3C5RYNQ, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.855767508 (104h10m24.45253852s) [noise=true , address=77.90.31.18:4814 , piece_id=OUCWOBYCYXTPTVHHOOILTZTLUB6PJ42HOV3YURYC3LTKMTDDGPSA, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.856577407 (104h10m24.453348417s) [noise=true , address=193.149.164.71:4306 , piece_id=SJ5TPY5AZVF55AK3MRAT6K5ZPA4V6N2JXBYILVM3NHTCTRZI6MLQ, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.857565751 (104h10m24.454336779s) [noise=true , address=64.49.13.140:4507 , piece_id=6PNJPCU3KVM4ZFBTXQVZG26IEG57G3HHDRLBXEAB3VTU6IEPCAWQ, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.858400866 (104h10m24.455171879s) [noise=true , address=64.49.13.170:4725 , piece_id=DEFHEGRC3SUXYVLYV2YR3KPBS4GLCPQBMV2GWR7KRY4ZBWDGAJJA, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.859258131 (104h10m24.456029145s) [noise=true , address=79.127.205.237:4245 , piece_id=ZO6RJQU6Z5XIV6NXYBV7PWACGAEP7PWJMA7VISBKQS7IRWOXNRZQ, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.934798292 (104h10m24.531569311s) [upload=116599]: Next returned nil segment
Mar 10 19:10:26.934817790 (104h10m24.531588824s) [upload=116599]: Waiting for error group managing segments...
Mar 10 19:10:26.999719507 (104h10m24.59649054s) [noise=true , address=145.79.227.2:4133 , piece_id=AXG6E4SMSMKGVOINK4HMHTXL6GRKQWU6NJOP633KYOKQ3HQHMFOA, seg_pos={0 0}, upload=116599]: Done uploading piece. err: <nil>
Mar 10 19:10:27.030173609 (104h10m24.626944622s) [noise=true , address=5.231.73.4:4012 , piece_id=5RZOPQ6PFL42RJPI4ZR4GS756AU332JPC34SVWK4M2FELGI3O7CA, seg_pos={0 0}, upload=116599]: Done uploading piece. err: <nil>
Mar 10 19:10:27.238694294 (104h10m24.83546533s) [noise=true , address=167.17.72.70:4758 , piece_id=3OZLHHR5C6OX4EJK23YRLEGJOJ3IXWPR35CZ6MQARXGX5NNGXUGQ, seg_pos={0 0}, upload=116599]: Done uploading piece. err: <nil>
Mar 10 19:10:27.294399709 (104h10m24.891170737s) [noise=true , address=167.17.72.68:4654 , piece_id=WHKUMMAPP7ZSKFMAGKFHINHWIICOCMREK75F3EON723TKLAWDDAQ, seg_pos={0 0}, upload=116599]: Done uploading piece. err: <nil>
Mar 10 19:10:27.393989371 (104h10m24.990760384s) [noise=true , address=79.127.205.233:4154 , piece_id=LDMVPR6RMS2F3YNIMVH2ELREQMP6WMEXRW5UDBHTZIPZEPWRD4BQ, seg_pos={0 0}, upload=116599]: Done uploading piece. err: <nil>
Mar 10 19:10:27.451000332 (104h10m25.047771355s) [noise=true , address=193.149.164.69:4092 , piece_id=T63NKUCMVMLADZYIMCW3FMJLYURBF7YVSMQ7AFLCYOU2K3C5RYNQ, seg_pos={0 0}, upload=116599]: Done uploading piece. err: <nil>
Mar 10 19:10:27.453000338 (104h10m25.04977136s) [noise=true , address=193.149.164.70:4116 , piece_id=FVAICHAZFMXEXK5CQ3SXYQ46T54PQBGYVPPBSKZ7WOHSGOKPHWRQ, seg_pos={0 0}, upload=116599]: Done uploading piece. err: <nil>
Mar 10 19:10:27.472395366 (104h10m25.069166393s) [noise=true , address=193.149.164.71:4306 , piece_id=SJ5TPY5AZVF55AK3MRAT6K5ZPA4V6N2JXBYILVM3NHTCTRZI6MLQ, seg_pos={0 0}, upload=116599]: Done uploading piece. err: <nil>
Mar 10 19:10:27.486423055 (104h10m25.083194088s) [noise=true , address=64.49.13.140:4507 , piece_id=6PNJPCU3KVM4ZFBTXQVZG26IEG57G3HHDRLBXEAB3VTU6IEPCAWQ, seg_pos={0 0}, upload=116599]: Done uploading piece. err: <nil>
Mar 10 19:10:27.487859041 (104h10m25.084630054s) [noise=true , address=64.49.13.139:4474 , piece_id=HFVZT3V5KD42X64HCTKPEG2LWZJZAVOH5MA5CFCIMPB7WK7MWQNQ, seg_pos={0 0}, upload=116599]: Done uploading piece. err: <nil>
Mar 10 19:10:27.489472801 (104h10m25.086243832s) [noise=true , address=64.49.13.170:4725 , piece_id=DEFHEGRC3SUXYVLYV2YR3KPBS4GLCPQBMV2GWR7KRY4ZBWDGAJJA, seg_pos={0 0}, upload=116599]: Done uploading piece. err: <nil>
Mar 10 19:10:28.942749218 (104h10m26.539520243s) [noise=true , address=79.127.205.237:4245 , piece_id=ZO6RJQU6Z5XIV6NXYBV7PWACGAEP7PWJMA7VISBKQS7IRWOXNRZQ, seg_pos={0 0}, upload=116599]: Done uploading piece. err: <nil>
Mar 10 19:11:26.775681128 (104h11m24.37245216s) [noise=true , address=77.90.31.17:4793 , piece_id=GB6MI5XNLXQL4KFADPIKGKDKIQ7LNZ65EOTKY7LLSQMYY7WS56FQ, seg_pos={0 0}, upload=116599]: Done uploading piece. err: ecclient: upload failed (node:1rxrH2F4LTfrK9SMsNNEuqVr4AjV1VbEYDc3veWbQf88rj2Es4, address:77.90.31.17:4793): protocol: rpc: dial tcp 77.90.31.17:4793: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:11:26.775748621 (104h11m24.37251965s) [noise=true , address=77.90.31.12:4604 , piece_id=JUIZARJIP27O4PO3IZ54Y734TUIG2I3DUCQNFWI5TH5ZMCBDGJMA, seg_pos={0 0}, upload=116599]: Done uploading piece. err: ecclient: upload failed (node:1vK9QCCj1rszVCR72YBc8BdKCPRno49RrcH9uGQb9rBJkZY4F7, address:77.90.31.12:4604): protocol: rpc: dial tcp 77.90.31.12:4604: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:11:26.775972253 (104h11m24.372743279s) [noise=true , address=77.90.31.16:4754 , piece_id=PE4KKNG6B7T5FTCJOYIKG2ML2XOAD6SYAKAIRTKSKQM66GVTFASQ, seg_pos={0 0}, upload=116599]: Done uploading piece. err: ecclient: upload failed (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8, address:77.90.31.16:4754): protocol: write tcp 159.223.143.39:58400->77.90.31.16:4754: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:11:26.776001817 (104h11m24.37277285s) [noise=true , address=77.90.31.14:4690 , piece_id=XUIROGKLTQGEBWNIYKM5LA6U4DZG3ZYFMCHMNYCZJ2R4Y5WB7Z5A, seg_pos={0 0}, upload=116599]: Done uploading piece. err: ecclient: upload failed (node:1WPKeCcN1uEhGjZHAJgCoJMaHxAWouGCdaU1vZwAS7kX22rxcv, address:77.90.31.14:4690): protocol: rpc: dial tcp 77.90.31.14:4690: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:11:26.776244713 (104h11m24.373015806s) [noise=true , address=77.90.31.18:4814 , piece_id=OUCWOBYCYXTPTVHHOOILTZTLUB6PJ42HOV3YURYC3LTKMTDDGPSA, seg_pos={0 0}, upload=116599]: Done uploading piece. err: ecclient: upload failed (node:1FCikLXgPYS2qM9zLTthSNHKsQNckzg9oNkKGxfvwJZZjhKkVf, address:77.90.31.18:4814): protocol: write tcp 159.223.143.39:36246->77.90.31.18:4814: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:11:26.776280556 (104h11m24.373051587s) [noise=true , address=77.90.31.11:4585 , piece_id=VX7QB2HCS64PW5BIRIJ3RUY7V5FMDVTO423ENNPOTSQUINRR6R5A, seg_pos={0 0}, upload=116599]: Done uploading piece. err: ecclient: upload failed (node:1Tve8WUWhJCK2cVWLbxxnz3WVzR2bi2wLx5XrF6QbhSN4HcsQG, address:77.90.31.11:4585): protocol: context deadline exceeded; context deadline exceeded
Mar 10 19:11:26.776313694 (104h11m24.373084705s) [seg_pos={0 0}, upload=116599]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil>
Mar 10 19:11:26.804828285 (104h11m24.401599298s) [upload=116599]: Done uploading segments...
Mar 10 19:11:26.805017989 (104h11m24.401789013s) [upload=116599]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12)
Duration: 1.060087222s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
1.021671687s trace:
https://jaeger.dev.storj.io/trace/69062766ac2178dd
FAIL (2026-03-10 19:53:38 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 12)
======LOGS======
Mar 10 19:10:26.813115721 (104h10m24.409886745s) [upload=116599]: Starting upload
Mar 10 19:10:26.813211715 (104h10m24.409982738s) [upload=116599]: Done starting upload
Mar 10 19:10:26.813505664 (104h10m24.410276708s) [upload=116599]: Uploading segments...
Mar 10 19:10:26.813650528 (104h10m24.410421539s) [upload=116599]: Got next segment. Inline: false
Mar 10 19:10:26.813743195 (104h10m24.410514218s) [upload=116599]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Mar 10 19:10:26.813750388 (104h10m24.410521411s) [upload=116599]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Mar 10 19:10:26.846063672 (104h10m24.442834685s) [seg_pos={0 0}, upload=116599]: Begin upload segment...
Mar 10 19:10:26.846148290 (104h10m24.442919315s) [seg_pos={0 0}, upload=116599]: Done begin upload segment.
Mar 10 19:10:26.846228010 (104h10m24.442999069s) [noise=true , address=77.90.31.16:4754 , piece_id=PE4KKNG6B7T5FTCJOYIKG2ML2XOAD6SYAKAIRTKSKQM66GVTFASQ, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.846301186 (104h10m24.44307221s) [noise=true , address=79.127.205.233:4154 , piece_id=LDMVPR6RMS2F3YNIMVH2ELREQMP6WMEXRW5UDBHTZIPZEPWRD4BQ, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.846940319 (104h10m24.443711359s) [noise=true , address=193.149.164.70:4116 , piece_id=FVAICHAZFMXEXK5CQ3SXYQ46T54PQBGYVPPBSKZ7WOHSGOKPHWRQ, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.849741507 (104h10m24.446512533s) [noise=true , address=167.17.72.68:4654 , piece_id=WHKUMMAPP7ZSKFMAGKFHINHWIICOCMREK75F3EON723TKLAWDDAQ, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.850583955 (104h10m24.447354987s) [noise=true , address=167.17.72.70:4758 , piece_id=3OZLHHR5C6OX4EJK23YRLEGJOJ3IXWPR35CZ6MQARXGX5NNGXUGQ, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.851500001 (104h10m24.448271012s) [noise=true , address=77.90.31.17:4793 , piece_id=GB6MI5XNLXQL4KFADPIKGKDKIQ7LNZ65EOTKY7LLSQMYY7WS56FQ, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.851893521 (104h10m24.448664555s) [noise=true , address=5.231.73.4:4012 , piece_id=5RZOPQ6PFL42RJPI4ZR4GS756AU332JPC34SVWK4M2FELGI3O7CA, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.853333272 (104h10m24.450104308s) [noise=true , address=77.90.31.14:4690 , piece_id=XUIROGKLTQGEBWNIYKM5LA6U4DZG3ZYFMCHMNYCZJ2R4Y5WB7Z5A, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.853969417 (104h10m24.450740435s) [noise=true , address=77.90.31.12:4604 , piece_id=JUIZARJIP27O4PO3IZ54Y734TUIG2I3DUCQNFWI5TH5ZMCBDGJMA, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.854283058 (104h10m24.451054099s) [noise=true , address=145.79.227.2:4133 , piece_id=AXG6E4SMSMKGVOINK4HMHTXL6GRKQWU6NJOP633KYOKQ3HQHMFOA, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.854903526 (104h10m24.45167454s) [noise=true , address=77.90.31.11:4585 , piece_id=VX7QB2HCS64PW5BIRIJ3RUY7V5FMDVTO423ENNPOTSQUINRR6R5A, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.855071429 (104h10m24.451842462s) [noise=true , address=64.49.13.139:4474 , piece_id=HFVZT3V5KD42X64HCTKPEG2LWZJZAVOH5MA5CFCIMPB7WK7MWQNQ, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.855704995 (104h10m24.452476028s) [noise=true , address=193.149.164.69:4092 , piece_id=T63NKUCMVMLADZYIMCW3FMJLYURBF7YVSMQ7AFLCYOU2K3C5RYNQ, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.855767508 (104h10m24.45253852s) [noise=true , address=77.90.31.18:4814 , piece_id=OUCWOBYCYXTPTVHHOOILTZTLUB6PJ42HOV3YURYC3LTKMTDDGPSA, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.856577407 (104h10m24.453348417s) [noise=true , address=193.149.164.71:4306 , piece_id=SJ5TPY5AZVF55AK3MRAT6K5ZPA4V6N2JXBYILVM3NHTCTRZI6MLQ, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.857565751 (104h10m24.454336779s) [noise=true , address=64.49.13.140:4507 , piece_id=6PNJPCU3KVM4ZFBTXQVZG26IEG57G3HHDRLBXEAB3VTU6IEPCAWQ, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.858400866 (104h10m24.455171879s) [noise=true , address=64.49.13.170:4725 , piece_id=DEFHEGRC3SUXYVLYV2YR3KPBS4GLCPQBMV2GWR7KRY4ZBWDGAJJA, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.859258131 (104h10m24.456029145s) [noise=true , address=79.127.205.237:4245 , piece_id=ZO6RJQU6Z5XIV6NXYBV7PWACGAEP7PWJMA7VISBKQS7IRWOXNRZQ, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.934798292 (104h10m24.531569311s) [upload=116599]: Next returned nil segment
Mar 10 19:10:26.934817790 (104h10m24.531588824s) [upload=116599]: Waiting for error group managing segments...
Mar 10 19:10:26.999719507 (104h10m24.59649054s) [noise=true , address=145.79.227.2:4133 , piece_id=AXG6E4SMSMKGVOINK4HMHTXL6GRKQWU6NJOP633KYOKQ3HQHMFOA, seg_pos={0 0}, upload=116599]: Done uploading piece. err: <nil>
Mar 10 19:10:27.030173609 (104h10m24.626944622s) [noise=true , address=5.231.73.4:4012 , piece_id=5RZOPQ6PFL42RJPI4ZR4GS756AU332JPC34SVWK4M2FELGI3O7CA, seg_pos={0 0}, upload=116599]: Done uploading piece. err: <nil>
Mar 10 19:10:27.238694294 (104h10m24.83546533s) [noise=true , address=167.17.72.70:4758 , piece_id=3OZLHHR5C6OX4EJK23YRLEGJOJ3IXWPR35CZ6MQARXGX5NNGXUGQ, seg_pos={0 0}, upload=116599]: Done uploading piece. err: <nil>
Mar 10 19:10:27.294399709 (104h10m24.891170737s) [noise=true , address=167.17.72.68:4654 , piece_id=WHKUMMAPP7ZSKFMAGKFHINHWIICOCMREK75F3EON723TKLAWDDAQ, seg_pos={0 0}, upload=116599]: Done uploading piece. err: <nil>
Mar 10 19:10:27.393989371 (104h10m24.990760384s) [noise=true , address=79.127.205.233:4154 , piece_id=LDMVPR6RMS2F3YNIMVH2ELREQMP6WMEXRW5UDBHTZIPZEPWRD4BQ, seg_pos={0 0}, upload=116599]: Done uploading piece. err: <nil>
Mar 10 19:10:27.451000332 (104h10m25.047771355s) [noise=true , address=193.149.164.69:4092 , piece_id=T63NKUCMVMLADZYIMCW3FMJLYURBF7YVSMQ7AFLCYOU2K3C5RYNQ, seg_pos={0 0}, upload=116599]: Done uploading piece. err: <nil>
Mar 10 19:10:27.453000338 (104h10m25.04977136s) [noise=true , address=193.149.164.70:4116 , piece_id=FVAICHAZFMXEXK5CQ3SXYQ46T54PQBGYVPPBSKZ7WOHSGOKPHWRQ, seg_pos={0 0}, upload=116599]: Done uploading piece. err: <nil>
Mar 10 19:10:27.472395366 (104h10m25.069166393s) [noise=true , address=193.149.164.71:4306 , piece_id=SJ5TPY5AZVF55AK3MRAT6K5ZPA4V6N2JXBYILVM3NHTCTRZI6MLQ, seg_pos={0 0}, upload=116599]: Done uploading piece. err: <nil>
Mar 10 19:10:27.486423055 (104h10m25.083194088s) [noise=true , address=64.49.13.140:4507 , piece_id=6PNJPCU3KVM4ZFBTXQVZG26IEG57G3HHDRLBXEAB3VTU6IEPCAWQ, seg_pos={0 0}, upload=116599]: Done uploading piece. err: <nil>
Mar 10 19:10:27.487859041 (104h10m25.084630054s) [noise=true , address=64.49.13.139:4474 , piece_id=HFVZT3V5KD42X64HCTKPEG2LWZJZAVOH5MA5CFCIMPB7WK7MWQNQ, seg_pos={0 0}, upload=116599]: Done uploading piece. err: <nil>
Mar 10 19:10:27.489472801 (104h10m25.086243832s) [noise=true , address=64.49.13.170:4725 , piece_id=DEFHEGRC3SUXYVLYV2YR3KPBS4GLCPQBMV2GWR7KRY4ZBWDGAJJA, seg_pos={0 0}, upload=116599]: Done uploading piece. err: <nil>
Mar 10 19:10:28.942749218 (104h10m26.539520243s) [noise=true , address=79.127.205.237:4245 , piece_id=ZO6RJQU6Z5XIV6NXYBV7PWACGAEP7PWJMA7VISBKQS7IRWOXNRZQ, seg_pos={0 0}, upload=116599]: Done uploading piece. err: <nil>
Mar 10 19:11:26.775681128 (104h11m24.37245216s) [noise=true , address=77.90.31.17:4793 , piece_id=GB6MI5XNLXQL4KFADPIKGKDKIQ7LNZ65EOTKY7LLSQMYY7WS56FQ, seg_pos={0 0}, upload=116599]: Done uploading piece. err: ecclient: upload failed (node:1rxrH2F4LTfrK9SMsNNEuqVr4AjV1VbEYDc3veWbQf88rj2Es4, address:77.90.31.17:4793): protocol: rpc: dial tcp 77.90.31.17:4793: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:11:26.775748621 (104h11m24.37251965s) [noise=true , address=77.90.31.12:4604 , piece_id=JUIZARJIP27O4PO3IZ54Y734TUIG2I3DUCQNFWI5TH5ZMCBDGJMA, seg_pos={0 0}, upload=116599]: Done uploading piece. err: ecclient: upload failed (node:1vK9QCCj1rszVCR72YBc8BdKCPRno49RrcH9uGQb9rBJkZY4F7, address:77.90.31.12:4604): protocol: rpc: dial tcp 77.90.31.12:4604: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:11:26.775972253 (104h11m24.372743279s) [noise=true , address=77.90.31.16:4754 , piece_id=PE4KKNG6B7T5FTCJOYIKG2ML2XOAD6SYAKAIRTKSKQM66GVTFASQ, seg_pos={0 0}, upload=116599]: Done uploading piece. err: ecclient: upload failed (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8, address:77.90.31.16:4754): protocol: write tcp 159.223.143.39:58400->77.90.31.16:4754: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:11:26.776001817 (104h11m24.37277285s) [noise=true , address=77.90.31.14:4690 , piece_id=XUIROGKLTQGEBWNIYKM5LA6U4DZG3ZYFMCHMNYCZJ2R4Y5WB7Z5A, seg_pos={0 0}, upload=116599]: Done uploading piece. err: ecclient: upload failed (node:1WPKeCcN1uEhGjZHAJgCoJMaHxAWouGCdaU1vZwAS7kX22rxcv, address:77.90.31.14:4690): protocol: rpc: dial tcp 77.90.31.14:4690: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:11:26.776244713 (104h11m24.373015806s) [noise=true , address=77.90.31.18:4814 , piece_id=OUCWOBYCYXTPTVHHOOILTZTLUB6PJ42HOV3YURYC3LTKMTDDGPSA, seg_pos={0 0}, upload=116599]: Done uploading piece. err: ecclient: upload failed (node:1FCikLXgPYS2qM9zLTthSNHKsQNckzg9oNkKGxfvwJZZjhKkVf, address:77.90.31.18:4814): protocol: write tcp 159.223.143.39:36246->77.90.31.18:4814: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:11:26.776280556 (104h11m24.373051587s) [noise=true , address=77.90.31.11:4585 , piece_id=VX7QB2HCS64PW5BIRIJ3RUY7V5FMDVTO423ENNPOTSQUINRR6R5A, seg_pos={0 0}, upload=116599]: Done uploading piece. err: ecclient: upload failed (node:1Tve8WUWhJCK2cVWLbxxnz3WVzR2bi2wLx5XrF6QbhSN4HcsQG, address:77.90.31.11:4585): protocol: context deadline exceeded; context deadline exceeded
Mar 10 19:11:26.776313694 (104h11m24.373084705s) [seg_pos={0 0}, upload=116599]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil>
Mar 10 19:11:26.804828285 (104h11m24.401599298s) [upload=116599]: Done uploading segments...
Mar 10 19:11:26.805017989 (104h11m24.401789013s) [upload=116599]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12)
Duration: 1.021696389s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
6 out of 60 is failed (threshold=3) [..X......XX.X...........X.......X...........................] duration:
2.170033839s trace:
https://jaeger.dev.storj.io/trace/78c3ad63aac6b527
FAIL (2026-03-10 19:53:38 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
:03.719970769 (104h35m1.316741801s) [noise=true , address=185.167.204.205:4033 , piece_id=6JIW66HORXSEPSDWIUONOUZNA2UNXKBA4A6VZ7PBWW3IN2SCJSPQ, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.720006314 (104h35m1.316777347s) [noise=true , address=185.167.204.205:4024 , piece_id=3CKPLBO33LOMNYBF47K2KGQFPO53ZXLMKRKWPVSR4FKQKXKMZMCA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.721245847 (104h35m1.318016883s) [noise=true , address=185.167.204.206:4046 , piece_id=5WVC4R6VY25QUMGSOAORUK3WPGIADBXLKN7ZSLJ4GQEVKMSHZPKA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.722131967 (104h35m1.318902999s) [noise=true , address=89.149.201.147:4004 , piece_id=TMKNRIR6RWRW3TUSA2MQL4LCNTJ3LKEFIT6BAVRX2V54QHNF26FA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.722154312 (104h35m1.318925349s) [noise=true , address=185.167.204.205:4028 , piece_id=IQ4CMHEIUNHIUBQV2PVFQ6BURDMWU2S55VGFP5Y7HPFSDVBY6PYA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.723422869 (104h35m1.320193903s) [noise=true , address=81.171.14.70:4047 , piece_id=DLCUNYJFEKIU3MWFXK24QGJ4GW2H5XE4Y2ARNL2W55EZUMSC4Z7Q, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.724205203 (104h35m1.320976237s) [noise=true , address=185.167.204.205:4022 , piece_id=F5UFUTAFPRBWMSU3QKV7ZSYQR6UUC7SBUDOC7AAYGDXTC5VYUBHA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.727803141 (104h35m1.324574176s) [noise=true , address=185.167.204.206:4067 , piece_id=RH2DWZ3FL7JX6EJ2WTJBPBSWDYVSHLYSLZCMMGRQXUDR6QXVZZ5A, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:04.167055423 (104h35m1.763826466s) [upload=117052]: Next returned nil segment
Mar 10 19:35:04.167072399 (104h35m1.763843424s) [upload=117052]: Waiting for error group managing segments...
Mar 10 19:35:04.350286979 (104h35m1.947058004s) [noise=true , address=81.171.14.70:4035 , piece_id=RNJYIOH4ETYQBSZFYTV3FX73J3PNFR2KBYNZEDXCVBOYMUAN5FCA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.351883208 (104h35m1.948654233s) [noise=true , address=81.171.14.70:4043 , piece_id=LSOHRZMEV5TEQUKZIJPYHVP43CIZKMXBTRVKIZIQNSSFNMAA5FLQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.356149085 (104h35m1.952920098s) [noise=true , address=185.167.204.205:4036 , piece_id=BBGCJ7U7PQDNB5NZTJWHUWXR6TFYF6D7LOR2UUBNV647YQX5JD2Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.369094269 (104h35m1.965865391s) [noise=true , address=89.149.201.147:4011 , piece_id=YHPLXK3ADD6VMYDWQ55PLROBCY2IOLOR2ZWEZ22FRRA4TXKSSWIQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.382982597 (104h35m1.979753611s) [noise=true , address=81.171.14.70:4034 , piece_id=BPXMDI54RTQVH5QKY6JCN5H3AKCPH62V76VBES4GQ6SLIZDIMMOA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.384672279 (104h35m1.981443307s) [noise=true , address=81.171.14.70:4047 , piece_id=DLCUNYJFEKIU3MWFXK24QGJ4GW2H5XE4Y2ARNL2W55EZUMSC4Z7Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.386337288 (104h35m1.98310832s) [noise=true , address=81.171.14.70:4025 , piece_id=R62A3AXM4OKUZFZJ2LB55UPB6VRGHBQLMXXVBWDTHUDWPGPIWEVQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.387583761 (104h35m1.984354793s) [noise=true , address=89.149.201.147:4009 , piece_id=XFSD3W6JWUXGXTHYE57SS5E5322JTRG7RKBI7MBF25GU7SEAKZJQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.388110220 (104h35m1.984881248s) [noise=true , address=81.171.14.70:4026 , piece_id=4CBQ4RZ44USXNYAGA3CIGNC3LBQXR675JVJX6CVTV3X7KVYPRHQA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.389451350 (104h35m1.986222362s) [noise=true , address=185.167.204.205:4012 , piece_id=7UFJIUKCHPKYZF42HTP6Q5D2FMHRSCRUQ7MH2MYJSEOZHZAJRZXA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.404093624 (104h35m2.00086464s) [noise=true , address=81.171.14.70:4048 , piece_id=4F2F4X4ZJ4FXUYQJQKB7YB3JZY53QNVRUSM3N4HQKGEYHMFYQB2A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.413111656 (104h35m2.009882687s) [noise=true , address=185.167.204.206:4071 , piece_id=ZQJLN55G37453GZS54ZDTPM7CEU5GH6LJR3BIKSJDC3C4QOKW5YQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.416183369 (104h35m2.012954404s) [noise=true , address=89.149.201.147:4014 , piece_id=WZOPRPDLFVOO6O2JYZZB75CLIAA4SVM7OEZXBX6WOJV3PUL2YSAQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.449375785 (104h35m2.046146815s) [noise=true , address=185.167.204.206:4039 , piece_id=DKKIZZMA3RAKMLUJTU6OANGBA3F6XWRD5ELVNXAGBIOZMFBVQWZQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.457716093 (104h35m2.054487108s) [noise=true , address=185.167.204.205:4024 , piece_id=3CKPLBO33LOMNYBF47K2KGQFPO53ZXLMKRKWPVSR4FKQKXKMZMCA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.491116161 (104h35m2.087887192s) [noise=true , address=185.167.204.205:4005 , piece_id=E7RO77MCAMFEAIE7ETPALCBV5S4X3DG5ZUD27ZE55RXYICIIXNEQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.507099744 (104h35m2.103870782s) [noise=true , address=185.167.204.205:4029 , piece_id=R33ONZELVLYL76YTJKOZASDJKFBACNNMC3PDUIITQRDGBWWZUNKA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.537139239 (104h35m2.133910269s) [noise=true , address=185.167.204.206:4067 , piece_id=RH2DWZ3FL7JX6EJ2WTJBPBSWDYVSHLYSLZCMMGRQXUDR6QXVZZ5A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.787100189 (104h35m2.383871224s) [noise=true , address=81.171.14.70:4033 , piece_id=ZFV477QXRX42OJIG4D5JLOYU3WERTD65PMHI5MFQLRTQJG3ENUTA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.962846444 (104h35m2.559617469s) [noise=true , address=89.149.201.147:4004 , piece_id=TMKNRIR6RWRW3TUSA2MQL4LCNTJ3LKEFIT6BAVRX2V54QHNF26FA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:08.299132133 (104h35m5.895903147s) [noise=true , address=185.167.204.205:4028 , piece_id=IQ4CMHEIUNHIUBQV2PVFQ6BURDMWU2S55VGFP5Y7HPFSDVBY6PYA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:08.481127955 (104h35m6.077898968s) [noise=true , address=185.167.204.206:4051 , piece_id=GO4OTIL5WFOS7V77FAT2NRFG2HRE2HXKMU2C3XEGDBGPQL5U5HUA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:10.314952727 (104h35m7.91172374s) [noise=true , address=185.167.204.206:4046 , piece_id=5WVC4R6VY25QUMGSOAORUK3WPGIADBXLKN7ZSLJ4GQEVKMSHZPKA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:36:03.322697363 (104h36m0.919468398s) [noise=true , address=185.167.204.206:4062 , piece_id=7NP2ZIMW6ZDXW3YHZSBO5PRTAZPLYLDWXDA4QPN5XH2C6KAGYLEA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:12BL6XLqsoUGm2FxrTgYTxLZakt2u2CAhZJG82oJ31eaE4vTtMr, address:185.167.204.206:4062): protocol: write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection; write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection
Mar 10 19:36:03.322771647 (104h36m0.919542676s) [noise=true , address=89.149.201.147:4007 , piece_id=AAJY5QBN55ENHTODH25WBVCFDT2SAQ2F27TKHFCIURCZSBMGOY6A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1Q7d4vj1KoWhDr6xd9gEfFrZ1sV3Yro2AFke2f3Ke1rXsGnvew, address:89.149.201.147:4007): protocol: read tcp 159.223.143.39:54460->89.149.201.147:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.322958851 (104h36m0.919729884s) [noise=true , address=185.167.204.205:4033 , piece_id=6JIW66HORXSEPSDWIUONOUZNA2UNXKBA4A6VZ7PBWW3IN2SCJSPQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:12Q73mXHLEfuX4VUbJ8PTjhMoXCrYSKjJbFxqjDdqF3U99N8FXe, address:185.167.204.205:4033): protocol: write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection; write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection
Mar 10 19:36:03.323134221 (104h36m0.919905244s) [noise=true , address=89.149.201.147:4018 , piece_id=V3KLJ75HJ3NRMBP2FIM33GC45QW4U5XMN2R4IAHSX32COGBJF47Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1Ku7o12bSmJ1YR3dGetGFcWqs1wJEh53Zkg3H7ELJrt4w1hkua, address:89.149.201.147:4018): protocol: read tcp 159.223.143.39:47122->89.149.201.147:4018: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.323461923 (104h36m0.920232956s) [noise=true , address=185.167.204.205:4022 , piece_id=F5UFUTAFPRBWMSU3QKV7ZSYQR6UUC7SBUDOC7AAYGDXTC5VYUBHA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1fT9qH1igPUNfrzpwa2VkBX7heh475sNqoN4Nsf9FYSwi1VmHj, address:185.167.204.205:4022): protocol: read tcp 159.223.143.39:51226->185.167.204.205:4022: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.323503666 (104h36m0.920274689s) [seg_pos={0 0}, upload=117052]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Mar 10 19:36:03.424787357 (104h36m1.021558384s) [upload=117052]: Done uploading segments...
Mar 10 19:36:03.424860036 (104h36m1.021631063s) [upload=117052]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 2.170144964s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................] duration:
1.101604944s trace:
https://jaeger.dev.storj.io/trace/69062766ac2178dd
FAIL (2026-03-10 19:52:50 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 12)
======LOGS======
Mar 10 19:10:26.813115721 (104h10m24.409886745s) [upload=116599]: Starting upload
Mar 10 19:10:26.813211715 (104h10m24.409982738s) [upload=116599]: Done starting upload
Mar 10 19:10:26.813505664 (104h10m24.410276708s) [upload=116599]: Uploading segments...
Mar 10 19:10:26.813650528 (104h10m24.410421539s) [upload=116599]: Got next segment. Inline: false
Mar 10 19:10:26.813743195 (104h10m24.410514218s) [upload=116599]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Mar 10 19:10:26.813750388 (104h10m24.410521411s) [upload=116599]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Mar 10 19:10:26.846063672 (104h10m24.442834685s) [seg_pos={0 0}, upload=116599]: Begin upload segment...
Mar 10 19:10:26.846148290 (104h10m24.442919315s) [seg_pos={0 0}, upload=116599]: Done begin upload segment.
Mar 10 19:10:26.846228010 (104h10m24.442999069s) [noise=true , address=77.90.31.16:4754 , piece_id=PE4KKNG6B7T5FTCJOYIKG2ML2XOAD6SYAKAIRTKSKQM66GVTFASQ, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.846301186 (104h10m24.44307221s) [noise=true , address=79.127.205.233:4154 , piece_id=LDMVPR6RMS2F3YNIMVH2ELREQMP6WMEXRW5UDBHTZIPZEPWRD4BQ, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.846940319 (104h10m24.443711359s) [noise=true , address=193.149.164.70:4116 , piece_id=FVAICHAZFMXEXK5CQ3SXYQ46T54PQBGYVPPBSKZ7WOHSGOKPHWRQ, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.849741507 (104h10m24.446512533s) [noise=true , address=167.17.72.68:4654 , piece_id=WHKUMMAPP7ZSKFMAGKFHINHWIICOCMREK75F3EON723TKLAWDDAQ, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.850583955 (104h10m24.447354987s) [noise=true , address=167.17.72.70:4758 , piece_id=3OZLHHR5C6OX4EJK23YRLEGJOJ3IXWPR35CZ6MQARXGX5NNGXUGQ, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.851500001 (104h10m24.448271012s) [noise=true , address=77.90.31.17:4793 , piece_id=GB6MI5XNLXQL4KFADPIKGKDKIQ7LNZ65EOTKY7LLSQMYY7WS56FQ, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.851893521 (104h10m24.448664555s) [noise=true , address=5.231.73.4:4012 , piece_id=5RZOPQ6PFL42RJPI4ZR4GS756AU332JPC34SVWK4M2FELGI3O7CA, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.853333272 (104h10m24.450104308s) [noise=true , address=77.90.31.14:4690 , piece_id=XUIROGKLTQGEBWNIYKM5LA6U4DZG3ZYFMCHMNYCZJ2R4Y5WB7Z5A, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.853969417 (104h10m24.450740435s) [noise=true , address=77.90.31.12:4604 , piece_id=JUIZARJIP27O4PO3IZ54Y734TUIG2I3DUCQNFWI5TH5ZMCBDGJMA, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.854283058 (104h10m24.451054099s) [noise=true , address=145.79.227.2:4133 , piece_id=AXG6E4SMSMKGVOINK4HMHTXL6GRKQWU6NJOP633KYOKQ3HQHMFOA, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.854903526 (104h10m24.45167454s) [noise=true , address=77.90.31.11:4585 , piece_id=VX7QB2HCS64PW5BIRIJ3RUY7V5FMDVTO423ENNPOTSQUINRR6R5A, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.855071429 (104h10m24.451842462s) [noise=true , address=64.49.13.139:4474 , piece_id=HFVZT3V5KD42X64HCTKPEG2LWZJZAVOH5MA5CFCIMPB7WK7MWQNQ, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.855704995 (104h10m24.452476028s) [noise=true , address=193.149.164.69:4092 , piece_id=T63NKUCMVMLADZYIMCW3FMJLYURBF7YVSMQ7AFLCYOU2K3C5RYNQ, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.855767508 (104h10m24.45253852s) [noise=true , address=77.90.31.18:4814 , piece_id=OUCWOBYCYXTPTVHHOOILTZTLUB6PJ42HOV3YURYC3LTKMTDDGPSA, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.856577407 (104h10m24.453348417s) [noise=true , address=193.149.164.71:4306 , piece_id=SJ5TPY5AZVF55AK3MRAT6K5ZPA4V6N2JXBYILVM3NHTCTRZI6MLQ, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.857565751 (104h10m24.454336779s) [noise=true , address=64.49.13.140:4507 , piece_id=6PNJPCU3KVM4ZFBTXQVZG26IEG57G3HHDRLBXEAB3VTU6IEPCAWQ, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.858400866 (104h10m24.455171879s) [noise=true , address=64.49.13.170:4725 , piece_id=DEFHEGRC3SUXYVLYV2YR3KPBS4GLCPQBMV2GWR7KRY4ZBWDGAJJA, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.859258131 (104h10m24.456029145s) [noise=true , address=79.127.205.237:4245 , piece_id=ZO6RJQU6Z5XIV6NXYBV7PWACGAEP7PWJMA7VISBKQS7IRWOXNRZQ, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.934798292 (104h10m24.531569311s) [upload=116599]: Next returned nil segment
Mar 10 19:10:26.934817790 (104h10m24.531588824s) [upload=116599]: Waiting for error group managing segments...
Mar 10 19:10:26.999719507 (104h10m24.59649054s) [noise=true , address=145.79.227.2:4133 , piece_id=AXG6E4SMSMKGVOINK4HMHTXL6GRKQWU6NJOP633KYOKQ3HQHMFOA, seg_pos={0 0}, upload=116599]: Done uploading piece. err: <nil>
Mar 10 19:10:27.030173609 (104h10m24.626944622s) [noise=true , address=5.231.73.4:4012 , piece_id=5RZOPQ6PFL42RJPI4ZR4GS756AU332JPC34SVWK4M2FELGI3O7CA, seg_pos={0 0}, upload=116599]: Done uploading piece. err: <nil>
Mar 10 19:10:27.238694294 (104h10m24.83546533s) [noise=true , address=167.17.72.70:4758 , piece_id=3OZLHHR5C6OX4EJK23YRLEGJOJ3IXWPR35CZ6MQARXGX5NNGXUGQ, seg_pos={0 0}, upload=116599]: Done uploading piece. err: <nil>
Mar 10 19:10:27.294399709 (104h10m24.891170737s) [noise=true , address=167.17.72.68:4654 , piece_id=WHKUMMAPP7ZSKFMAGKFHINHWIICOCMREK75F3EON723TKLAWDDAQ, seg_pos={0 0}, upload=116599]: Done uploading piece. err: <nil>
Mar 10 19:10:27.393989371 (104h10m24.990760384s) [noise=true , address=79.127.205.233:4154 , piece_id=LDMVPR6RMS2F3YNIMVH2ELREQMP6WMEXRW5UDBHTZIPZEPWRD4BQ, seg_pos={0 0}, upload=116599]: Done uploading piece. err: <nil>
Mar 10 19:10:27.451000332 (104h10m25.047771355s) [noise=true , address=193.149.164.69:4092 , piece_id=T63NKUCMVMLADZYIMCW3FMJLYURBF7YVSMQ7AFLCYOU2K3C5RYNQ, seg_pos={0 0}, upload=116599]: Done uploading piece. err: <nil>
Mar 10 19:10:27.453000338 (104h10m25.04977136s) [noise=true , address=193.149.164.70:4116 , piece_id=FVAICHAZFMXEXK5CQ3SXYQ46T54PQBGYVPPBSKZ7WOHSGOKPHWRQ, seg_pos={0 0}, upload=116599]: Done uploading piece. err: <nil>
Mar 10 19:10:27.472395366 (104h10m25.069166393s) [noise=true , address=193.149.164.71:4306 , piece_id=SJ5TPY5AZVF55AK3MRAT6K5ZPA4V6N2JXBYILVM3NHTCTRZI6MLQ, seg_pos={0 0}, upload=116599]: Done uploading piece. err: <nil>
Mar 10 19:10:27.486423055 (104h10m25.083194088s) [noise=true , address=64.49.13.140:4507 , piece_id=6PNJPCU3KVM4ZFBTXQVZG26IEG57G3HHDRLBXEAB3VTU6IEPCAWQ, seg_pos={0 0}, upload=116599]: Done uploading piece. err: <nil>
Mar 10 19:10:27.487859041 (104h10m25.084630054s) [noise=true , address=64.49.13.139:4474 , piece_id=HFVZT3V5KD42X64HCTKPEG2LWZJZAVOH5MA5CFCIMPB7WK7MWQNQ, seg_pos={0 0}, upload=116599]: Done uploading piece. err: <nil>
Mar 10 19:10:27.489472801 (104h10m25.086243832s) [noise=true , address=64.49.13.170:4725 , piece_id=DEFHEGRC3SUXYVLYV2YR3KPBS4GLCPQBMV2GWR7KRY4ZBWDGAJJA, seg_pos={0 0}, upload=116599]: Done uploading piece. err: <nil>
Mar 10 19:10:28.942749218 (104h10m26.539520243s) [noise=true , address=79.127.205.237:4245 , piece_id=ZO6RJQU6Z5XIV6NXYBV7PWACGAEP7PWJMA7VISBKQS7IRWOXNRZQ, seg_pos={0 0}, upload=116599]: Done uploading piece. err: <nil>
Mar 10 19:11:26.775681128 (104h11m24.37245216s) [noise=true , address=77.90.31.17:4793 , piece_id=GB6MI5XNLXQL4KFADPIKGKDKIQ7LNZ65EOTKY7LLSQMYY7WS56FQ, seg_pos={0 0}, upload=116599]: Done uploading piece. err: ecclient: upload failed (node:1rxrH2F4LTfrK9SMsNNEuqVr4AjV1VbEYDc3veWbQf88rj2Es4, address:77.90.31.17:4793): protocol: rpc: dial tcp 77.90.31.17:4793: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:11:26.775748621 (104h11m24.37251965s) [noise=true , address=77.90.31.12:4604 , piece_id=JUIZARJIP27O4PO3IZ54Y734TUIG2I3DUCQNFWI5TH5ZMCBDGJMA, seg_pos={0 0}, upload=116599]: Done uploading piece. err: ecclient: upload failed (node:1vK9QCCj1rszVCR72YBc8BdKCPRno49RrcH9uGQb9rBJkZY4F7, address:77.90.31.12:4604): protocol: rpc: dial tcp 77.90.31.12:4604: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:11:26.775972253 (104h11m24.372743279s) [noise=true , address=77.90.31.16:4754 , piece_id=PE4KKNG6B7T5FTCJOYIKG2ML2XOAD6SYAKAIRTKSKQM66GVTFASQ, seg_pos={0 0}, upload=116599]: Done uploading piece. err: ecclient: upload failed (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8, address:77.90.31.16:4754): protocol: write tcp 159.223.143.39:58400->77.90.31.16:4754: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:11:26.776001817 (104h11m24.37277285s) [noise=true , address=77.90.31.14:4690 , piece_id=XUIROGKLTQGEBWNIYKM5LA6U4DZG3ZYFMCHMNYCZJ2R4Y5WB7Z5A, seg_pos={0 0}, upload=116599]: Done uploading piece. err: ecclient: upload failed (node:1WPKeCcN1uEhGjZHAJgCoJMaHxAWouGCdaU1vZwAS7kX22rxcv, address:77.90.31.14:4690): protocol: rpc: dial tcp 77.90.31.14:4690: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:11:26.776244713 (104h11m24.373015806s) [noise=true , address=77.90.31.18:4814 , piece_id=OUCWOBYCYXTPTVHHOOILTZTLUB6PJ42HOV3YURYC3LTKMTDDGPSA, seg_pos={0 0}, upload=116599]: Done uploading piece. err: ecclient: upload failed (node:1FCikLXgPYS2qM9zLTthSNHKsQNckzg9oNkKGxfvwJZZjhKkVf, address:77.90.31.18:4814): protocol: write tcp 159.223.143.39:36246->77.90.31.18:4814: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:11:26.776280556 (104h11m24.373051587s) [noise=true , address=77.90.31.11:4585 , piece_id=VX7QB2HCS64PW5BIRIJ3RUY7V5FMDVTO423ENNPOTSQUINRR6R5A, seg_pos={0 0}, upload=116599]: Done uploading piece. err: ecclient: upload failed (node:1Tve8WUWhJCK2cVWLbxxnz3WVzR2bi2wLx5XrF6QbhSN4HcsQG, address:77.90.31.11:4585): protocol: context deadline exceeded; context deadline exceeded
Mar 10 19:11:26.776313694 (104h11m24.373084705s) [seg_pos={0 0}, upload=116599]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil>
Mar 10 19:11:26.804828285 (104h11m24.401599298s) [upload=116599]: Done uploading segments...
Mar 10 19:11:26.805017989 (104h11m24.401789013s) [upload=116599]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12)
Duration: 1.101637578s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
6 out of 60 is failed (threshold=3) [...X......XX.X...........X.......X..........................] duration:
2.137898585s trace:
https://jaeger.dev.storj.io/trace/78c3ad63aac6b527
FAIL (2026-03-10 19:52:43 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
:03.719970769 (104h35m1.316741801s) [noise=true , address=185.167.204.205:4033 , piece_id=6JIW66HORXSEPSDWIUONOUZNA2UNXKBA4A6VZ7PBWW3IN2SCJSPQ, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.720006314 (104h35m1.316777347s) [noise=true , address=185.167.204.205:4024 , piece_id=3CKPLBO33LOMNYBF47K2KGQFPO53ZXLMKRKWPVSR4FKQKXKMZMCA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.721245847 (104h35m1.318016883s) [noise=true , address=185.167.204.206:4046 , piece_id=5WVC4R6VY25QUMGSOAORUK3WPGIADBXLKN7ZSLJ4GQEVKMSHZPKA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.722131967 (104h35m1.318902999s) [noise=true , address=89.149.201.147:4004 , piece_id=TMKNRIR6RWRW3TUSA2MQL4LCNTJ3LKEFIT6BAVRX2V54QHNF26FA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.722154312 (104h35m1.318925349s) [noise=true , address=185.167.204.205:4028 , piece_id=IQ4CMHEIUNHIUBQV2PVFQ6BURDMWU2S55VGFP5Y7HPFSDVBY6PYA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.723422869 (104h35m1.320193903s) [noise=true , address=81.171.14.70:4047 , piece_id=DLCUNYJFEKIU3MWFXK24QGJ4GW2H5XE4Y2ARNL2W55EZUMSC4Z7Q, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.724205203 (104h35m1.320976237s) [noise=true , address=185.167.204.205:4022 , piece_id=F5UFUTAFPRBWMSU3QKV7ZSYQR6UUC7SBUDOC7AAYGDXTC5VYUBHA, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:03.727803141 (104h35m1.324574176s) [noise=true , address=185.167.204.206:4067 , piece_id=RH2DWZ3FL7JX6EJ2WTJBPBSWDYVSHLYSLZCMMGRQXUDR6QXVZZ5A, seg_pos={0 0}, upload=117052]: Uploading piece...
Mar 10 19:35:04.167055423 (104h35m1.763826466s) [upload=117052]: Next returned nil segment
Mar 10 19:35:04.167072399 (104h35m1.763843424s) [upload=117052]: Waiting for error group managing segments...
Mar 10 19:35:04.350286979 (104h35m1.947058004s) [noise=true , address=81.171.14.70:4035 , piece_id=RNJYIOH4ETYQBSZFYTV3FX73J3PNFR2KBYNZEDXCVBOYMUAN5FCA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.351883208 (104h35m1.948654233s) [noise=true , address=81.171.14.70:4043 , piece_id=LSOHRZMEV5TEQUKZIJPYHVP43CIZKMXBTRVKIZIQNSSFNMAA5FLQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.356149085 (104h35m1.952920098s) [noise=true , address=185.167.204.205:4036 , piece_id=BBGCJ7U7PQDNB5NZTJWHUWXR6TFYF6D7LOR2UUBNV647YQX5JD2Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.369094269 (104h35m1.965865391s) [noise=true , address=89.149.201.147:4011 , piece_id=YHPLXK3ADD6VMYDWQ55PLROBCY2IOLOR2ZWEZ22FRRA4TXKSSWIQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.382982597 (104h35m1.979753611s) [noise=true , address=81.171.14.70:4034 , piece_id=BPXMDI54RTQVH5QKY6JCN5H3AKCPH62V76VBES4GQ6SLIZDIMMOA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.384672279 (104h35m1.981443307s) [noise=true , address=81.171.14.70:4047 , piece_id=DLCUNYJFEKIU3MWFXK24QGJ4GW2H5XE4Y2ARNL2W55EZUMSC4Z7Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.386337288 (104h35m1.98310832s) [noise=true , address=81.171.14.70:4025 , piece_id=R62A3AXM4OKUZFZJ2LB55UPB6VRGHBQLMXXVBWDTHUDWPGPIWEVQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.387583761 (104h35m1.984354793s) [noise=true , address=89.149.201.147:4009 , piece_id=XFSD3W6JWUXGXTHYE57SS5E5322JTRG7RKBI7MBF25GU7SEAKZJQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.388110220 (104h35m1.984881248s) [noise=true , address=81.171.14.70:4026 , piece_id=4CBQ4RZ44USXNYAGA3CIGNC3LBQXR675JVJX6CVTV3X7KVYPRHQA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.389451350 (104h35m1.986222362s) [noise=true , address=185.167.204.205:4012 , piece_id=7UFJIUKCHPKYZF42HTP6Q5D2FMHRSCRUQ7MH2MYJSEOZHZAJRZXA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.404093624 (104h35m2.00086464s) [noise=true , address=81.171.14.70:4048 , piece_id=4F2F4X4ZJ4FXUYQJQKB7YB3JZY53QNVRUSM3N4HQKGEYHMFYQB2A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.413111656 (104h35m2.009882687s) [noise=true , address=185.167.204.206:4071 , piece_id=ZQJLN55G37453GZS54ZDTPM7CEU5GH6LJR3BIKSJDC3C4QOKW5YQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.416183369 (104h35m2.012954404s) [noise=true , address=89.149.201.147:4014 , piece_id=WZOPRPDLFVOO6O2JYZZB75CLIAA4SVM7OEZXBX6WOJV3PUL2YSAQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.449375785 (104h35m2.046146815s) [noise=true , address=185.167.204.206:4039 , piece_id=DKKIZZMA3RAKMLUJTU6OANGBA3F6XWRD5ELVNXAGBIOZMFBVQWZQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.457716093 (104h35m2.054487108s) [noise=true , address=185.167.204.205:4024 , piece_id=3CKPLBO33LOMNYBF47K2KGQFPO53ZXLMKRKWPVSR4FKQKXKMZMCA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.491116161 (104h35m2.087887192s) [noise=true , address=185.167.204.205:4005 , piece_id=E7RO77MCAMFEAIE7ETPALCBV5S4X3DG5ZUD27ZE55RXYICIIXNEQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.507099744 (104h35m2.103870782s) [noise=true , address=185.167.204.205:4029 , piece_id=R33ONZELVLYL76YTJKOZASDJKFBACNNMC3PDUIITQRDGBWWZUNKA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.537139239 (104h35m2.133910269s) [noise=true , address=185.167.204.206:4067 , piece_id=RH2DWZ3FL7JX6EJ2WTJBPBSWDYVSHLYSLZCMMGRQXUDR6QXVZZ5A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.787100189 (104h35m2.383871224s) [noise=true , address=81.171.14.70:4033 , piece_id=ZFV477QXRX42OJIG4D5JLOYU3WERTD65PMHI5MFQLRTQJG3ENUTA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:04.962846444 (104h35m2.559617469s) [noise=true , address=89.149.201.147:4004 , piece_id=TMKNRIR6RWRW3TUSA2MQL4LCNTJ3LKEFIT6BAVRX2V54QHNF26FA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:08.299132133 (104h35m5.895903147s) [noise=true , address=185.167.204.205:4028 , piece_id=IQ4CMHEIUNHIUBQV2PVFQ6BURDMWU2S55VGFP5Y7HPFSDVBY6PYA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:08.481127955 (104h35m6.077898968s) [noise=true , address=185.167.204.206:4051 , piece_id=GO4OTIL5WFOS7V77FAT2NRFG2HRE2HXKMU2C3XEGDBGPQL5U5HUA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:35:10.314952727 (104h35m7.91172374s) [noise=true , address=185.167.204.206:4046 , piece_id=5WVC4R6VY25QUMGSOAORUK3WPGIADBXLKN7ZSLJ4GQEVKMSHZPKA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: <nil>
Mar 10 19:36:03.322697363 (104h36m0.919468398s) [noise=true , address=185.167.204.206:4062 , piece_id=7NP2ZIMW6ZDXW3YHZSBO5PRTAZPLYLDWXDA4QPN5XH2C6KAGYLEA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:12BL6XLqsoUGm2FxrTgYTxLZakt2u2CAhZJG82oJ31eaE4vTtMr, address:185.167.204.206:4062): protocol: write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection; write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:41266->185.167.204.206:4062: use of closed network connection
Mar 10 19:36:03.322771647 (104h36m0.919542676s) [noise=true , address=89.149.201.147:4007 , piece_id=AAJY5QBN55ENHTODH25WBVCFDT2SAQ2F27TKHFCIURCZSBMGOY6A, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1Q7d4vj1KoWhDr6xd9gEfFrZ1sV3Yro2AFke2f3Ke1rXsGnvew, address:89.149.201.147:4007): protocol: read tcp 159.223.143.39:54460->89.149.201.147:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.322958851 (104h36m0.919729884s) [noise=true , address=185.167.204.205:4033 , piece_id=6JIW66HORXSEPSDWIUONOUZNA2UNXKBA4A6VZ7PBWW3IN2SCJSPQ, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:12Q73mXHLEfuX4VUbJ8PTjhMoXCrYSKjJbFxqjDdqF3U99N8FXe, address:185.167.204.205:4033): protocol: write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection; write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:47916->185.167.204.205:4033: use of closed network connection
Mar 10 19:36:03.323134221 (104h36m0.919905244s) [noise=true , address=89.149.201.147:4018 , piece_id=V3KLJ75HJ3NRMBP2FIM33GC45QW4U5XMN2R4IAHSX32COGBJF47Q, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1Ku7o12bSmJ1YR3dGetGFcWqs1wJEh53Zkg3H7ELJrt4w1hkua, address:89.149.201.147:4018): protocol: read tcp 159.223.143.39:47122->89.149.201.147:4018: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.323461923 (104h36m0.920232956s) [noise=true , address=185.167.204.205:4022 , piece_id=F5UFUTAFPRBWMSU3QKV7ZSYQR6UUC7SBUDOC7AAYGDXTC5VYUBHA, seg_pos={0 0}, upload=117052]: Done uploading piece. err: ecclient: upload failed (node:1fT9qH1igPUNfrzpwa2VkBX7heh475sNqoN4Nsf9FYSwi1VmHj, address:185.167.204.205:4022): protocol: read tcp 159.223.143.39:51226->185.167.204.205:4022: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:36:03.323503666 (104h36m0.920274689s) [seg_pos={0 0}, upload=117052]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Mar 10 19:36:03.424787357 (104h36m1.021558384s) [upload=117052]: Done uploading segments...
Mar 10 19:36:03.424860036 (104h36m1.021631063s) [upload=117052]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 2.137941639s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
1.218116496s trace:
https://jaeger.dev.storj.io/trace/69062766ac2178dd
FAIL (2026-03-10 19:52:11 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 12)
======LOGS======
Mar 10 19:10:26.813115721 (104h10m24.409886745s) [upload=116599]: Starting upload
Mar 10 19:10:26.813211715 (104h10m24.409982738s) [upload=116599]: Done starting upload
Mar 10 19:10:26.813505664 (104h10m24.410276708s) [upload=116599]: Uploading segments...
Mar 10 19:10:26.813650528 (104h10m24.410421539s) [upload=116599]: Got next segment. Inline: false
Mar 10 19:10:26.813743195 (104h10m24.410514218s) [upload=116599]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Mar 10 19:10:26.813750388 (104h10m24.410521411s) [upload=116599]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Mar 10 19:10:26.846063672 (104h10m24.442834685s) [seg_pos={0 0}, upload=116599]: Begin upload segment...
Mar 10 19:10:26.846148290 (104h10m24.442919315s) [seg_pos={0 0}, upload=116599]: Done begin upload segment.
Mar 10 19:10:26.846228010 (104h10m24.442999069s) [noise=true , address=77.90.31.16:4754 , piece_id=PE4KKNG6B7T5FTCJOYIKG2ML2XOAD6SYAKAIRTKSKQM66GVTFASQ, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.846301186 (104h10m24.44307221s) [noise=true , address=79.127.205.233:4154 , piece_id=LDMVPR6RMS2F3YNIMVH2ELREQMP6WMEXRW5UDBHTZIPZEPWRD4BQ, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.846940319 (104h10m24.443711359s) [noise=true , address=193.149.164.70:4116 , piece_id=FVAICHAZFMXEXK5CQ3SXYQ46T54PQBGYVPPBSKZ7WOHSGOKPHWRQ, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.849741507 (104h10m24.446512533s) [noise=true , address=167.17.72.68:4654 , piece_id=WHKUMMAPP7ZSKFMAGKFHINHWIICOCMREK75F3EON723TKLAWDDAQ, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.850583955 (104h10m24.447354987s) [noise=true , address=167.17.72.70:4758 , piece_id=3OZLHHR5C6OX4EJK23YRLEGJOJ3IXWPR35CZ6MQARXGX5NNGXUGQ, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.851500001 (104h10m24.448271012s) [noise=true , address=77.90.31.17:4793 , piece_id=GB6MI5XNLXQL4KFADPIKGKDKIQ7LNZ65EOTKY7LLSQMYY7WS56FQ, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.851893521 (104h10m24.448664555s) [noise=true , address=5.231.73.4:4012 , piece_id=5RZOPQ6PFL42RJPI4ZR4GS756AU332JPC34SVWK4M2FELGI3O7CA, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.853333272 (104h10m24.450104308s) [noise=true , address=77.90.31.14:4690 , piece_id=XUIROGKLTQGEBWNIYKM5LA6U4DZG3ZYFMCHMNYCZJ2R4Y5WB7Z5A, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.853969417 (104h10m24.450740435s) [noise=true , address=77.90.31.12:4604 , piece_id=JUIZARJIP27O4PO3IZ54Y734TUIG2I3DUCQNFWI5TH5ZMCBDGJMA, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.854283058 (104h10m24.451054099s) [noise=true , address=145.79.227.2:4133 , piece_id=AXG6E4SMSMKGVOINK4HMHTXL6GRKQWU6NJOP633KYOKQ3HQHMFOA, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.854903526 (104h10m24.45167454s) [noise=true , address=77.90.31.11:4585 , piece_id=VX7QB2HCS64PW5BIRIJ3RUY7V5FMDVTO423ENNPOTSQUINRR6R5A, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.855071429 (104h10m24.451842462s) [noise=true , address=64.49.13.139:4474 , piece_id=HFVZT3V5KD42X64HCTKPEG2LWZJZAVOH5MA5CFCIMPB7WK7MWQNQ, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.855704995 (104h10m24.452476028s) [noise=true , address=193.149.164.69:4092 , piece_id=T63NKUCMVMLADZYIMCW3FMJLYURBF7YVSMQ7AFLCYOU2K3C5RYNQ, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.855767508 (104h10m24.45253852s) [noise=true , address=77.90.31.18:4814 , piece_id=OUCWOBYCYXTPTVHHOOILTZTLUB6PJ42HOV3YURYC3LTKMTDDGPSA, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.856577407 (104h10m24.453348417s) [noise=true , address=193.149.164.71:4306 , piece_id=SJ5TPY5AZVF55AK3MRAT6K5ZPA4V6N2JXBYILVM3NHTCTRZI6MLQ, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.857565751 (104h10m24.454336779s) [noise=true , address=64.49.13.140:4507 , piece_id=6PNJPCU3KVM4ZFBTXQVZG26IEG57G3HHDRLBXEAB3VTU6IEPCAWQ, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.858400866 (104h10m24.455171879s) [noise=true , address=64.49.13.170:4725 , piece_id=DEFHEGRC3SUXYVLYV2YR3KPBS4GLCPQBMV2GWR7KRY4ZBWDGAJJA, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.859258131 (104h10m24.456029145s) [noise=true , address=79.127.205.237:4245 , piece_id=ZO6RJQU6Z5XIV6NXYBV7PWACGAEP7PWJMA7VISBKQS7IRWOXNRZQ, seg_pos={0 0}, upload=116599]: Uploading piece...
Mar 10 19:10:26.934798292 (104h10m24.531569311s) [upload=116599]: Next returned nil segment
Mar 10 19:10:26.934817790 (104h10m24.531588824s) [upload=116599]: Waiting for error group managing segments...
Mar 10 19:10:26.999719507 (104h10m24.59649054s) [noise=true , address=145.79.227.2:4133 , piece_id=AXG6E4SMSMKGVOINK4HMHTXL6GRKQWU6NJOP633KYOKQ3HQHMFOA, seg_pos={0 0}, upload=116599]: Done uploading piece. err: <nil>
Mar 10 19:10:27.030173609 (104h10m24.626944622s) [noise=true , address=5.231.73.4:4012 , piece_id=5RZOPQ6PFL42RJPI4ZR4GS756AU332JPC34SVWK4M2FELGI3O7CA, seg_pos={0 0}, upload=116599]: Done uploading piece. err: <nil>
Mar 10 19:10:27.238694294 (104h10m24.83546533s) [noise=true , address=167.17.72.70:4758 , piece_id=3OZLHHR5C6OX4EJK23YRLEGJOJ3IXWPR35CZ6MQARXGX5NNGXUGQ, seg_pos={0 0}, upload=116599]: Done uploading piece. err: <nil>
Mar 10 19:10:27.294399709 (104h10m24.891170737s) [noise=true , address=167.17.72.68:4654 , piece_id=WHKUMMAPP7ZSKFMAGKFHINHWIICOCMREK75F3EON723TKLAWDDAQ, seg_pos={0 0}, upload=116599]: Done uploading piece. err: <nil>
Mar 10 19:10:27.393989371 (104h10m24.990760384s) [noise=true , address=79.127.205.233:4154 , piece_id=LDMVPR6RMS2F3YNIMVH2ELREQMP6WMEXRW5UDBHTZIPZEPWRD4BQ, seg_pos={0 0}, upload=116599]: Done uploading piece. err: <nil>
Mar 10 19:10:27.451000332 (104h10m25.047771355s) [noise=true , address=193.149.164.69:4092 , piece_id=T63NKUCMVMLADZYIMCW3FMJLYURBF7YVSMQ7AFLCYOU2K3C5RYNQ, seg_pos={0 0}, upload=116599]: Done uploading piece. err: <nil>
Mar 10 19:10:27.453000338 (104h10m25.04977136s) [noise=true , address=193.149.164.70:4116 , piece_id=FVAICHAZFMXEXK5CQ3SXYQ46T54PQBGYVPPBSKZ7WOHSGOKPHWRQ, seg_pos={0 0}, upload=116599]: Done uploading piece. err: <nil>
Mar 10 19:10:27.472395366 (104h10m25.069166393s) [noise=true , address=193.149.164.71:4306 , piece_id=SJ5TPY5AZVF55AK3MRAT6K5ZPA4V6N2JXBYILVM3NHTCTRZI6MLQ, seg_pos={0 0}, upload=116599]: Done uploading piece. err: <nil>
Mar 10 19:10:27.486423055 (104h10m25.083194088s) [noise=true , address=64.49.13.140:4507 , piece_id=6PNJPCU3KVM4ZFBTXQVZG26IEG57G3HHDRLBXEAB3VTU6IEPCAWQ, seg_pos={0 0}, upload=116599]: Done uploading piece. err: <nil>
Mar 10 19:10:27.487859041 (104h10m25.084630054s) [noise=true , address=64.49.13.139:4474 , piece_id=HFVZT3V5KD42X64HCTKPEG2LWZJZAVOH5MA5CFCIMPB7WK7MWQNQ, seg_pos={0 0}, upload=116599]: Done uploading piece. err: <nil>
Mar 10 19:10:27.489472801 (104h10m25.086243832s) [noise=true , address=64.49.13.170:4725 , piece_id=DEFHEGRC3SUXYVLYV2YR3KPBS4GLCPQBMV2GWR7KRY4ZBWDGAJJA, seg_pos={0 0}, upload=116599]: Done uploading piece. err: <nil>
Mar 10 19:10:28.942749218 (104h10m26.539520243s) [noise=true , address=79.127.205.237:4245 , piece_id=ZO6RJQU6Z5XIV6NXYBV7PWACGAEP7PWJMA7VISBKQS7IRWOXNRZQ, seg_pos={0 0}, upload=116599]: Done uploading piece. err: <nil>
Mar 10 19:11:26.775681128 (104h11m24.37245216s) [noise=true , address=77.90.31.17:4793 , piece_id=GB6MI5XNLXQL4KFADPIKGKDKIQ7LNZ65EOTKY7LLSQMYY7WS56FQ, seg_pos={0 0}, upload=116599]: Done uploading piece. err: ecclient: upload failed (node:1rxrH2F4LTfrK9SMsNNEuqVr4AjV1VbEYDc3veWbQf88rj2Es4, address:77.90.31.17:4793): protocol: rpc: dial tcp 77.90.31.17:4793: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:11:26.775748621 (104h11m24.37251965s) [noise=true , address=77.90.31.12:4604 , piece_id=JUIZARJIP27O4PO3IZ54Y734TUIG2I3DUCQNFWI5TH5ZMCBDGJMA, seg_pos={0 0}, upload=116599]: Done uploading piece. err: ecclient: upload failed (node:1vK9QCCj1rszVCR72YBc8BdKCPRno49RrcH9uGQb9rBJkZY4F7, address:77.90.31.12:4604): protocol: rpc: dial tcp 77.90.31.12:4604: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:11:26.775972253 (104h11m24.372743279s) [noise=true , address=77.90.31.16:4754 , piece_id=PE4KKNG6B7T5FTCJOYIKG2ML2XOAD6SYAKAIRTKSKQM66GVTFASQ, seg_pos={0 0}, upload=116599]: Done uploading piece. err: ecclient: upload failed (node:12KKCnEfLWFbLY7ViZ8uzgKfNDeVby4XaoRiBKiQCH39mq82RJ8, address:77.90.31.16:4754): protocol: write tcp 159.223.143.39:58400->77.90.31.16:4754: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:11:26.776001817 (104h11m24.37277285s) [noise=true , address=77.90.31.14:4690 , piece_id=XUIROGKLTQGEBWNIYKM5LA6U4DZG3ZYFMCHMNYCZJ2R4Y5WB7Z5A, seg_pos={0 0}, upload=116599]: Done uploading piece. err: ecclient: upload failed (node:1WPKeCcN1uEhGjZHAJgCoJMaHxAWouGCdaU1vZwAS7kX22rxcv, address:77.90.31.14:4690): protocol: rpc: dial tcp 77.90.31.14:4690: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:11:26.776244713 (104h11m24.373015806s) [noise=true , address=77.90.31.18:4814 , piece_id=OUCWOBYCYXTPTVHHOOILTZTLUB6PJ42HOV3YURYC3LTKMTDDGPSA, seg_pos={0 0}, upload=116599]: Done uploading piece. err: ecclient: upload failed (node:1FCikLXgPYS2qM9zLTthSNHKsQNckzg9oNkKGxfvwJZZjhKkVf, address:77.90.31.18:4814): protocol: write tcp 159.223.143.39:36246->77.90.31.18:4814: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Mar 10 19:11:26.776280556 (104h11m24.373051587s) [noise=true , address=77.90.31.11:4585 , piece_id=VX7QB2HCS64PW5BIRIJ3RUY7V5FMDVTO423ENNPOTSQUINRR6R5A, seg_pos={0 0}, upload=116599]: Done uploading piece. err: ecclient: upload failed (node:1Tve8WUWhJCK2cVWLbxxnz3WVzR2bi2wLx5XrF6QbhSN4HcsQG, address:77.90.31.11:4585): protocol: context deadline exceeded; context deadline exceeded
Mar 10 19:11:26.776313694 (104h11m24.373084705s) [seg_pos={0 0}, upload=116599]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil>
Mar 10 19:11:26.804828285 (104h11m24.401599298s) [upload=116599]: Done uploading segments...
Mar 10 19:11:26.805017989 (104h11m24.401789013s) [upload=116599]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12)
Duration: 1.218152629s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|