Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 472.453321ms trace: https://jaeger.dev.storj.io/trace/62a1509ecc5b5058 FAIL (2026-01-31 09:05:41 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Jan 31 08:42:01.166771110 (376h46m54.650228297s) [upload=391707]: Starting upload Jan 31 08:42:01.166986060 (376h46m54.650443258s) [upload=391707]: Done starting upload Jan 31 08:42:01.167072896 (376h46m54.650530094s) [upload=391707]: Uploading segments... Jan 31 08:42:01.167397789 (376h46m54.650854991s) [upload=391707]: Got next segment. Inline: false Jan 31 08:42:01.167473452 (376h46m54.650930657s) [upload=391707]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 31 08:42:01.167480315 (376h46m54.650937524s) [upload=391707]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 31 08:42:01.195346764 (376h46m54.678803969s) [seg_pos={0 0}, upload=391707]: Begin upload segment... Jan 31 08:42:01.195428575 (376h46m54.678885757s) [seg_pos={0 0}, upload=391707]: Done begin upload segment. Jan 31 08:42:01.195463009 (376h46m54.678920214s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.195847779 (376h46m54.679304986s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.197051385 (376h46m54.680508591s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.197594770 (376h46m54.681051978s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198049760 (376h46m54.681506963s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198528799 (376h46m54.681986007s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198996489 (376h46m54.682453701s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.199526016 (376h46m54.682983298s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.199772893 (376h46m54.683230102s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200006077 (376h46m54.683463277s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200298977 (376h46m54.68375618s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200764860 (376h46m54.684222068s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.201330795 (376h46m54.68478802s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.201862657 (376h46m54.685319865s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.202435299 (376h46m54.685892502s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.203149033 (376h46m54.686606237s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.288234960 (376h46m54.771692157s) [upload=391707]: Next returned nil segment Jan 31 08:42:01.288251604 (376h46m54.771708818s) [upload=391707]: Waiting for error group managing segments... Jan 31 08:42:01.293400797 (376h46m54.776858002s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.294083673 (376h46m54.777540862s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.294285106 (376h46m54.777742305s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295108197 (376h46m54.778565394s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295172272 (376h46m54.778629467s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295900158 (376h46m54.779357349s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295955977 (376h46m54.779413173s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296392614 (376h46m54.779849814s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296441340 (376h46m54.779898537s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296874674 (376h46m54.780331861s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296909101 (376h46m54.780366298s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.297409404 (376h46m54.780866604s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:31.121919846 (376h47m24.605377048s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: read tcp 159.223.143.39:60086->45.91.116.7:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122001831 (376h47m24.605459038s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: read tcp 159.223.143.39:60218->45.91.116.5:4297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122048345 (376h47m24.605505567s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: read tcp 159.223.143.39:45254->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122106214 (376h47m24.605563415s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12qpELr7Yr1aJtgau8q34qPBtADW6QAXPNySAzcRLB39hXoXueo, address:45.91.116.8:4070): protocol: read tcp 159.223.143.39:50716->45.91.116.8:4070: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122131285 (376h47m24.605588469s) [seg_pos={0 0}, upload=391707]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jan 31 08:42:31.141063957 (376h47m24.624521154s) [upload=391707]: Done uploading segments... Jan 31 08:42:31.141101292 (376h47m24.624558493s) [upload=391707]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 472.485023ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...........................X................................] duration: 465.46834ms trace: https://jaeger.dev.storj.io/trace/6c4cbd1edfce8e6 FAIL (2026-01-31 09:05:25 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 465.533902ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [............................X...............................] duration: 476.718607ms trace: https://jaeger.dev.storj.io/trace/6c4cbd1edfce8e6 FAIL (2026-01-31 09:05:02 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 476.807972ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 494.470684ms trace: https://jaeger.dev.storj.io/trace/33d822523a7c81ef FAIL (2026-01-31 09:05:01 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 495.34466ms edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.............................X..............................] duration: 474.866167ms trace: https://jaeger.dev.storj.io/trace/6c4cbd1edfce8e6 FAIL (2026-01-31 09:04:44 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 474.898502ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 585.452721ms trace: https://jaeger.dev.storj.io/trace/62a1509ecc5b5058 FAIL (2026-01-31 09:04:32 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Jan 31 08:42:01.166771110 (376h46m54.650228297s) [upload=391707]: Starting upload Jan 31 08:42:01.166986060 (376h46m54.650443258s) [upload=391707]: Done starting upload Jan 31 08:42:01.167072896 (376h46m54.650530094s) [upload=391707]: Uploading segments... Jan 31 08:42:01.167397789 (376h46m54.650854991s) [upload=391707]: Got next segment. Inline: false Jan 31 08:42:01.167473452 (376h46m54.650930657s) [upload=391707]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 31 08:42:01.167480315 (376h46m54.650937524s) [upload=391707]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 31 08:42:01.195346764 (376h46m54.678803969s) [seg_pos={0 0}, upload=391707]: Begin upload segment... Jan 31 08:42:01.195428575 (376h46m54.678885757s) [seg_pos={0 0}, upload=391707]: Done begin upload segment. Jan 31 08:42:01.195463009 (376h46m54.678920214s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.195847779 (376h46m54.679304986s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.197051385 (376h46m54.680508591s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.197594770 (376h46m54.681051978s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198049760 (376h46m54.681506963s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198528799 (376h46m54.681986007s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198996489 (376h46m54.682453701s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.199526016 (376h46m54.682983298s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.199772893 (376h46m54.683230102s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200006077 (376h46m54.683463277s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200298977 (376h46m54.68375618s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200764860 (376h46m54.684222068s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.201330795 (376h46m54.68478802s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.201862657 (376h46m54.685319865s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.202435299 (376h46m54.685892502s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.203149033 (376h46m54.686606237s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.288234960 (376h46m54.771692157s) [upload=391707]: Next returned nil segment Jan 31 08:42:01.288251604 (376h46m54.771708818s) [upload=391707]: Waiting for error group managing segments... Jan 31 08:42:01.293400797 (376h46m54.776858002s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.294083673 (376h46m54.777540862s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.294285106 (376h46m54.777742305s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295108197 (376h46m54.778565394s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295172272 (376h46m54.778629467s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295900158 (376h46m54.779357349s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295955977 (376h46m54.779413173s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296392614 (376h46m54.779849814s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296441340 (376h46m54.779898537s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296874674 (376h46m54.780331861s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296909101 (376h46m54.780366298s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.297409404 (376h46m54.780866604s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:31.121919846 (376h47m24.605377048s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: read tcp 159.223.143.39:60086->45.91.116.7:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122001831 (376h47m24.605459038s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: read tcp 159.223.143.39:60218->45.91.116.5:4297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122048345 (376h47m24.605505567s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: read tcp 159.223.143.39:45254->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122106214 (376h47m24.605563415s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12qpELr7Yr1aJtgau8q34qPBtADW6QAXPNySAzcRLB39hXoXueo, address:45.91.116.8:4070): protocol: read tcp 159.223.143.39:50716->45.91.116.8:4070: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122131285 (376h47m24.605588469s) [seg_pos={0 0}, upload=391707]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jan 31 08:42:31.141063957 (376h47m24.624521154s) [upload=391707]: Done uploading segments... Jan 31 08:42:31.141101292 (376h47m24.624558493s) [upload=391707]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 585.571714ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 478.683235ms trace: https://jaeger.dev.storj.io/trace/33d822523a7c81ef FAIL (2026-01-31 09:04:08 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 478.714149ms edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..............................X.............................] duration: 462.95989ms trace: https://jaeger.dev.storj.io/trace/6c4cbd1edfce8e6 FAIL (2026-01-31 09:03:54 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 462.985974ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 425.004423ms trace: https://jaeger.dev.storj.io/trace/62a1509ecc5b5058 FAIL (2026-01-31 09:03:33 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Jan 31 08:42:01.166771110 (376h46m54.650228297s) [upload=391707]: Starting upload Jan 31 08:42:01.166986060 (376h46m54.650443258s) [upload=391707]: Done starting upload Jan 31 08:42:01.167072896 (376h46m54.650530094s) [upload=391707]: Uploading segments... Jan 31 08:42:01.167397789 (376h46m54.650854991s) [upload=391707]: Got next segment. Inline: false Jan 31 08:42:01.167473452 (376h46m54.650930657s) [upload=391707]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 31 08:42:01.167480315 (376h46m54.650937524s) [upload=391707]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 31 08:42:01.195346764 (376h46m54.678803969s) [seg_pos={0 0}, upload=391707]: Begin upload segment... Jan 31 08:42:01.195428575 (376h46m54.678885757s) [seg_pos={0 0}, upload=391707]: Done begin upload segment. Jan 31 08:42:01.195463009 (376h46m54.678920214s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.195847779 (376h46m54.679304986s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.197051385 (376h46m54.680508591s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.197594770 (376h46m54.681051978s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198049760 (376h46m54.681506963s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198528799 (376h46m54.681986007s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198996489 (376h46m54.682453701s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.199526016 (376h46m54.682983298s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.199772893 (376h46m54.683230102s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200006077 (376h46m54.683463277s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200298977 (376h46m54.68375618s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200764860 (376h46m54.684222068s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.201330795 (376h46m54.68478802s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.201862657 (376h46m54.685319865s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.202435299 (376h46m54.685892502s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.203149033 (376h46m54.686606237s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.288234960 (376h46m54.771692157s) [upload=391707]: Next returned nil segment Jan 31 08:42:01.288251604 (376h46m54.771708818s) [upload=391707]: Waiting for error group managing segments... Jan 31 08:42:01.293400797 (376h46m54.776858002s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.294083673 (376h46m54.777540862s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.294285106 (376h46m54.777742305s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295108197 (376h46m54.778565394s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295172272 (376h46m54.778629467s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295900158 (376h46m54.779357349s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295955977 (376h46m54.779413173s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296392614 (376h46m54.779849814s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296441340 (376h46m54.779898537s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296874674 (376h46m54.780331861s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296909101 (376h46m54.780366298s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.297409404 (376h46m54.780866604s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:31.121919846 (376h47m24.605377048s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: read tcp 159.223.143.39:60086->45.91.116.7:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122001831 (376h47m24.605459038s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: read tcp 159.223.143.39:60218->45.91.116.5:4297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122048345 (376h47m24.605505567s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: read tcp 159.223.143.39:45254->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122106214 (376h47m24.605563415s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12qpELr7Yr1aJtgau8q34qPBtADW6QAXPNySAzcRLB39hXoXueo, address:45.91.116.8:4070): protocol: read tcp 159.223.143.39:50716->45.91.116.8:4070: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122131285 (376h47m24.605588469s) [seg_pos={0 0}, upload=391707]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jan 31 08:42:31.141063957 (376h47m24.624521154s) [upload=391707]: Done uploading segments... Jan 31 08:42:31.141101292 (376h47m24.624558493s) [upload=391707]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 425.098906ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 534.191217ms trace: https://jaeger.dev.storj.io/trace/33d822523a7c81ef FAIL (2026-01-31 09:03:25 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 534.299732ms edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...............................X............................] duration: 437.764579ms trace: https://jaeger.dev.storj.io/trace/6c4cbd1edfce8e6 FAIL (2026-01-31 09:03:12 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 437.809359ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 483.166968ms trace: https://jaeger.dev.storj.io/trace/33d822523a7c81ef FAIL (2026-01-31 09:02:43 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 483.196595ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 595.242075ms trace: https://jaeger.dev.storj.io/trace/62a1509ecc5b5058 FAIL (2026-01-31 09:02:35 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Jan 31 08:42:01.166771110 (376h46m54.650228297s) [upload=391707]: Starting upload Jan 31 08:42:01.166986060 (376h46m54.650443258s) [upload=391707]: Done starting upload Jan 31 08:42:01.167072896 (376h46m54.650530094s) [upload=391707]: Uploading segments... Jan 31 08:42:01.167397789 (376h46m54.650854991s) [upload=391707]: Got next segment. Inline: false Jan 31 08:42:01.167473452 (376h46m54.650930657s) [upload=391707]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 31 08:42:01.167480315 (376h46m54.650937524s) [upload=391707]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 31 08:42:01.195346764 (376h46m54.678803969s) [seg_pos={0 0}, upload=391707]: Begin upload segment... Jan 31 08:42:01.195428575 (376h46m54.678885757s) [seg_pos={0 0}, upload=391707]: Done begin upload segment. Jan 31 08:42:01.195463009 (376h46m54.678920214s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.195847779 (376h46m54.679304986s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.197051385 (376h46m54.680508591s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.197594770 (376h46m54.681051978s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198049760 (376h46m54.681506963s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198528799 (376h46m54.681986007s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198996489 (376h46m54.682453701s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.199526016 (376h46m54.682983298s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.199772893 (376h46m54.683230102s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200006077 (376h46m54.683463277s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200298977 (376h46m54.68375618s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200764860 (376h46m54.684222068s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.201330795 (376h46m54.68478802s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.201862657 (376h46m54.685319865s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.202435299 (376h46m54.685892502s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.203149033 (376h46m54.686606237s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.288234960 (376h46m54.771692157s) [upload=391707]: Next returned nil segment Jan 31 08:42:01.288251604 (376h46m54.771708818s) [upload=391707]: Waiting for error group managing segments... Jan 31 08:42:01.293400797 (376h46m54.776858002s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.294083673 (376h46m54.777540862s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.294285106 (376h46m54.777742305s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295108197 (376h46m54.778565394s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295172272 (376h46m54.778629467s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295900158 (376h46m54.779357349s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295955977 (376h46m54.779413173s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296392614 (376h46m54.779849814s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296441340 (376h46m54.779898537s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296874674 (376h46m54.780331861s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296909101 (376h46m54.780366298s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.297409404 (376h46m54.780866604s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:31.121919846 (376h47m24.605377048s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: read tcp 159.223.143.39:60086->45.91.116.7:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122001831 (376h47m24.605459038s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: read tcp 159.223.143.39:60218->45.91.116.5:4297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122048345 (376h47m24.605505567s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: read tcp 159.223.143.39:45254->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122106214 (376h47m24.605563415s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12qpELr7Yr1aJtgau8q34qPBtADW6QAXPNySAzcRLB39hXoXueo, address:45.91.116.8:4070): protocol: read tcp 159.223.143.39:50716->45.91.116.8:4070: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122131285 (376h47m24.605588469s) [seg_pos={0 0}, upload=391707]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jan 31 08:42:31.141063957 (376h47m24.624521154s) [upload=391707]: Done uploading segments... Jan 31 08:42:31.141101292 (376h47m24.624558493s) [upload=391707]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 595.268943ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [................................X...........................] duration: 439.042198ms trace: https://jaeger.dev.storj.io/trace/6c4cbd1edfce8e6 FAIL (2026-01-31 09:02:28 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 439.159653ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 542.449604ms trace: https://jaeger.dev.storj.io/trace/62a1509ecc5b5058 FAIL (2026-01-31 09:01:51 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Jan 31 08:42:01.166771110 (376h46m54.650228297s) [upload=391707]: Starting upload Jan 31 08:42:01.166986060 (376h46m54.650443258s) [upload=391707]: Done starting upload Jan 31 08:42:01.167072896 (376h46m54.650530094s) [upload=391707]: Uploading segments... Jan 31 08:42:01.167397789 (376h46m54.650854991s) [upload=391707]: Got next segment. Inline: false Jan 31 08:42:01.167473452 (376h46m54.650930657s) [upload=391707]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 31 08:42:01.167480315 (376h46m54.650937524s) [upload=391707]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 31 08:42:01.195346764 (376h46m54.678803969s) [seg_pos={0 0}, upload=391707]: Begin upload segment... Jan 31 08:42:01.195428575 (376h46m54.678885757s) [seg_pos={0 0}, upload=391707]: Done begin upload segment. Jan 31 08:42:01.195463009 (376h46m54.678920214s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.195847779 (376h46m54.679304986s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.197051385 (376h46m54.680508591s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.197594770 (376h46m54.681051978s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198049760 (376h46m54.681506963s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198528799 (376h46m54.681986007s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198996489 (376h46m54.682453701s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.199526016 (376h46m54.682983298s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.199772893 (376h46m54.683230102s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200006077 (376h46m54.683463277s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200298977 (376h46m54.68375618s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200764860 (376h46m54.684222068s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.201330795 (376h46m54.68478802s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.201862657 (376h46m54.685319865s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.202435299 (376h46m54.685892502s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.203149033 (376h46m54.686606237s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.288234960 (376h46m54.771692157s) [upload=391707]: Next returned nil segment Jan 31 08:42:01.288251604 (376h46m54.771708818s) [upload=391707]: Waiting for error group managing segments... Jan 31 08:42:01.293400797 (376h46m54.776858002s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.294083673 (376h46m54.777540862s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.294285106 (376h46m54.777742305s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295108197 (376h46m54.778565394s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295172272 (376h46m54.778629467s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295900158 (376h46m54.779357349s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295955977 (376h46m54.779413173s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296392614 (376h46m54.779849814s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296441340 (376h46m54.779898537s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296874674 (376h46m54.780331861s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296909101 (376h46m54.780366298s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.297409404 (376h46m54.780866604s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:31.121919846 (376h47m24.605377048s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: read tcp 159.223.143.39:60086->45.91.116.7:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122001831 (376h47m24.605459038s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: read tcp 159.223.143.39:60218->45.91.116.5:4297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122048345 (376h47m24.605505567s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: read tcp 159.223.143.39:45254->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122106214 (376h47m24.605563415s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12qpELr7Yr1aJtgau8q34qPBtADW6QAXPNySAzcRLB39hXoXueo, address:45.91.116.8:4070): protocol: read tcp 159.223.143.39:50716->45.91.116.8:4070: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122131285 (376h47m24.605588469s) [seg_pos={0 0}, upload=391707]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jan 31 08:42:31.141063957 (376h47m24.624521154s) [upload=391707]: Done uploading segments... Jan 31 08:42:31.141101292 (376h47m24.624558493s) [upload=391707]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 542.538039ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.................................X..........................] duration: 432.993754ms trace: https://jaeger.dev.storj.io/trace/6c4cbd1edfce8e6 FAIL (2026-01-31 09:01:41 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 433.022285ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 549.353737ms trace: https://jaeger.dev.storj.io/trace/33d822523a7c81ef FAIL (2026-01-31 09:01:32 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 549.382262ms edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..................................X.........................] duration: 398.470955ms trace: https://jaeger.dev.storj.io/trace/6c4cbd1edfce8e6 FAIL (2026-01-31 09:01:16 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 398.498008ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 518.57445ms trace: https://jaeger.dev.storj.io/trace/33d822523a7c81ef FAIL (2026-01-31 09:01:09 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 518.625717ms edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...................................X........................] duration: 442.410675ms trace: https://jaeger.dev.storj.io/trace/6c4cbd1edfce8e6 FAIL (2026-01-31 09:00:49 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 442.434549ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 462.391985ms trace: https://jaeger.dev.storj.io/trace/62a1509ecc5b5058 FAIL (2026-01-31 09:00:46 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Jan 31 08:42:01.166771110 (376h46m54.650228297s) [upload=391707]: Starting upload Jan 31 08:42:01.166986060 (376h46m54.650443258s) [upload=391707]: Done starting upload Jan 31 08:42:01.167072896 (376h46m54.650530094s) [upload=391707]: Uploading segments... Jan 31 08:42:01.167397789 (376h46m54.650854991s) [upload=391707]: Got next segment. Inline: false Jan 31 08:42:01.167473452 (376h46m54.650930657s) [upload=391707]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 31 08:42:01.167480315 (376h46m54.650937524s) [upload=391707]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 31 08:42:01.195346764 (376h46m54.678803969s) [seg_pos={0 0}, upload=391707]: Begin upload segment... Jan 31 08:42:01.195428575 (376h46m54.678885757s) [seg_pos={0 0}, upload=391707]: Done begin upload segment. Jan 31 08:42:01.195463009 (376h46m54.678920214s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.195847779 (376h46m54.679304986s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.197051385 (376h46m54.680508591s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.197594770 (376h46m54.681051978s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198049760 (376h46m54.681506963s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198528799 (376h46m54.681986007s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198996489 (376h46m54.682453701s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.199526016 (376h46m54.682983298s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.199772893 (376h46m54.683230102s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200006077 (376h46m54.683463277s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200298977 (376h46m54.68375618s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200764860 (376h46m54.684222068s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.201330795 (376h46m54.68478802s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.201862657 (376h46m54.685319865s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.202435299 (376h46m54.685892502s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.203149033 (376h46m54.686606237s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.288234960 (376h46m54.771692157s) [upload=391707]: Next returned nil segment Jan 31 08:42:01.288251604 (376h46m54.771708818s) [upload=391707]: Waiting for error group managing segments... Jan 31 08:42:01.293400797 (376h46m54.776858002s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.294083673 (376h46m54.777540862s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.294285106 (376h46m54.777742305s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295108197 (376h46m54.778565394s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295172272 (376h46m54.778629467s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295900158 (376h46m54.779357349s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295955977 (376h46m54.779413173s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296392614 (376h46m54.779849814s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296441340 (376h46m54.779898537s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296874674 (376h46m54.780331861s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296909101 (376h46m54.780366298s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.297409404 (376h46m54.780866604s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:31.121919846 (376h47m24.605377048s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: read tcp 159.223.143.39:60086->45.91.116.7:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122001831 (376h47m24.605459038s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: read tcp 159.223.143.39:60218->45.91.116.5:4297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122048345 (376h47m24.605505567s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: read tcp 159.223.143.39:45254->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122106214 (376h47m24.605563415s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12qpELr7Yr1aJtgau8q34qPBtADW6QAXPNySAzcRLB39hXoXueo, address:45.91.116.8:4070): protocol: read tcp 159.223.143.39:50716->45.91.116.8:4070: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122131285 (376h47m24.605588469s) [seg_pos={0 0}, upload=391707]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jan 31 08:42:31.141063957 (376h47m24.624521154s) [upload=391707]: Done uploading segments... Jan 31 08:42:31.141101292 (376h47m24.624558493s) [upload=391707]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 462.479806ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 469.033177ms trace: https://jaeger.dev.storj.io/trace/33d822523a7c81ef FAIL (2026-01-31 09:00:15 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 469.072148ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 834.618133ms trace: https://jaeger.dev.storj.io/trace/62a1509ecc5b5058 FAIL (2026-01-31 09:00:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Jan 31 08:42:01.166771110 (376h46m54.650228297s) [upload=391707]: Starting upload Jan 31 08:42:01.166986060 (376h46m54.650443258s) [upload=391707]: Done starting upload Jan 31 08:42:01.167072896 (376h46m54.650530094s) [upload=391707]: Uploading segments... Jan 31 08:42:01.167397789 (376h46m54.650854991s) [upload=391707]: Got next segment. Inline: false Jan 31 08:42:01.167473452 (376h46m54.650930657s) [upload=391707]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 31 08:42:01.167480315 (376h46m54.650937524s) [upload=391707]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 31 08:42:01.195346764 (376h46m54.678803969s) [seg_pos={0 0}, upload=391707]: Begin upload segment... Jan 31 08:42:01.195428575 (376h46m54.678885757s) [seg_pos={0 0}, upload=391707]: Done begin upload segment. Jan 31 08:42:01.195463009 (376h46m54.678920214s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.195847779 (376h46m54.679304986s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.197051385 (376h46m54.680508591s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.197594770 (376h46m54.681051978s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198049760 (376h46m54.681506963s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198528799 (376h46m54.681986007s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198996489 (376h46m54.682453701s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.199526016 (376h46m54.682983298s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.199772893 (376h46m54.683230102s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200006077 (376h46m54.683463277s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200298977 (376h46m54.68375618s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200764860 (376h46m54.684222068s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.201330795 (376h46m54.68478802s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.201862657 (376h46m54.685319865s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.202435299 (376h46m54.685892502s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.203149033 (376h46m54.686606237s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.288234960 (376h46m54.771692157s) [upload=391707]: Next returned nil segment Jan 31 08:42:01.288251604 (376h46m54.771708818s) [upload=391707]: Waiting for error group managing segments... Jan 31 08:42:01.293400797 (376h46m54.776858002s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.294083673 (376h46m54.777540862s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.294285106 (376h46m54.777742305s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295108197 (376h46m54.778565394s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295172272 (376h46m54.778629467s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295900158 (376h46m54.779357349s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295955977 (376h46m54.779413173s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296392614 (376h46m54.779849814s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296441340 (376h46m54.779898537s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296874674 (376h46m54.780331861s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296909101 (376h46m54.780366298s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.297409404 (376h46m54.780866604s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:31.121919846 (376h47m24.605377048s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: read tcp 159.223.143.39:60086->45.91.116.7:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122001831 (376h47m24.605459038s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: read tcp 159.223.143.39:60218->45.91.116.5:4297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122048345 (376h47m24.605505567s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: read tcp 159.223.143.39:45254->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122106214 (376h47m24.605563415s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12qpELr7Yr1aJtgau8q34qPBtADW6QAXPNySAzcRLB39hXoXueo, address:45.91.116.8:4070): protocol: read tcp 159.223.143.39:50716->45.91.116.8:4070: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122131285 (376h47m24.605588469s) [seg_pos={0 0}, upload=391707]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jan 31 08:42:31.141063957 (376h47m24.624521154s) [upload=391707]: Done uploading segments... Jan 31 08:42:31.141101292 (376h47m24.624558493s) [upload=391707]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 834.642911ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 443.377394ms trace: https://jaeger.dev.storj.io/trace/33d822523a7c81ef FAIL (2026-01-31 08:59:52 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 443.413735ms edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [....................................X.......................] duration: 447.350497ms trace: https://jaeger.dev.storj.io/trace/6c4cbd1edfce8e6 FAIL (2026-01-31 08:59:52 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 447.376978ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 429.934666ms trace: https://jaeger.dev.storj.io/trace/62a1509ecc5b5058 FAIL (2026-01-31 08:59:23 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Jan 31 08:42:01.166771110 (376h46m54.650228297s) [upload=391707]: Starting upload Jan 31 08:42:01.166986060 (376h46m54.650443258s) [upload=391707]: Done starting upload Jan 31 08:42:01.167072896 (376h46m54.650530094s) [upload=391707]: Uploading segments... Jan 31 08:42:01.167397789 (376h46m54.650854991s) [upload=391707]: Got next segment. Inline: false Jan 31 08:42:01.167473452 (376h46m54.650930657s) [upload=391707]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 31 08:42:01.167480315 (376h46m54.650937524s) [upload=391707]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 31 08:42:01.195346764 (376h46m54.678803969s) [seg_pos={0 0}, upload=391707]: Begin upload segment... Jan 31 08:42:01.195428575 (376h46m54.678885757s) [seg_pos={0 0}, upload=391707]: Done begin upload segment. Jan 31 08:42:01.195463009 (376h46m54.678920214s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.195847779 (376h46m54.679304986s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.197051385 (376h46m54.680508591s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.197594770 (376h46m54.681051978s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198049760 (376h46m54.681506963s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198528799 (376h46m54.681986007s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198996489 (376h46m54.682453701s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.199526016 (376h46m54.682983298s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.199772893 (376h46m54.683230102s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200006077 (376h46m54.683463277s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200298977 (376h46m54.68375618s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200764860 (376h46m54.684222068s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.201330795 (376h46m54.68478802s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.201862657 (376h46m54.685319865s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.202435299 (376h46m54.685892502s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.203149033 (376h46m54.686606237s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.288234960 (376h46m54.771692157s) [upload=391707]: Next returned nil segment Jan 31 08:42:01.288251604 (376h46m54.771708818s) [upload=391707]: Waiting for error group managing segments... Jan 31 08:42:01.293400797 (376h46m54.776858002s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.294083673 (376h46m54.777540862s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.294285106 (376h46m54.777742305s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295108197 (376h46m54.778565394s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295172272 (376h46m54.778629467s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295900158 (376h46m54.779357349s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295955977 (376h46m54.779413173s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296392614 (376h46m54.779849814s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296441340 (376h46m54.779898537s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296874674 (376h46m54.780331861s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296909101 (376h46m54.780366298s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.297409404 (376h46m54.780866604s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:31.121919846 (376h47m24.605377048s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: read tcp 159.223.143.39:60086->45.91.116.7:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122001831 (376h47m24.605459038s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: read tcp 159.223.143.39:60218->45.91.116.5:4297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122048345 (376h47m24.605505567s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: read tcp 159.223.143.39:45254->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122106214 (376h47m24.605563415s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12qpELr7Yr1aJtgau8q34qPBtADW6QAXPNySAzcRLB39hXoXueo, address:45.91.116.8:4070): protocol: read tcp 159.223.143.39:50716->45.91.116.8:4070: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122131285 (376h47m24.605588469s) [seg_pos={0 0}, upload=391707]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jan 31 08:42:31.141063957 (376h47m24.624521154s) [upload=391707]: Done uploading segments... Jan 31 08:42:31.141101292 (376h47m24.624558493s) [upload=391707]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 429.96212ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.....................................X......................] duration: 442.840041ms trace: https://jaeger.dev.storj.io/trace/6c4cbd1edfce8e6 FAIL (2026-01-31 08:59:23 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 442.864823ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 442.949704ms trace: https://jaeger.dev.storj.io/trace/33d822523a7c81ef FAIL (2026-01-31 08:59:09 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 442.9776ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 414.323204ms trace: https://jaeger.dev.storj.io/trace/62a1509ecc5b5058 FAIL (2026-01-31 08:59:01 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Jan 31 08:42:01.166771110 (376h46m54.650228297s) [upload=391707]: Starting upload Jan 31 08:42:01.166986060 (376h46m54.650443258s) [upload=391707]: Done starting upload Jan 31 08:42:01.167072896 (376h46m54.650530094s) [upload=391707]: Uploading segments... Jan 31 08:42:01.167397789 (376h46m54.650854991s) [upload=391707]: Got next segment. Inline: false Jan 31 08:42:01.167473452 (376h46m54.650930657s) [upload=391707]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 31 08:42:01.167480315 (376h46m54.650937524s) [upload=391707]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 31 08:42:01.195346764 (376h46m54.678803969s) [seg_pos={0 0}, upload=391707]: Begin upload segment... Jan 31 08:42:01.195428575 (376h46m54.678885757s) [seg_pos={0 0}, upload=391707]: Done begin upload segment. Jan 31 08:42:01.195463009 (376h46m54.678920214s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.195847779 (376h46m54.679304986s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.197051385 (376h46m54.680508591s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.197594770 (376h46m54.681051978s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198049760 (376h46m54.681506963s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198528799 (376h46m54.681986007s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198996489 (376h46m54.682453701s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.199526016 (376h46m54.682983298s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.199772893 (376h46m54.683230102s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200006077 (376h46m54.683463277s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200298977 (376h46m54.68375618s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200764860 (376h46m54.684222068s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.201330795 (376h46m54.68478802s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.201862657 (376h46m54.685319865s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.202435299 (376h46m54.685892502s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.203149033 (376h46m54.686606237s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.288234960 (376h46m54.771692157s) [upload=391707]: Next returned nil segment Jan 31 08:42:01.288251604 (376h46m54.771708818s) [upload=391707]: Waiting for error group managing segments... Jan 31 08:42:01.293400797 (376h46m54.776858002s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.294083673 (376h46m54.777540862s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.294285106 (376h46m54.777742305s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295108197 (376h46m54.778565394s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295172272 (376h46m54.778629467s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295900158 (376h46m54.779357349s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295955977 (376h46m54.779413173s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296392614 (376h46m54.779849814s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296441340 (376h46m54.779898537s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296874674 (376h46m54.780331861s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296909101 (376h46m54.780366298s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.297409404 (376h46m54.780866604s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:31.121919846 (376h47m24.605377048s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: read tcp 159.223.143.39:60086->45.91.116.7:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122001831 (376h47m24.605459038s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: read tcp 159.223.143.39:60218->45.91.116.5:4297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122048345 (376h47m24.605505567s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: read tcp 159.223.143.39:45254->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122106214 (376h47m24.605563415s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12qpELr7Yr1aJtgau8q34qPBtADW6QAXPNySAzcRLB39hXoXueo, address:45.91.116.8:4070): protocol: read tcp 159.223.143.39:50716->45.91.116.8:4070: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122131285 (376h47m24.605588469s) [seg_pos={0 0}, upload=391707]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jan 31 08:42:31.141063957 (376h47m24.624521154s) [upload=391707]: Done uploading segments... Jan 31 08:42:31.141101292 (376h47m24.624558493s) [upload=391707]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 414.367577ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [......................................X.....................] duration: 451.901525ms trace: https://jaeger.dev.storj.io/trace/6c4cbd1edfce8e6 FAIL (2026-01-31 08:58:57 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 451.927451ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.......................................X....................] duration: 612.967838ms trace: https://jaeger.dev.storj.io/trace/6c4cbd1edfce8e6 FAIL (2026-01-31 08:58:23 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 613.064198ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 446.841184ms trace: https://jaeger.dev.storj.io/trace/33d822523a7c81ef FAIL (2026-01-31 08:58:08 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 446.889607ms edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [........................................X...................] duration: 443.821708ms trace: https://jaeger.dev.storj.io/trace/6c4cbd1edfce8e6 FAIL (2026-01-31 08:58:03 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 443.861213ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 394.268843ms trace: https://jaeger.dev.storj.io/trace/62a1509ecc5b5058 FAIL (2026-01-31 08:57:55 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Jan 31 08:42:01.166771110 (376h46m54.650228297s) [upload=391707]: Starting upload Jan 31 08:42:01.166986060 (376h46m54.650443258s) [upload=391707]: Done starting upload Jan 31 08:42:01.167072896 (376h46m54.650530094s) [upload=391707]: Uploading segments... Jan 31 08:42:01.167397789 (376h46m54.650854991s) [upload=391707]: Got next segment. Inline: false Jan 31 08:42:01.167473452 (376h46m54.650930657s) [upload=391707]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 31 08:42:01.167480315 (376h46m54.650937524s) [upload=391707]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 31 08:42:01.195346764 (376h46m54.678803969s) [seg_pos={0 0}, upload=391707]: Begin upload segment... Jan 31 08:42:01.195428575 (376h46m54.678885757s) [seg_pos={0 0}, upload=391707]: Done begin upload segment. Jan 31 08:42:01.195463009 (376h46m54.678920214s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.195847779 (376h46m54.679304986s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.197051385 (376h46m54.680508591s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.197594770 (376h46m54.681051978s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198049760 (376h46m54.681506963s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198528799 (376h46m54.681986007s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198996489 (376h46m54.682453701s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.199526016 (376h46m54.682983298s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.199772893 (376h46m54.683230102s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200006077 (376h46m54.683463277s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200298977 (376h46m54.68375618s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200764860 (376h46m54.684222068s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.201330795 (376h46m54.68478802s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.201862657 (376h46m54.685319865s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.202435299 (376h46m54.685892502s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.203149033 (376h46m54.686606237s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.288234960 (376h46m54.771692157s) [upload=391707]: Next returned nil segment Jan 31 08:42:01.288251604 (376h46m54.771708818s) [upload=391707]: Waiting for error group managing segments... Jan 31 08:42:01.293400797 (376h46m54.776858002s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.294083673 (376h46m54.777540862s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.294285106 (376h46m54.777742305s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295108197 (376h46m54.778565394s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295172272 (376h46m54.778629467s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295900158 (376h46m54.779357349s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295955977 (376h46m54.779413173s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296392614 (376h46m54.779849814s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296441340 (376h46m54.779898537s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296874674 (376h46m54.780331861s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296909101 (376h46m54.780366298s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.297409404 (376h46m54.780866604s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:31.121919846 (376h47m24.605377048s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: read tcp 159.223.143.39:60086->45.91.116.7:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122001831 (376h47m24.605459038s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: read tcp 159.223.143.39:60218->45.91.116.5:4297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122048345 (376h47m24.605505567s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: read tcp 159.223.143.39:45254->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122106214 (376h47m24.605563415s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12qpELr7Yr1aJtgau8q34qPBtADW6QAXPNySAzcRLB39hXoXueo, address:45.91.116.8:4070): protocol: read tcp 159.223.143.39:50716->45.91.116.8:4070: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122131285 (376h47m24.605588469s) [seg_pos={0 0}, upload=391707]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jan 31 08:42:31.141063957 (376h47m24.624521154s) [upload=391707]: Done uploading segments... Jan 31 08:42:31.141101292 (376h47m24.624558493s) [upload=391707]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 394.300587ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 551.124521ms trace: https://jaeger.dev.storj.io/trace/33d822523a7c81ef FAIL (2026-01-31 08:57:48 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 551.150324ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 483.364741ms trace: https://jaeger.dev.storj.io/trace/33d822523a7c81ef FAIL (2026-01-31 08:57:32 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 483.393709ms edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.........................................X..................] duration: 453.902829ms trace: https://jaeger.dev.storj.io/trace/6c4cbd1edfce8e6 FAIL (2026-01-31 08:57:24 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 453.933979ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 495.389721ms trace: https://jaeger.dev.storj.io/trace/62a1509ecc5b5058 FAIL (2026-01-31 08:57:23 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Jan 31 08:42:01.166771110 (376h46m54.650228297s) [upload=391707]: Starting upload Jan 31 08:42:01.166986060 (376h46m54.650443258s) [upload=391707]: Done starting upload Jan 31 08:42:01.167072896 (376h46m54.650530094s) [upload=391707]: Uploading segments... Jan 31 08:42:01.167397789 (376h46m54.650854991s) [upload=391707]: Got next segment. Inline: false Jan 31 08:42:01.167473452 (376h46m54.650930657s) [upload=391707]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 31 08:42:01.167480315 (376h46m54.650937524s) [upload=391707]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 31 08:42:01.195346764 (376h46m54.678803969s) [seg_pos={0 0}, upload=391707]: Begin upload segment... Jan 31 08:42:01.195428575 (376h46m54.678885757s) [seg_pos={0 0}, upload=391707]: Done begin upload segment. Jan 31 08:42:01.195463009 (376h46m54.678920214s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.195847779 (376h46m54.679304986s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.197051385 (376h46m54.680508591s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.197594770 (376h46m54.681051978s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198049760 (376h46m54.681506963s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198528799 (376h46m54.681986007s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198996489 (376h46m54.682453701s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.199526016 (376h46m54.682983298s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.199772893 (376h46m54.683230102s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200006077 (376h46m54.683463277s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200298977 (376h46m54.68375618s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200764860 (376h46m54.684222068s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.201330795 (376h46m54.68478802s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.201862657 (376h46m54.685319865s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.202435299 (376h46m54.685892502s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.203149033 (376h46m54.686606237s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.288234960 (376h46m54.771692157s) [upload=391707]: Next returned nil segment Jan 31 08:42:01.288251604 (376h46m54.771708818s) [upload=391707]: Waiting for error group managing segments... Jan 31 08:42:01.293400797 (376h46m54.776858002s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.294083673 (376h46m54.777540862s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.294285106 (376h46m54.777742305s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295108197 (376h46m54.778565394s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295172272 (376h46m54.778629467s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295900158 (376h46m54.779357349s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295955977 (376h46m54.779413173s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296392614 (376h46m54.779849814s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296441340 (376h46m54.779898537s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296874674 (376h46m54.780331861s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296909101 (376h46m54.780366298s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.297409404 (376h46m54.780866604s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:31.121919846 (376h47m24.605377048s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: read tcp 159.223.143.39:60086->45.91.116.7:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122001831 (376h47m24.605459038s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: read tcp 159.223.143.39:60218->45.91.116.5:4297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122048345 (376h47m24.605505567s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: read tcp 159.223.143.39:45254->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122106214 (376h47m24.605563415s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12qpELr7Yr1aJtgau8q34qPBtADW6QAXPNySAzcRLB39hXoXueo, address:45.91.116.8:4070): protocol: read tcp 159.223.143.39:50716->45.91.116.8:4070: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122131285 (376h47m24.605588469s) [seg_pos={0 0}, upload=391707]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jan 31 08:42:31.141063957 (376h47m24.624521154s) [upload=391707]: Done uploading segments... Jan 31 08:42:31.141101292 (376h47m24.624558493s) [upload=391707]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 495.419828ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..........................................X.................] duration: 395.577649ms trace: https://jaeger.dev.storj.io/trace/6c4cbd1edfce8e6 FAIL (2026-01-31 08:56:55 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 395.676823ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 442.197695ms trace: https://jaeger.dev.storj.io/trace/33d822523a7c81ef FAIL (2026-01-31 08:56:36 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 442.316092ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 369.817167ms trace: https://jaeger.dev.storj.io/trace/62a1509ecc5b5058 FAIL (2026-01-31 08:56:17 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Jan 31 08:42:01.166771110 (376h46m54.650228297s) [upload=391707]: Starting upload Jan 31 08:42:01.166986060 (376h46m54.650443258s) [upload=391707]: Done starting upload Jan 31 08:42:01.167072896 (376h46m54.650530094s) [upload=391707]: Uploading segments... Jan 31 08:42:01.167397789 (376h46m54.650854991s) [upload=391707]: Got next segment. Inline: false Jan 31 08:42:01.167473452 (376h46m54.650930657s) [upload=391707]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 31 08:42:01.167480315 (376h46m54.650937524s) [upload=391707]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 31 08:42:01.195346764 (376h46m54.678803969s) [seg_pos={0 0}, upload=391707]: Begin upload segment... Jan 31 08:42:01.195428575 (376h46m54.678885757s) [seg_pos={0 0}, upload=391707]: Done begin upload segment. Jan 31 08:42:01.195463009 (376h46m54.678920214s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.195847779 (376h46m54.679304986s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.197051385 (376h46m54.680508591s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.197594770 (376h46m54.681051978s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198049760 (376h46m54.681506963s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198528799 (376h46m54.681986007s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198996489 (376h46m54.682453701s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.199526016 (376h46m54.682983298s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.199772893 (376h46m54.683230102s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200006077 (376h46m54.683463277s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200298977 (376h46m54.68375618s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200764860 (376h46m54.684222068s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.201330795 (376h46m54.68478802s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.201862657 (376h46m54.685319865s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.202435299 (376h46m54.685892502s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.203149033 (376h46m54.686606237s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.288234960 (376h46m54.771692157s) [upload=391707]: Next returned nil segment Jan 31 08:42:01.288251604 (376h46m54.771708818s) [upload=391707]: Waiting for error group managing segments... Jan 31 08:42:01.293400797 (376h46m54.776858002s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.294083673 (376h46m54.777540862s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.294285106 (376h46m54.777742305s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295108197 (376h46m54.778565394s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295172272 (376h46m54.778629467s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295900158 (376h46m54.779357349s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295955977 (376h46m54.779413173s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296392614 (376h46m54.779849814s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296441340 (376h46m54.779898537s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296874674 (376h46m54.780331861s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296909101 (376h46m54.780366298s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.297409404 (376h46m54.780866604s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:31.121919846 (376h47m24.605377048s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: read tcp 159.223.143.39:60086->45.91.116.7:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122001831 (376h47m24.605459038s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: read tcp 159.223.143.39:60218->45.91.116.5:4297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122048345 (376h47m24.605505567s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: read tcp 159.223.143.39:45254->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122106214 (376h47m24.605563415s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12qpELr7Yr1aJtgau8q34qPBtADW6QAXPNySAzcRLB39hXoXueo, address:45.91.116.8:4070): protocol: read tcp 159.223.143.39:50716->45.91.116.8:4070: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122131285 (376h47m24.605588469s) [seg_pos={0 0}, upload=391707]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jan 31 08:42:31.141063957 (376h47m24.624521154s) [upload=391707]: Done uploading segments... Jan 31 08:42:31.141101292 (376h47m24.624558493s) [upload=391707]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 369.845241ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 473.068975ms trace: https://jaeger.dev.storj.io/trace/33d822523a7c81ef FAIL (2026-01-31 08:56:05 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 473.130355ms edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...........................................X................] duration: 400.2272ms trace: https://jaeger.dev.storj.io/trace/6c4cbd1edfce8e6 FAIL (2026-01-31 08:55:47 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 400.249388ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 471.441844ms trace: https://jaeger.dev.storj.io/trace/33d822523a7c81ef FAIL (2026-01-31 08:55:34 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 471.471296ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 378.528996ms trace: https://jaeger.dev.storj.io/trace/62a1509ecc5b5058 FAIL (2026-01-31 08:55:22 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Jan 31 08:42:01.166771110 (376h46m54.650228297s) [upload=391707]: Starting upload Jan 31 08:42:01.166986060 (376h46m54.650443258s) [upload=391707]: Done starting upload Jan 31 08:42:01.167072896 (376h46m54.650530094s) [upload=391707]: Uploading segments... Jan 31 08:42:01.167397789 (376h46m54.650854991s) [upload=391707]: Got next segment. Inline: false Jan 31 08:42:01.167473452 (376h46m54.650930657s) [upload=391707]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 31 08:42:01.167480315 (376h46m54.650937524s) [upload=391707]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 31 08:42:01.195346764 (376h46m54.678803969s) [seg_pos={0 0}, upload=391707]: Begin upload segment... Jan 31 08:42:01.195428575 (376h46m54.678885757s) [seg_pos={0 0}, upload=391707]: Done begin upload segment. Jan 31 08:42:01.195463009 (376h46m54.678920214s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.195847779 (376h46m54.679304986s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.197051385 (376h46m54.680508591s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.197594770 (376h46m54.681051978s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198049760 (376h46m54.681506963s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198528799 (376h46m54.681986007s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198996489 (376h46m54.682453701s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.199526016 (376h46m54.682983298s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.199772893 (376h46m54.683230102s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200006077 (376h46m54.683463277s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200298977 (376h46m54.68375618s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200764860 (376h46m54.684222068s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.201330795 (376h46m54.68478802s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.201862657 (376h46m54.685319865s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.202435299 (376h46m54.685892502s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.203149033 (376h46m54.686606237s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.288234960 (376h46m54.771692157s) [upload=391707]: Next returned nil segment Jan 31 08:42:01.288251604 (376h46m54.771708818s) [upload=391707]: Waiting for error group managing segments... Jan 31 08:42:01.293400797 (376h46m54.776858002s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.294083673 (376h46m54.777540862s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.294285106 (376h46m54.777742305s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295108197 (376h46m54.778565394s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295172272 (376h46m54.778629467s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295900158 (376h46m54.779357349s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295955977 (376h46m54.779413173s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296392614 (376h46m54.779849814s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296441340 (376h46m54.779898537s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296874674 (376h46m54.780331861s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296909101 (376h46m54.780366298s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.297409404 (376h46m54.780866604s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:31.121919846 (376h47m24.605377048s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: read tcp 159.223.143.39:60086->45.91.116.7:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122001831 (376h47m24.605459038s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: read tcp 159.223.143.39:60218->45.91.116.5:4297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122048345 (376h47m24.605505567s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: read tcp 159.223.143.39:45254->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122106214 (376h47m24.605563415s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12qpELr7Yr1aJtgau8q34qPBtADW6QAXPNySAzcRLB39hXoXueo, address:45.91.116.8:4070): protocol: read tcp 159.223.143.39:50716->45.91.116.8:4070: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122131285 (376h47m24.605588469s) [seg_pos={0 0}, upload=391707]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jan 31 08:42:31.141063957 (376h47m24.624521154s) [upload=391707]: Done uploading segments... Jan 31 08:42:31.141101292 (376h47m24.624558493s) [upload=391707]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 378.561412ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [............................................X...............] duration: 451.39823ms trace: https://jaeger.dev.storj.io/trace/6c4cbd1edfce8e6 FAIL (2026-01-31 08:54:52 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 451.424514ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 438.285391ms trace: https://jaeger.dev.storj.io/trace/62a1509ecc5b5058 FAIL (2026-01-31 08:54:35 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Jan 31 08:42:01.166771110 (376h46m54.650228297s) [upload=391707]: Starting upload Jan 31 08:42:01.166986060 (376h46m54.650443258s) [upload=391707]: Done starting upload Jan 31 08:42:01.167072896 (376h46m54.650530094s) [upload=391707]: Uploading segments... Jan 31 08:42:01.167397789 (376h46m54.650854991s) [upload=391707]: Got next segment. Inline: false Jan 31 08:42:01.167473452 (376h46m54.650930657s) [upload=391707]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 31 08:42:01.167480315 (376h46m54.650937524s) [upload=391707]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 31 08:42:01.195346764 (376h46m54.678803969s) [seg_pos={0 0}, upload=391707]: Begin upload segment... Jan 31 08:42:01.195428575 (376h46m54.678885757s) [seg_pos={0 0}, upload=391707]: Done begin upload segment. Jan 31 08:42:01.195463009 (376h46m54.678920214s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.195847779 (376h46m54.679304986s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.197051385 (376h46m54.680508591s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.197594770 (376h46m54.681051978s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198049760 (376h46m54.681506963s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198528799 (376h46m54.681986007s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198996489 (376h46m54.682453701s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.199526016 (376h46m54.682983298s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.199772893 (376h46m54.683230102s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200006077 (376h46m54.683463277s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200298977 (376h46m54.68375618s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200764860 (376h46m54.684222068s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.201330795 (376h46m54.68478802s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.201862657 (376h46m54.685319865s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.202435299 (376h46m54.685892502s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.203149033 (376h46m54.686606237s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.288234960 (376h46m54.771692157s) [upload=391707]: Next returned nil segment Jan 31 08:42:01.288251604 (376h46m54.771708818s) [upload=391707]: Waiting for error group managing segments... Jan 31 08:42:01.293400797 (376h46m54.776858002s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.294083673 (376h46m54.777540862s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.294285106 (376h46m54.777742305s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295108197 (376h46m54.778565394s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295172272 (376h46m54.778629467s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295900158 (376h46m54.779357349s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295955977 (376h46m54.779413173s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296392614 (376h46m54.779849814s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296441340 (376h46m54.779898537s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296874674 (376h46m54.780331861s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296909101 (376h46m54.780366298s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.297409404 (376h46m54.780866604s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:31.121919846 (376h47m24.605377048s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: read tcp 159.223.143.39:60086->45.91.116.7:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122001831 (376h47m24.605459038s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: read tcp 159.223.143.39:60218->45.91.116.5:4297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122048345 (376h47m24.605505567s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: read tcp 159.223.143.39:45254->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122106214 (376h47m24.605563415s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12qpELr7Yr1aJtgau8q34qPBtADW6QAXPNySAzcRLB39hXoXueo, address:45.91.116.8:4070): protocol: read tcp 159.223.143.39:50716->45.91.116.8:4070: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122131285 (376h47m24.605588469s) [seg_pos={0 0}, upload=391707]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jan 31 08:42:31.141063957 (376h47m24.624521154s) [upload=391707]: Done uploading segments... Jan 31 08:42:31.141101292 (376h47m24.624558493s) [upload=391707]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 438.300676ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.............................................X..............] duration: 411.186471ms trace: https://jaeger.dev.storj.io/trace/6c4cbd1edfce8e6 FAIL (2026-01-31 08:54:24 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 411.413728ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 452.938861ms trace: https://jaeger.dev.storj.io/trace/33d822523a7c81ef FAIL (2026-01-31 08:54:23 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 452.964156ms edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..............................................X.............] duration: 423.77115ms trace: https://jaeger.dev.storj.io/trace/6c4cbd1edfce8e6 FAIL (2026-01-31 08:53:43 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 423.840602ms core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)