Orbiter History

Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure

Orbiter Check History

Showing 50 of 177752 reports (Page 81 of 3556)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 455.092984ms trace: https://jaeger.dev.storj.io/trace/62a1509ecc5b5058 FAIL (2026-01-31 08:53:42 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Jan 31 08:42:01.166771110 (376h46m54.650228297s) [upload=391707]: Starting upload Jan 31 08:42:01.166986060 (376h46m54.650443258s) [upload=391707]: Done starting upload Jan 31 08:42:01.167072896 (376h46m54.650530094s) [upload=391707]: Uploading segments... Jan 31 08:42:01.167397789 (376h46m54.650854991s) [upload=391707]: Got next segment. Inline: false Jan 31 08:42:01.167473452 (376h46m54.650930657s) [upload=391707]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 31 08:42:01.167480315 (376h46m54.650937524s) [upload=391707]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 31 08:42:01.195346764 (376h46m54.678803969s) [seg_pos={0 0}, upload=391707]: Begin upload segment... Jan 31 08:42:01.195428575 (376h46m54.678885757s) [seg_pos={0 0}, upload=391707]: Done begin upload segment. Jan 31 08:42:01.195463009 (376h46m54.678920214s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.195847779 (376h46m54.679304986s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.197051385 (376h46m54.680508591s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.197594770 (376h46m54.681051978s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198049760 (376h46m54.681506963s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198528799 (376h46m54.681986007s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198996489 (376h46m54.682453701s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.199526016 (376h46m54.682983298s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.199772893 (376h46m54.683230102s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200006077 (376h46m54.683463277s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200298977 (376h46m54.68375618s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200764860 (376h46m54.684222068s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.201330795 (376h46m54.68478802s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.201862657 (376h46m54.685319865s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.202435299 (376h46m54.685892502s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.203149033 (376h46m54.686606237s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.288234960 (376h46m54.771692157s) [upload=391707]: Next returned nil segment Jan 31 08:42:01.288251604 (376h46m54.771708818s) [upload=391707]: Waiting for error group managing segments... Jan 31 08:42:01.293400797 (376h46m54.776858002s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.294083673 (376h46m54.777540862s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.294285106 (376h46m54.777742305s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295108197 (376h46m54.778565394s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295172272 (376h46m54.778629467s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295900158 (376h46m54.779357349s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295955977 (376h46m54.779413173s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296392614 (376h46m54.779849814s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296441340 (376h46m54.779898537s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296874674 (376h46m54.780331861s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296909101 (376h46m54.780366298s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.297409404 (376h46m54.780866604s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:31.121919846 (376h47m24.605377048s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: read tcp 159.223.143.39:60086->45.91.116.7:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122001831 (376h47m24.605459038s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: read tcp 159.223.143.39:60218->45.91.116.5:4297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122048345 (376h47m24.605505567s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: read tcp 159.223.143.39:45254->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122106214 (376h47m24.605563415s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12qpELr7Yr1aJtgau8q34qPBtADW6QAXPNySAzcRLB39hXoXueo, address:45.91.116.8:4070): protocol: read tcp 159.223.143.39:50716->45.91.116.8:4070: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122131285 (376h47m24.605588469s) [seg_pos={0 0}, upload=391707]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jan 31 08:42:31.141063957 (376h47m24.624521154s) [upload=391707]: Done uploading segments... Jan 31 08:42:31.141101292 (376h47m24.624558493s) [upload=391707]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 455.132076ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 431.345864ms trace: https://jaeger.dev.storj.io/trace/33d822523a7c81ef FAIL (2026-01-31 08:53:11 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 431.375101ms edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...............................................X............] duration: 400.89312ms trace: https://jaeger.dev.storj.io/trace/6c4cbd1edfce8e6 FAIL (2026-01-31 08:52:59 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 400.919097ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 490.060084ms trace: https://jaeger.dev.storj.io/trace/62a1509ecc5b5058 FAIL (2026-01-31 08:52:29 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Jan 31 08:42:01.166771110 (376h46m54.650228297s) [upload=391707]: Starting upload Jan 31 08:42:01.166986060 (376h46m54.650443258s) [upload=391707]: Done starting upload Jan 31 08:42:01.167072896 (376h46m54.650530094s) [upload=391707]: Uploading segments... Jan 31 08:42:01.167397789 (376h46m54.650854991s) [upload=391707]: Got next segment. Inline: false Jan 31 08:42:01.167473452 (376h46m54.650930657s) [upload=391707]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 31 08:42:01.167480315 (376h46m54.650937524s) [upload=391707]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 31 08:42:01.195346764 (376h46m54.678803969s) [seg_pos={0 0}, upload=391707]: Begin upload segment... Jan 31 08:42:01.195428575 (376h46m54.678885757s) [seg_pos={0 0}, upload=391707]: Done begin upload segment. Jan 31 08:42:01.195463009 (376h46m54.678920214s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.195847779 (376h46m54.679304986s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.197051385 (376h46m54.680508591s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.197594770 (376h46m54.681051978s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198049760 (376h46m54.681506963s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198528799 (376h46m54.681986007s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198996489 (376h46m54.682453701s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.199526016 (376h46m54.682983298s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.199772893 (376h46m54.683230102s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200006077 (376h46m54.683463277s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200298977 (376h46m54.68375618s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200764860 (376h46m54.684222068s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.201330795 (376h46m54.68478802s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.201862657 (376h46m54.685319865s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.202435299 (376h46m54.685892502s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.203149033 (376h46m54.686606237s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.288234960 (376h46m54.771692157s) [upload=391707]: Next returned nil segment Jan 31 08:42:01.288251604 (376h46m54.771708818s) [upload=391707]: Waiting for error group managing segments... Jan 31 08:42:01.293400797 (376h46m54.776858002s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.294083673 (376h46m54.777540862s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.294285106 (376h46m54.777742305s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295108197 (376h46m54.778565394s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295172272 (376h46m54.778629467s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295900158 (376h46m54.779357349s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295955977 (376h46m54.779413173s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296392614 (376h46m54.779849814s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296441340 (376h46m54.779898537s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296874674 (376h46m54.780331861s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296909101 (376h46m54.780366298s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.297409404 (376h46m54.780866604s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:31.121919846 (376h47m24.605377048s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: read tcp 159.223.143.39:60086->45.91.116.7:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122001831 (376h47m24.605459038s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: read tcp 159.223.143.39:60218->45.91.116.5:4297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122048345 (376h47m24.605505567s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: read tcp 159.223.143.39:45254->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122106214 (376h47m24.605563415s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12qpELr7Yr1aJtgau8q34qPBtADW6QAXPNySAzcRLB39hXoXueo, address:45.91.116.8:4070): protocol: read tcp 159.223.143.39:50716->45.91.116.8:4070: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122131285 (376h47m24.605588469s) [seg_pos={0 0}, upload=391707]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jan 31 08:42:31.141063957 (376h47m24.624521154s) [upload=391707]: Done uploading segments... Jan 31 08:42:31.141101292 (376h47m24.624558493s) [upload=391707]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 490.089463ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [................................................X...........] duration: 419.901731ms trace: https://jaeger.dev.storj.io/trace/6c4cbd1edfce8e6 FAIL (2026-01-31 08:52:04 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 419.929316ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 504.571912ms trace: https://jaeger.dev.storj.io/trace/33d822523a7c81ef FAIL (2026-01-31 08:52:03 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 504.600205ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 380.289943ms trace: https://jaeger.dev.storj.io/trace/62a1509ecc5b5058 FAIL (2026-01-31 08:51:35 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Jan 31 08:42:01.166771110 (376h46m54.650228297s) [upload=391707]: Starting upload Jan 31 08:42:01.166986060 (376h46m54.650443258s) [upload=391707]: Done starting upload Jan 31 08:42:01.167072896 (376h46m54.650530094s) [upload=391707]: Uploading segments... Jan 31 08:42:01.167397789 (376h46m54.650854991s) [upload=391707]: Got next segment. Inline: false Jan 31 08:42:01.167473452 (376h46m54.650930657s) [upload=391707]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 31 08:42:01.167480315 (376h46m54.650937524s) [upload=391707]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 31 08:42:01.195346764 (376h46m54.678803969s) [seg_pos={0 0}, upload=391707]: Begin upload segment... Jan 31 08:42:01.195428575 (376h46m54.678885757s) [seg_pos={0 0}, upload=391707]: Done begin upload segment. Jan 31 08:42:01.195463009 (376h46m54.678920214s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.195847779 (376h46m54.679304986s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.197051385 (376h46m54.680508591s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.197594770 (376h46m54.681051978s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198049760 (376h46m54.681506963s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198528799 (376h46m54.681986007s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198996489 (376h46m54.682453701s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.199526016 (376h46m54.682983298s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.199772893 (376h46m54.683230102s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200006077 (376h46m54.683463277s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200298977 (376h46m54.68375618s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200764860 (376h46m54.684222068s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.201330795 (376h46m54.68478802s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.201862657 (376h46m54.685319865s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.202435299 (376h46m54.685892502s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.203149033 (376h46m54.686606237s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.288234960 (376h46m54.771692157s) [upload=391707]: Next returned nil segment Jan 31 08:42:01.288251604 (376h46m54.771708818s) [upload=391707]: Waiting for error group managing segments... Jan 31 08:42:01.293400797 (376h46m54.776858002s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.294083673 (376h46m54.777540862s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.294285106 (376h46m54.777742305s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295108197 (376h46m54.778565394s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295172272 (376h46m54.778629467s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295900158 (376h46m54.779357349s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295955977 (376h46m54.779413173s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296392614 (376h46m54.779849814s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296441340 (376h46m54.779898537s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296874674 (376h46m54.780331861s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296909101 (376h46m54.780366298s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.297409404 (376h46m54.780866604s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:31.121919846 (376h47m24.605377048s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: read tcp 159.223.143.39:60086->45.91.116.7:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122001831 (376h47m24.605459038s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: read tcp 159.223.143.39:60218->45.91.116.5:4297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122048345 (376h47m24.605505567s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: read tcp 159.223.143.39:45254->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122106214 (376h47m24.605563415s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12qpELr7Yr1aJtgau8q34qPBtADW6QAXPNySAzcRLB39hXoXueo, address:45.91.116.8:4070): protocol: read tcp 159.223.143.39:50716->45.91.116.8:4070: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122131285 (376h47m24.605588469s) [seg_pos={0 0}, upload=391707]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jan 31 08:42:31.141063957 (376h47m24.624521154s) [upload=391707]: Done uploading segments... Jan 31 08:42:31.141101292 (376h47m24.624558493s) [upload=391707]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 380.375368ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 448.36425ms trace: https://jaeger.dev.storj.io/trace/33d822523a7c81ef FAIL (2026-01-31 08:51:06 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 448.54213ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 470.124578ms trace: https://jaeger.dev.storj.io/trace/62a1509ecc5b5058 FAIL (2026-01-31 08:50:57 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Jan 31 08:42:01.166771110 (376h46m54.650228297s) [upload=391707]: Starting upload Jan 31 08:42:01.166986060 (376h46m54.650443258s) [upload=391707]: Done starting upload Jan 31 08:42:01.167072896 (376h46m54.650530094s) [upload=391707]: Uploading segments... Jan 31 08:42:01.167397789 (376h46m54.650854991s) [upload=391707]: Got next segment. Inline: false Jan 31 08:42:01.167473452 (376h46m54.650930657s) [upload=391707]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 31 08:42:01.167480315 (376h46m54.650937524s) [upload=391707]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 31 08:42:01.195346764 (376h46m54.678803969s) [seg_pos={0 0}, upload=391707]: Begin upload segment... Jan 31 08:42:01.195428575 (376h46m54.678885757s) [seg_pos={0 0}, upload=391707]: Done begin upload segment. Jan 31 08:42:01.195463009 (376h46m54.678920214s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.195847779 (376h46m54.679304986s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.197051385 (376h46m54.680508591s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.197594770 (376h46m54.681051978s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198049760 (376h46m54.681506963s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198528799 (376h46m54.681986007s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198996489 (376h46m54.682453701s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.199526016 (376h46m54.682983298s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.199772893 (376h46m54.683230102s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200006077 (376h46m54.683463277s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200298977 (376h46m54.68375618s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200764860 (376h46m54.684222068s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.201330795 (376h46m54.68478802s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.201862657 (376h46m54.685319865s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.202435299 (376h46m54.685892502s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.203149033 (376h46m54.686606237s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.288234960 (376h46m54.771692157s) [upload=391707]: Next returned nil segment Jan 31 08:42:01.288251604 (376h46m54.771708818s) [upload=391707]: Waiting for error group managing segments... Jan 31 08:42:01.293400797 (376h46m54.776858002s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.294083673 (376h46m54.777540862s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.294285106 (376h46m54.777742305s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295108197 (376h46m54.778565394s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295172272 (376h46m54.778629467s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295900158 (376h46m54.779357349s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295955977 (376h46m54.779413173s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296392614 (376h46m54.779849814s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296441340 (376h46m54.779898537s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296874674 (376h46m54.780331861s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296909101 (376h46m54.780366298s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.297409404 (376h46m54.780866604s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:31.121919846 (376h47m24.605377048s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: read tcp 159.223.143.39:60086->45.91.116.7:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122001831 (376h47m24.605459038s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: read tcp 159.223.143.39:60218->45.91.116.5:4297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122048345 (376h47m24.605505567s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: read tcp 159.223.143.39:45254->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122106214 (376h47m24.605563415s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12qpELr7Yr1aJtgau8q34qPBtADW6QAXPNySAzcRLB39hXoXueo, address:45.91.116.8:4070): protocol: read tcp 159.223.143.39:50716->45.91.116.8:4070: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122131285 (376h47m24.605588469s) [seg_pos={0 0}, upload=391707]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jan 31 08:42:31.141063957 (376h47m24.624521154s) [upload=391707]: Done uploading segments... Jan 31 08:42:31.141101292 (376h47m24.624558493s) [upload=391707]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 470.159105ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.................................................X..........] duration: 429.71733ms trace: https://jaeger.dev.storj.io/trace/6c4cbd1edfce8e6 FAIL (2026-01-31 08:50:53 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 429.85538ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 466.536722ms trace: https://jaeger.dev.storj.io/trace/33d822523a7c81ef FAIL (2026-01-31 08:50:46 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 466.567784ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 477.454337ms trace: https://jaeger.dev.storj.io/trace/33d822523a7c81ef FAIL (2026-01-31 08:50:11 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 477.511901ms edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..................................................X.........] duration: 425.529365ms trace: https://jaeger.dev.storj.io/trace/6c4cbd1edfce8e6 FAIL (2026-01-31 08:49:58 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 425.623703ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 514.835518ms trace: https://jaeger.dev.storj.io/trace/62a1509ecc5b5058 FAIL (2026-01-31 08:49:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Jan 31 08:42:01.166771110 (376h46m54.650228297s) [upload=391707]: Starting upload Jan 31 08:42:01.166986060 (376h46m54.650443258s) [upload=391707]: Done starting upload Jan 31 08:42:01.167072896 (376h46m54.650530094s) [upload=391707]: Uploading segments... Jan 31 08:42:01.167397789 (376h46m54.650854991s) [upload=391707]: Got next segment. Inline: false Jan 31 08:42:01.167473452 (376h46m54.650930657s) [upload=391707]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 31 08:42:01.167480315 (376h46m54.650937524s) [upload=391707]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 31 08:42:01.195346764 (376h46m54.678803969s) [seg_pos={0 0}, upload=391707]: Begin upload segment... Jan 31 08:42:01.195428575 (376h46m54.678885757s) [seg_pos={0 0}, upload=391707]: Done begin upload segment. Jan 31 08:42:01.195463009 (376h46m54.678920214s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.195847779 (376h46m54.679304986s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.197051385 (376h46m54.680508591s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.197594770 (376h46m54.681051978s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198049760 (376h46m54.681506963s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198528799 (376h46m54.681986007s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198996489 (376h46m54.682453701s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.199526016 (376h46m54.682983298s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.199772893 (376h46m54.683230102s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200006077 (376h46m54.683463277s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200298977 (376h46m54.68375618s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200764860 (376h46m54.684222068s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.201330795 (376h46m54.68478802s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.201862657 (376h46m54.685319865s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.202435299 (376h46m54.685892502s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.203149033 (376h46m54.686606237s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.288234960 (376h46m54.771692157s) [upload=391707]: Next returned nil segment Jan 31 08:42:01.288251604 (376h46m54.771708818s) [upload=391707]: Waiting for error group managing segments... Jan 31 08:42:01.293400797 (376h46m54.776858002s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.294083673 (376h46m54.777540862s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.294285106 (376h46m54.777742305s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295108197 (376h46m54.778565394s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295172272 (376h46m54.778629467s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295900158 (376h46m54.779357349s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295955977 (376h46m54.779413173s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296392614 (376h46m54.779849814s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296441340 (376h46m54.779898537s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296874674 (376h46m54.780331861s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296909101 (376h46m54.780366298s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.297409404 (376h46m54.780866604s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:31.121919846 (376h47m24.605377048s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: read tcp 159.223.143.39:60086->45.91.116.7:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122001831 (376h47m24.605459038s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: read tcp 159.223.143.39:60218->45.91.116.5:4297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122048345 (376h47m24.605505567s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: read tcp 159.223.143.39:45254->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122106214 (376h47m24.605563415s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12qpELr7Yr1aJtgau8q34qPBtADW6QAXPNySAzcRLB39hXoXueo, address:45.91.116.8:4070): protocol: read tcp 159.223.143.39:50716->45.91.116.8:4070: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122131285 (376h47m24.605588469s) [seg_pos={0 0}, upload=391707]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jan 31 08:42:31.141063957 (376h47m24.624521154s) [upload=391707]: Done uploading segments... Jan 31 08:42:31.141101292 (376h47m24.624558493s) [upload=391707]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 514.86677ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...................................................X........] duration: 452.057898ms trace: https://jaeger.dev.storj.io/trace/6c4cbd1edfce8e6 FAIL (2026-01-31 08:49:29 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 452.085092ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 451.380052ms trace: https://jaeger.dev.storj.io/trace/33d822523a7c81ef FAIL (2026-01-31 08:49:27 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 451.402894ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 454.488073ms trace: https://jaeger.dev.storj.io/trace/33d822523a7c81ef FAIL (2026-01-31 08:49:08 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 454.514283ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 470.063057ms trace: https://jaeger.dev.storj.io/trace/62a1509ecc5b5058 FAIL (2026-01-31 08:48:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Jan 31 08:42:01.166771110 (376h46m54.650228297s) [upload=391707]: Starting upload Jan 31 08:42:01.166986060 (376h46m54.650443258s) [upload=391707]: Done starting upload Jan 31 08:42:01.167072896 (376h46m54.650530094s) [upload=391707]: Uploading segments... Jan 31 08:42:01.167397789 (376h46m54.650854991s) [upload=391707]: Got next segment. Inline: false Jan 31 08:42:01.167473452 (376h46m54.650930657s) [upload=391707]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 31 08:42:01.167480315 (376h46m54.650937524s) [upload=391707]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 31 08:42:01.195346764 (376h46m54.678803969s) [seg_pos={0 0}, upload=391707]: Begin upload segment... Jan 31 08:42:01.195428575 (376h46m54.678885757s) [seg_pos={0 0}, upload=391707]: Done begin upload segment. Jan 31 08:42:01.195463009 (376h46m54.678920214s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.195847779 (376h46m54.679304986s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.197051385 (376h46m54.680508591s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.197594770 (376h46m54.681051978s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198049760 (376h46m54.681506963s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198528799 (376h46m54.681986007s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198996489 (376h46m54.682453701s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.199526016 (376h46m54.682983298s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.199772893 (376h46m54.683230102s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200006077 (376h46m54.683463277s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200298977 (376h46m54.68375618s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200764860 (376h46m54.684222068s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.201330795 (376h46m54.68478802s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.201862657 (376h46m54.685319865s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.202435299 (376h46m54.685892502s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.203149033 (376h46m54.686606237s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.288234960 (376h46m54.771692157s) [upload=391707]: Next returned nil segment Jan 31 08:42:01.288251604 (376h46m54.771708818s) [upload=391707]: Waiting for error group managing segments... Jan 31 08:42:01.293400797 (376h46m54.776858002s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.294083673 (376h46m54.777540862s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.294285106 (376h46m54.777742305s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295108197 (376h46m54.778565394s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295172272 (376h46m54.778629467s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295900158 (376h46m54.779357349s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295955977 (376h46m54.779413173s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296392614 (376h46m54.779849814s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296441340 (376h46m54.779898537s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296874674 (376h46m54.780331861s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296909101 (376h46m54.780366298s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.297409404 (376h46m54.780866604s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:31.121919846 (376h47m24.605377048s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: read tcp 159.223.143.39:60086->45.91.116.7:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122001831 (376h47m24.605459038s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: read tcp 159.223.143.39:60218->45.91.116.5:4297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122048345 (376h47m24.605505567s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: read tcp 159.223.143.39:45254->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122106214 (376h47m24.605563415s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12qpELr7Yr1aJtgau8q34qPBtADW6QAXPNySAzcRLB39hXoXueo, address:45.91.116.8:4070): protocol: read tcp 159.223.143.39:50716->45.91.116.8:4070: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122131285 (376h47m24.605588469s) [seg_pos={0 0}, upload=391707]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jan 31 08:42:31.141063957 (376h47m24.624521154s) [upload=391707]: Done uploading segments... Jan 31 08:42:31.141101292 (376h47m24.624558493s) [upload=391707]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 470.08911ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 463.048068ms trace: https://jaeger.dev.storj.io/trace/33d822523a7c81ef FAIL (2026-01-31 08:48:32 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 463.097855ms edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [....................................................X.......] duration: 472.259214ms trace: https://jaeger.dev.storj.io/trace/6c4cbd1edfce8e6 FAIL (2026-01-31 08:48:22 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 472.311873ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 525.85447ms trace: https://jaeger.dev.storj.io/trace/33d822523a7c81ef FAIL (2026-01-31 08:48:16 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 525.890962ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 443.060377ms trace: https://jaeger.dev.storj.io/trace/62a1509ecc5b5058 FAIL (2026-01-31 08:47:58 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Jan 31 08:42:01.166771110 (376h46m54.650228297s) [upload=391707]: Starting upload Jan 31 08:42:01.166986060 (376h46m54.650443258s) [upload=391707]: Done starting upload Jan 31 08:42:01.167072896 (376h46m54.650530094s) [upload=391707]: Uploading segments... Jan 31 08:42:01.167397789 (376h46m54.650854991s) [upload=391707]: Got next segment. Inline: false Jan 31 08:42:01.167473452 (376h46m54.650930657s) [upload=391707]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 31 08:42:01.167480315 (376h46m54.650937524s) [upload=391707]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 31 08:42:01.195346764 (376h46m54.678803969s) [seg_pos={0 0}, upload=391707]: Begin upload segment... Jan 31 08:42:01.195428575 (376h46m54.678885757s) [seg_pos={0 0}, upload=391707]: Done begin upload segment. Jan 31 08:42:01.195463009 (376h46m54.678920214s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.195847779 (376h46m54.679304986s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.197051385 (376h46m54.680508591s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.197594770 (376h46m54.681051978s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198049760 (376h46m54.681506963s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198528799 (376h46m54.681986007s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198996489 (376h46m54.682453701s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.199526016 (376h46m54.682983298s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.199772893 (376h46m54.683230102s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200006077 (376h46m54.683463277s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200298977 (376h46m54.68375618s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200764860 (376h46m54.684222068s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.201330795 (376h46m54.68478802s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.201862657 (376h46m54.685319865s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.202435299 (376h46m54.685892502s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.203149033 (376h46m54.686606237s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.288234960 (376h46m54.771692157s) [upload=391707]: Next returned nil segment Jan 31 08:42:01.288251604 (376h46m54.771708818s) [upload=391707]: Waiting for error group managing segments... Jan 31 08:42:01.293400797 (376h46m54.776858002s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.294083673 (376h46m54.777540862s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.294285106 (376h46m54.777742305s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295108197 (376h46m54.778565394s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295172272 (376h46m54.778629467s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295900158 (376h46m54.779357349s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295955977 (376h46m54.779413173s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296392614 (376h46m54.779849814s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296441340 (376h46m54.779898537s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296874674 (376h46m54.780331861s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296909101 (376h46m54.780366298s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.297409404 (376h46m54.780866604s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:31.121919846 (376h47m24.605377048s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: read tcp 159.223.143.39:60086->45.91.116.7:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122001831 (376h47m24.605459038s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: read tcp 159.223.143.39:60218->45.91.116.5:4297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122048345 (376h47m24.605505567s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: read tcp 159.223.143.39:45254->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122106214 (376h47m24.605563415s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12qpELr7Yr1aJtgau8q34qPBtADW6QAXPNySAzcRLB39hXoXueo, address:45.91.116.8:4070): protocol: read tcp 159.223.143.39:50716->45.91.116.8:4070: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122131285 (376h47m24.605588469s) [seg_pos={0 0}, upload=391707]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jan 31 08:42:31.141063957 (376h47m24.624521154s) [upload=391707]: Done uploading segments... Jan 31 08:42:31.141101292 (376h47m24.624558493s) [upload=391707]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 443.091028ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.....................................................X......] duration: 463.595918ms trace: https://jaeger.dev.storj.io/trace/6c4cbd1edfce8e6 FAIL (2026-01-31 08:47:20 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 463.69501ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 459.171012ms trace: https://jaeger.dev.storj.io/trace/33d822523a7c81ef FAIL (2026-01-31 08:47:13 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 459.193898ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 493.226656ms trace: https://jaeger.dev.storj.io/trace/62a1509ecc5b5058 FAIL (2026-01-31 08:47:07 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Jan 31 08:42:01.166771110 (376h46m54.650228297s) [upload=391707]: Starting upload Jan 31 08:42:01.166986060 (376h46m54.650443258s) [upload=391707]: Done starting upload Jan 31 08:42:01.167072896 (376h46m54.650530094s) [upload=391707]: Uploading segments... Jan 31 08:42:01.167397789 (376h46m54.650854991s) [upload=391707]: Got next segment. Inline: false Jan 31 08:42:01.167473452 (376h46m54.650930657s) [upload=391707]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 31 08:42:01.167480315 (376h46m54.650937524s) [upload=391707]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 31 08:42:01.195346764 (376h46m54.678803969s) [seg_pos={0 0}, upload=391707]: Begin upload segment... Jan 31 08:42:01.195428575 (376h46m54.678885757s) [seg_pos={0 0}, upload=391707]: Done begin upload segment. Jan 31 08:42:01.195463009 (376h46m54.678920214s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.195847779 (376h46m54.679304986s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.197051385 (376h46m54.680508591s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.197594770 (376h46m54.681051978s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198049760 (376h46m54.681506963s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198528799 (376h46m54.681986007s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198996489 (376h46m54.682453701s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.199526016 (376h46m54.682983298s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.199772893 (376h46m54.683230102s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200006077 (376h46m54.683463277s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200298977 (376h46m54.68375618s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200764860 (376h46m54.684222068s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.201330795 (376h46m54.68478802s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.201862657 (376h46m54.685319865s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.202435299 (376h46m54.685892502s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.203149033 (376h46m54.686606237s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.288234960 (376h46m54.771692157s) [upload=391707]: Next returned nil segment Jan 31 08:42:01.288251604 (376h46m54.771708818s) [upload=391707]: Waiting for error group managing segments... Jan 31 08:42:01.293400797 (376h46m54.776858002s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.294083673 (376h46m54.777540862s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.294285106 (376h46m54.777742305s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295108197 (376h46m54.778565394s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295172272 (376h46m54.778629467s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295900158 (376h46m54.779357349s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295955977 (376h46m54.779413173s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296392614 (376h46m54.779849814s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296441340 (376h46m54.779898537s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296874674 (376h46m54.780331861s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296909101 (376h46m54.780366298s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.297409404 (376h46m54.780866604s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:31.121919846 (376h47m24.605377048s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: read tcp 159.223.143.39:60086->45.91.116.7:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122001831 (376h47m24.605459038s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: read tcp 159.223.143.39:60218->45.91.116.5:4297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122048345 (376h47m24.605505567s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: read tcp 159.223.143.39:45254->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122106214 (376h47m24.605563415s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12qpELr7Yr1aJtgau8q34qPBtADW6QAXPNySAzcRLB39hXoXueo, address:45.91.116.8:4070): protocol: read tcp 159.223.143.39:50716->45.91.116.8:4070: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122131285 (376h47m24.605588469s) [seg_pos={0 0}, upload=391707]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jan 31 08:42:31.141063957 (376h47m24.624521154s) [upload=391707]: Done uploading segments... Jan 31 08:42:31.141101292 (376h47m24.624558493s) [upload=391707]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 493.257193ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 469.599386ms trace: https://jaeger.dev.storj.io/trace/62a1509ecc5b5058 FAIL (2026-01-31 08:46:27 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Jan 31 08:42:01.166771110 (376h46m54.650228297s) [upload=391707]: Starting upload Jan 31 08:42:01.166986060 (376h46m54.650443258s) [upload=391707]: Done starting upload Jan 31 08:42:01.167072896 (376h46m54.650530094s) [upload=391707]: Uploading segments... Jan 31 08:42:01.167397789 (376h46m54.650854991s) [upload=391707]: Got next segment. Inline: false Jan 31 08:42:01.167473452 (376h46m54.650930657s) [upload=391707]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 31 08:42:01.167480315 (376h46m54.650937524s) [upload=391707]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 31 08:42:01.195346764 (376h46m54.678803969s) [seg_pos={0 0}, upload=391707]: Begin upload segment... Jan 31 08:42:01.195428575 (376h46m54.678885757s) [seg_pos={0 0}, upload=391707]: Done begin upload segment. Jan 31 08:42:01.195463009 (376h46m54.678920214s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.195847779 (376h46m54.679304986s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.197051385 (376h46m54.680508591s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.197594770 (376h46m54.681051978s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198049760 (376h46m54.681506963s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198528799 (376h46m54.681986007s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198996489 (376h46m54.682453701s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.199526016 (376h46m54.682983298s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.199772893 (376h46m54.683230102s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200006077 (376h46m54.683463277s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200298977 (376h46m54.68375618s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200764860 (376h46m54.684222068s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.201330795 (376h46m54.68478802s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.201862657 (376h46m54.685319865s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.202435299 (376h46m54.685892502s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.203149033 (376h46m54.686606237s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.288234960 (376h46m54.771692157s) [upload=391707]: Next returned nil segment Jan 31 08:42:01.288251604 (376h46m54.771708818s) [upload=391707]: Waiting for error group managing segments... Jan 31 08:42:01.293400797 (376h46m54.776858002s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.294083673 (376h46m54.777540862s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.294285106 (376h46m54.777742305s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295108197 (376h46m54.778565394s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295172272 (376h46m54.778629467s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295900158 (376h46m54.779357349s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295955977 (376h46m54.779413173s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296392614 (376h46m54.779849814s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296441340 (376h46m54.779898537s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296874674 (376h46m54.780331861s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296909101 (376h46m54.780366298s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.297409404 (376h46m54.780866604s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:31.121919846 (376h47m24.605377048s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: read tcp 159.223.143.39:60086->45.91.116.7:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122001831 (376h47m24.605459038s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: read tcp 159.223.143.39:60218->45.91.116.5:4297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122048345 (376h47m24.605505567s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: read tcp 159.223.143.39:45254->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122106214 (376h47m24.605563415s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12qpELr7Yr1aJtgau8q34qPBtADW6QAXPNySAzcRLB39hXoXueo, address:45.91.116.8:4070): protocol: read tcp 159.223.143.39:50716->45.91.116.8:4070: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122131285 (376h47m24.605588469s) [seg_pos={0 0}, upload=391707]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jan 31 08:42:31.141063957 (376h47m24.624521154s) [upload=391707]: Done uploading segments... Jan 31 08:42:31.141101292 (376h47m24.624558493s) [upload=391707]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 469.632633ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [......................................................X.....] duration: 425.261936ms trace: https://jaeger.dev.storj.io/trace/6c4cbd1edfce8e6 FAIL (2026-01-31 08:46:19 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 425.290632ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 447.874981ms trace: https://jaeger.dev.storj.io/trace/33d822523a7c81ef FAIL (2026-01-31 08:46:13 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 447.905666ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 461.157801ms trace: https://jaeger.dev.storj.io/trace/33d822523a7c81ef FAIL (2026-01-31 08:45:36 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 461.226843ms edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.......................................................X....] duration: 435.331917ms trace: https://jaeger.dev.storj.io/trace/6c4cbd1edfce8e6 FAIL (2026-01-31 08:45:28 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 435.390202ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 506.081898ms trace: https://jaeger.dev.storj.io/trace/62a1509ecc5b5058 FAIL (2026-01-31 08:45:15 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Jan 31 08:42:01.166771110 (376h46m54.650228297s) [upload=391707]: Starting upload Jan 31 08:42:01.166986060 (376h46m54.650443258s) [upload=391707]: Done starting upload Jan 31 08:42:01.167072896 (376h46m54.650530094s) [upload=391707]: Uploading segments... Jan 31 08:42:01.167397789 (376h46m54.650854991s) [upload=391707]: Got next segment. Inline: false Jan 31 08:42:01.167473452 (376h46m54.650930657s) [upload=391707]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 31 08:42:01.167480315 (376h46m54.650937524s) [upload=391707]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 31 08:42:01.195346764 (376h46m54.678803969s) [seg_pos={0 0}, upload=391707]: Begin upload segment... Jan 31 08:42:01.195428575 (376h46m54.678885757s) [seg_pos={0 0}, upload=391707]: Done begin upload segment. Jan 31 08:42:01.195463009 (376h46m54.678920214s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.195847779 (376h46m54.679304986s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.197051385 (376h46m54.680508591s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.197594770 (376h46m54.681051978s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198049760 (376h46m54.681506963s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198528799 (376h46m54.681986007s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198996489 (376h46m54.682453701s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.199526016 (376h46m54.682983298s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.199772893 (376h46m54.683230102s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200006077 (376h46m54.683463277s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200298977 (376h46m54.68375618s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200764860 (376h46m54.684222068s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.201330795 (376h46m54.68478802s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.201862657 (376h46m54.685319865s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.202435299 (376h46m54.685892502s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.203149033 (376h46m54.686606237s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.288234960 (376h46m54.771692157s) [upload=391707]: Next returned nil segment Jan 31 08:42:01.288251604 (376h46m54.771708818s) [upload=391707]: Waiting for error group managing segments... Jan 31 08:42:01.293400797 (376h46m54.776858002s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.294083673 (376h46m54.777540862s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.294285106 (376h46m54.777742305s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295108197 (376h46m54.778565394s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295172272 (376h46m54.778629467s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295900158 (376h46m54.779357349s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295955977 (376h46m54.779413173s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296392614 (376h46m54.779849814s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296441340 (376h46m54.779898537s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296874674 (376h46m54.780331861s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296909101 (376h46m54.780366298s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.297409404 (376h46m54.780866604s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:31.121919846 (376h47m24.605377048s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: read tcp 159.223.143.39:60086->45.91.116.7:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122001831 (376h47m24.605459038s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: read tcp 159.223.143.39:60218->45.91.116.5:4297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122048345 (376h47m24.605505567s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: read tcp 159.223.143.39:45254->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122106214 (376h47m24.605563415s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12qpELr7Yr1aJtgau8q34qPBtADW6QAXPNySAzcRLB39hXoXueo, address:45.91.116.8:4070): protocol: read tcp 159.223.143.39:50716->45.91.116.8:4070: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122131285 (376h47m24.605588469s) [seg_pos={0 0}, upload=391707]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jan 31 08:42:31.141063957 (376h47m24.624521154s) [upload=391707]: Done uploading segments... Jan 31 08:42:31.141101292 (376h47m24.624558493s) [upload=391707]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 506.11073ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 465.53019ms trace: https://jaeger.dev.storj.io/trace/62a1509ecc5b5058 FAIL (2026-01-31 08:44:51 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Jan 31 08:42:01.166771110 (376h46m54.650228297s) [upload=391707]: Starting upload Jan 31 08:42:01.166986060 (376h46m54.650443258s) [upload=391707]: Done starting upload Jan 31 08:42:01.167072896 (376h46m54.650530094s) [upload=391707]: Uploading segments... Jan 31 08:42:01.167397789 (376h46m54.650854991s) [upload=391707]: Got next segment. Inline: false Jan 31 08:42:01.167473452 (376h46m54.650930657s) [upload=391707]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 31 08:42:01.167480315 (376h46m54.650937524s) [upload=391707]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 31 08:42:01.195346764 (376h46m54.678803969s) [seg_pos={0 0}, upload=391707]: Begin upload segment... Jan 31 08:42:01.195428575 (376h46m54.678885757s) [seg_pos={0 0}, upload=391707]: Done begin upload segment. Jan 31 08:42:01.195463009 (376h46m54.678920214s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.195847779 (376h46m54.679304986s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.197051385 (376h46m54.680508591s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.197594770 (376h46m54.681051978s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198049760 (376h46m54.681506963s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198528799 (376h46m54.681986007s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198996489 (376h46m54.682453701s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.199526016 (376h46m54.682983298s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.199772893 (376h46m54.683230102s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200006077 (376h46m54.683463277s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200298977 (376h46m54.68375618s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200764860 (376h46m54.684222068s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.201330795 (376h46m54.68478802s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.201862657 (376h46m54.685319865s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.202435299 (376h46m54.685892502s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.203149033 (376h46m54.686606237s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.288234960 (376h46m54.771692157s) [upload=391707]: Next returned nil segment Jan 31 08:42:01.288251604 (376h46m54.771708818s) [upload=391707]: Waiting for error group managing segments... Jan 31 08:42:01.293400797 (376h46m54.776858002s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.294083673 (376h46m54.777540862s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.294285106 (376h46m54.777742305s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295108197 (376h46m54.778565394s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295172272 (376h46m54.778629467s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295900158 (376h46m54.779357349s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295955977 (376h46m54.779413173s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296392614 (376h46m54.779849814s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296441340 (376h46m54.779898537s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296874674 (376h46m54.780331861s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296909101 (376h46m54.780366298s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.297409404 (376h46m54.780866604s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:31.121919846 (376h47m24.605377048s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: read tcp 159.223.143.39:60086->45.91.116.7:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122001831 (376h47m24.605459038s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: read tcp 159.223.143.39:60218->45.91.116.5:4297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122048345 (376h47m24.605505567s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: read tcp 159.223.143.39:45254->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122106214 (376h47m24.605563415s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12qpELr7Yr1aJtgau8q34qPBtADW6QAXPNySAzcRLB39hXoXueo, address:45.91.116.8:4070): protocol: read tcp 159.223.143.39:50716->45.91.116.8:4070: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122131285 (376h47m24.605588469s) [seg_pos={0 0}, upload=391707]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jan 31 08:42:31.141063957 (376h47m24.624521154s) [upload=391707]: Done uploading segments... Jan 31 08:42:31.141101292 (376h47m24.624558493s) [upload=391707]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 465.556333ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 477.276385ms trace: https://jaeger.dev.storj.io/trace/33d822523a7c81ef FAIL (2026-01-31 08:44:46 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 477.320965ms edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [........................................................X...] duration: 392.671758ms trace: https://jaeger.dev.storj.io/trace/6c4cbd1edfce8e6 FAIL (2026-01-31 08:44:27 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 392.884275ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 459.527098ms trace: https://jaeger.dev.storj.io/trace/33d822523a7c81ef FAIL (2026-01-31 08:44:11 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 459.548443ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 385.853275ms trace: https://jaeger.dev.storj.io/trace/62a1509ecc5b5058 FAIL (2026-01-31 08:43:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Jan 31 08:42:01.166771110 (376h46m54.650228297s) [upload=391707]: Starting upload Jan 31 08:42:01.166986060 (376h46m54.650443258s) [upload=391707]: Done starting upload Jan 31 08:42:01.167072896 (376h46m54.650530094s) [upload=391707]: Uploading segments... Jan 31 08:42:01.167397789 (376h46m54.650854991s) [upload=391707]: Got next segment. Inline: false Jan 31 08:42:01.167473452 (376h46m54.650930657s) [upload=391707]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 31 08:42:01.167480315 (376h46m54.650937524s) [upload=391707]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 31 08:42:01.195346764 (376h46m54.678803969s) [seg_pos={0 0}, upload=391707]: Begin upload segment... Jan 31 08:42:01.195428575 (376h46m54.678885757s) [seg_pos={0 0}, upload=391707]: Done begin upload segment. Jan 31 08:42:01.195463009 (376h46m54.678920214s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.195847779 (376h46m54.679304986s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.197051385 (376h46m54.680508591s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.197594770 (376h46m54.681051978s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198049760 (376h46m54.681506963s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198528799 (376h46m54.681986007s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198996489 (376h46m54.682453701s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.199526016 (376h46m54.682983298s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.199772893 (376h46m54.683230102s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200006077 (376h46m54.683463277s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200298977 (376h46m54.68375618s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200764860 (376h46m54.684222068s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.201330795 (376h46m54.68478802s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.201862657 (376h46m54.685319865s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.202435299 (376h46m54.685892502s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.203149033 (376h46m54.686606237s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.288234960 (376h46m54.771692157s) [upload=391707]: Next returned nil segment Jan 31 08:42:01.288251604 (376h46m54.771708818s) [upload=391707]: Waiting for error group managing segments... Jan 31 08:42:01.293400797 (376h46m54.776858002s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.294083673 (376h46m54.777540862s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.294285106 (376h46m54.777742305s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295108197 (376h46m54.778565394s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295172272 (376h46m54.778629467s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295900158 (376h46m54.779357349s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295955977 (376h46m54.779413173s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296392614 (376h46m54.779849814s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296441340 (376h46m54.779898537s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296874674 (376h46m54.780331861s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296909101 (376h46m54.780366298s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.297409404 (376h46m54.780866604s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:31.121919846 (376h47m24.605377048s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: read tcp 159.223.143.39:60086->45.91.116.7:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122001831 (376h47m24.605459038s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: read tcp 159.223.143.39:60218->45.91.116.5:4297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122048345 (376h47m24.605505567s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: read tcp 159.223.143.39:45254->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122106214 (376h47m24.605563415s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12qpELr7Yr1aJtgau8q34qPBtADW6QAXPNySAzcRLB39hXoXueo, address:45.91.116.8:4070): protocol: read tcp 159.223.143.39:50716->45.91.116.8:4070: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122131285 (376h47m24.605588469s) [seg_pos={0 0}, upload=391707]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jan 31 08:42:31.141063957 (376h47m24.624521154s) [upload=391707]: Done uploading segments... Jan 31 08:42:31.141101292 (376h47m24.624558493s) [upload=391707]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 385.942999ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.........................................................X..] duration: 410.611166ms trace: https://jaeger.dev.storj.io/trace/6c4cbd1edfce8e6 FAIL (2026-01-31 08:43:32 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 410.669244ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 479.903146ms trace: https://jaeger.dev.storj.io/trace/33d822523a7c81ef FAIL (2026-01-31 08:43:19 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 480.040163ms edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..........................................................X.] duration: 464.109583ms trace: https://jaeger.dev.storj.io/trace/6c4cbd1edfce8e6 FAIL (2026-01-31 08:43:13 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 464.135604ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 504.405835ms trace: https://jaeger.dev.storj.io/trace/62a1509ecc5b5058 FAIL (2026-01-31 08:42:51 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Jan 31 08:42:01.166771110 (376h46m54.650228297s) [upload=391707]: Starting upload Jan 31 08:42:01.166986060 (376h46m54.650443258s) [upload=391707]: Done starting upload Jan 31 08:42:01.167072896 (376h46m54.650530094s) [upload=391707]: Uploading segments... Jan 31 08:42:01.167397789 (376h46m54.650854991s) [upload=391707]: Got next segment. Inline: false Jan 31 08:42:01.167473452 (376h46m54.650930657s) [upload=391707]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 31 08:42:01.167480315 (376h46m54.650937524s) [upload=391707]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 31 08:42:01.195346764 (376h46m54.678803969s) [seg_pos={0 0}, upload=391707]: Begin upload segment... Jan 31 08:42:01.195428575 (376h46m54.678885757s) [seg_pos={0 0}, upload=391707]: Done begin upload segment. Jan 31 08:42:01.195463009 (376h46m54.678920214s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.195847779 (376h46m54.679304986s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.197051385 (376h46m54.680508591s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.197594770 (376h46m54.681051978s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198049760 (376h46m54.681506963s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198528799 (376h46m54.681986007s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198996489 (376h46m54.682453701s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.199526016 (376h46m54.682983298s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.199772893 (376h46m54.683230102s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200006077 (376h46m54.683463277s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200298977 (376h46m54.68375618s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200764860 (376h46m54.684222068s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.201330795 (376h46m54.68478802s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.201862657 (376h46m54.685319865s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.202435299 (376h46m54.685892502s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.203149033 (376h46m54.686606237s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.288234960 (376h46m54.771692157s) [upload=391707]: Next returned nil segment Jan 31 08:42:01.288251604 (376h46m54.771708818s) [upload=391707]: Waiting for error group managing segments... Jan 31 08:42:01.293400797 (376h46m54.776858002s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.294083673 (376h46m54.777540862s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.294285106 (376h46m54.777742305s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295108197 (376h46m54.778565394s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295172272 (376h46m54.778629467s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295900158 (376h46m54.779357349s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295955977 (376h46m54.779413173s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296392614 (376h46m54.779849814s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296441340 (376h46m54.779898537s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296874674 (376h46m54.780331861s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296909101 (376h46m54.780366298s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.297409404 (376h46m54.780866604s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:31.121919846 (376h47m24.605377048s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: read tcp 159.223.143.39:60086->45.91.116.7:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122001831 (376h47m24.605459038s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: read tcp 159.223.143.39:60218->45.91.116.5:4297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122048345 (376h47m24.605505567s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: read tcp 159.223.143.39:45254->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122106214 (376h47m24.605563415s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12qpELr7Yr1aJtgau8q34qPBtADW6QAXPNySAzcRLB39hXoXueo, address:45.91.116.8:4070): protocol: read tcp 159.223.143.39:50716->45.91.116.8:4070: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122131285 (376h47m24.605588469s) [seg_pos={0 0}, upload=391707]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jan 31 08:42:31.141063957 (376h47m24.624521154s) [upload=391707]: Done uploading segments... Jan 31 08:42:31.141101292 (376h47m24.624558493s) [upload=391707]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 504.434481ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.001120046s trace: https://jaeger.dev.storj.io/trace/33d822523a7c81ef FAIL (2026-01-31 08:42:43 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001143735s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 392.313657ms trace: https://jaeger.dev.storj.io/trace/62a1509ecc5b5058 FAIL (2026-01-31 08:42:35 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Jan 31 08:42:01.166771110 (376h46m54.650228297s) [upload=391707]: Starting upload Jan 31 08:42:01.166986060 (376h46m54.650443258s) [upload=391707]: Done starting upload Jan 31 08:42:01.167072896 (376h46m54.650530094s) [upload=391707]: Uploading segments... Jan 31 08:42:01.167397789 (376h46m54.650854991s) [upload=391707]: Got next segment. Inline: false Jan 31 08:42:01.167473452 (376h46m54.650930657s) [upload=391707]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 31 08:42:01.167480315 (376h46m54.650937524s) [upload=391707]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 31 08:42:01.195346764 (376h46m54.678803969s) [seg_pos={0 0}, upload=391707]: Begin upload segment... Jan 31 08:42:01.195428575 (376h46m54.678885757s) [seg_pos={0 0}, upload=391707]: Done begin upload segment. Jan 31 08:42:01.195463009 (376h46m54.678920214s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.195847779 (376h46m54.679304986s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.197051385 (376h46m54.680508591s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.197594770 (376h46m54.681051978s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198049760 (376h46m54.681506963s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198528799 (376h46m54.681986007s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198996489 (376h46m54.682453701s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.199526016 (376h46m54.682983298s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.199772893 (376h46m54.683230102s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200006077 (376h46m54.683463277s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200298977 (376h46m54.68375618s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200764860 (376h46m54.684222068s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.201330795 (376h46m54.68478802s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.201862657 (376h46m54.685319865s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.202435299 (376h46m54.685892502s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.203149033 (376h46m54.686606237s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.288234960 (376h46m54.771692157s) [upload=391707]: Next returned nil segment Jan 31 08:42:01.288251604 (376h46m54.771708818s) [upload=391707]: Waiting for error group managing segments... Jan 31 08:42:01.293400797 (376h46m54.776858002s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.294083673 (376h46m54.777540862s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.294285106 (376h46m54.777742305s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295108197 (376h46m54.778565394s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295172272 (376h46m54.778629467s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295900158 (376h46m54.779357349s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295955977 (376h46m54.779413173s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296392614 (376h46m54.779849814s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296441340 (376h46m54.779898537s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296874674 (376h46m54.780331861s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296909101 (376h46m54.780366298s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.297409404 (376h46m54.780866604s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:31.121919846 (376h47m24.605377048s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: read tcp 159.223.143.39:60086->45.91.116.7:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122001831 (376h47m24.605459038s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: read tcp 159.223.143.39:60218->45.91.116.5:4297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122048345 (376h47m24.605505567s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: read tcp 159.223.143.39:45254->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122106214 (376h47m24.605563415s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12qpELr7Yr1aJtgau8q34qPBtADW6QAXPNySAzcRLB39hXoXueo, address:45.91.116.8:4070): protocol: read tcp 159.223.143.39:50716->45.91.116.8:4070: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122131285 (376h47m24.605588469s) [seg_pos={0 0}, upload=391707]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jan 31 08:42:31.141063957 (376h47m24.624521154s) [upload=391707]: Done uploading segments... Jan 31 08:42:31.141101292 (376h47m24.624558493s) [upload=391707]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 392.342448ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.021677124s trace: https://jaeger.dev.storj.io/trace/62a1509ecc5b5058 FAIL (2026-01-31 08:42:31 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Jan 31 08:42:01.166771110 (376h46m54.650228297s) [upload=391707]: Starting upload Jan 31 08:42:01.166986060 (376h46m54.650443258s) [upload=391707]: Done starting upload Jan 31 08:42:01.167072896 (376h46m54.650530094s) [upload=391707]: Uploading segments... Jan 31 08:42:01.167397789 (376h46m54.650854991s) [upload=391707]: Got next segment. Inline: false Jan 31 08:42:01.167473452 (376h46m54.650930657s) [upload=391707]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 31 08:42:01.167480315 (376h46m54.650937524s) [upload=391707]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 31 08:42:01.195346764 (376h46m54.678803969s) [seg_pos={0 0}, upload=391707]: Begin upload segment... Jan 31 08:42:01.195428575 (376h46m54.678885757s) [seg_pos={0 0}, upload=391707]: Done begin upload segment. Jan 31 08:42:01.195463009 (376h46m54.678920214s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.195847779 (376h46m54.679304986s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.197051385 (376h46m54.680508591s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.197594770 (376h46m54.681051978s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198049760 (376h46m54.681506963s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198528799 (376h46m54.681986007s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.198996489 (376h46m54.682453701s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.199526016 (376h46m54.682983298s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.199772893 (376h46m54.683230102s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200006077 (376h46m54.683463277s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200298977 (376h46m54.68375618s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.200764860 (376h46m54.684222068s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.201330795 (376h46m54.68478802s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.201862657 (376h46m54.685319865s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.202435299 (376h46m54.685892502s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.203149033 (376h46m54.686606237s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Uploading piece... Jan 31 08:42:01.288234960 (376h46m54.771692157s) [upload=391707]: Next returned nil segment Jan 31 08:42:01.288251604 (376h46m54.771708818s) [upload=391707]: Waiting for error group managing segments... Jan 31 08:42:01.293400797 (376h46m54.776858002s) [noise=true , address=77.90.31.19:4844 , piece_id=OOSM5KL3DLPGXCQ4UI7JIF7H3CZ2TNRN35FUXBD2N3F4D4DB44HQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.294083673 (376h46m54.777540862s) [noise=true , address=77.90.31.14:4663 , piece_id=TSJWQMUOQXG62R2IX5XMS2X7QWK56DZXIAWSWLBFVXSSWZAW7CTQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.294285106 (376h46m54.777742305s) [noise=true , address=145.79.227.47:4265 , piece_id=Y7NQBPEPRYTHMGIXXFPLYVQ6OQCVAXMWJPRIVCSFAFSTCIRCLK7A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295108197 (376h46m54.778565394s) [noise=true , address=145.79.227.41:4087 , piece_id=YS5JMZLEXXL6O33YEZSZNH4R4W6QACS3XL5P3THSTNYNQPMJYBOA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295172272 (376h46m54.778629467s) [noise=true , address=145.79.227.35:4482 , piece_id=3KIO5DQKH5FKM22JG4JEOEFT45H3XHHNX6SVAB56NMQC43P4HJ7Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295900158 (376h46m54.779357349s) [noise=true , address=145.79.227.30:4761 , piece_id=WJDRFHX52HPXTRKWDTY7OBNOW7SDJTHWQHFOSVL7YYTGPBRZNHBQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.295955977 (376h46m54.779413173s) [noise=true , address=145.79.227.48:4302 , piece_id=MWXXVBXXQKZ5E5M4V5DC2S2KGL4ARRUREQERPH47PUPC6Z7SJW6A, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296392614 (376h46m54.779849814s) [noise=true , address=145.79.227.36:4517 , piece_id=6EUEWRVVKLSBBVLVY6S33D4QLDCYD455QP4GN3RGIA65CO7J5TVA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296441340 (376h46m54.779898537s) [noise=true , address=145.79.227.34:4906 , piece_id=E3MAVCWQIVD7Q5MOPAKAZ7FFYQ4LHLJHU5XUA3QMQTCQZ3CUWDYQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296874674 (376h46m54.780331861s) [noise=true , address=145.79.227.40:4049 , piece_id=TCQX7KJYEGI3NRLW3UWZNSRMEILBIHGN2ABW64TRMN6U7XMC36FQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.296909101 (376h46m54.780366298s) [noise=true , address=145.79.227.43:4158 , piece_id=SHVKLK6QZB7LM3H2EEUNDYC25NASIJAU7FENSP6JW7ZP56GG6EDQ, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:01.297409404 (376h46m54.780866604s) [noise=true , address=145.79.227.37:4566 , piece_id=MZG2PRH45I62HWBUQDKZHMDT4XE4BFOBW66TTXVEAA2W52CJYX4Q, seg_pos={0 0}, upload=391707]: Done uploading piece. err: <nil> Jan 31 08:42:31.121919846 (376h47m24.605377048s) [noise=true , address=45.91.116.7:4012 , piece_id=ALZYURAGTVTA4IZM24T53QUMMLALXAMO7O424EDM53BX3RNI33NA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: read tcp 159.223.143.39:60086->45.91.116.7:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122001831 (376h47m24.605459038s) [noise=true , address=45.91.116.5:4297 , piece_id=R4SR4HHRWNHXI7NGGI2VJRSTMXYQHOK7KC6CKECL3C2J5OMQC3RA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: read tcp 159.223.143.39:60218->45.91.116.5:4297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122048345 (376h47m24.605505567s) [noise=true , address=45.91.116.10:4083 , piece_id=OKPB7UVIY3SCHHNV323ZTHZVY4GT4HUOINPDI5NUNMAMWG4NQAAA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: read tcp 159.223.143.39:45254->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122106214 (376h47m24.605563415s) [noise=true , address=45.91.116.8:4070 , piece_id=QKO4TG426T3Z5IHURJXGRCYOXLARE66VETAEIJPJ4X6XC35WV4VA, seg_pos={0 0}, upload=391707]: Done uploading piece. err: ecclient: upload failed (node:12qpELr7Yr1aJtgau8q34qPBtADW6QAXPNySAzcRLB39hXoXueo, address:45.91.116.8:4070): protocol: read tcp 159.223.143.39:50716->45.91.116.8:4070: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 31 08:42:31.122131285 (376h47m24.605588469s) [seg_pos={0 0}, upload=391707]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jan 31 08:42:31.141063957 (376h47m24.624521154s) [upload=391707]: Done uploading segments... Jan 31 08:42:31.141101292 (376h47m24.624558493s) [upload=391707]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.021704636s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...........................................................X] duration: 5.00087511s trace: https://jaeger.dev.storj.io/trace/6c4cbd1edfce8e6 FAIL (2026-01-31 08:42:16 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.000921549s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 313.031901ms remote: "tcp:136.0.77.3:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011 trace: https://jaeger.dev.storj.io/trace/6bb500136786b111 FAIL (2026-01-30 17:11:55 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011": context deadline exceeded Duration: 313.058916ms edge gateway vhost bgp ap1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 313.177525ms remote: "tcp:136.0.77.3:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011 trace: https://jaeger.dev.storj.io/trace/6bb500136786b111 FAIL (2026-01-30 17:11:21 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011": context deadline exceeded Duration: 313.234839ms edge gateway vhost bgp ap1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 270.310849ms remote: "tcp:136.0.77.3:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011 trace: https://jaeger.dev.storj.io/trace/6bb500136786b111 FAIL (2026-01-30 17:10:51 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011": context deadline exceeded Duration: 270.393796ms edge gateway vhost bgp ap1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 873.121614ms remote: "tcp:136.0.77.3:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011 trace: https://jaeger.dev.storj.io/trace/6bb500136786b111 FAIL (2026-01-30 17:10:00 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011": context deadline exceeded Duration: 873.152033ms edge gateway vhost bgp ap1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 316.202745ms remote: "tcp:136.0.77.3:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011 trace: https://jaeger.dev.storj.io/trace/6bb500136786b111 FAIL (2026-01-30 17:09:25 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011": context deadline exceeded Duration: 316.226284ms edge gateway vhost bgp ap1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 305.140191ms remote: "tcp:136.0.77.3:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011 trace: https://jaeger.dev.storj.io/trace/6bb500136786b111 FAIL (2026-01-30 17:08:22 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011": context deadline exceeded Duration: 305.169133ms edge gateway vhost bgp ap1

1
...
79
80
81
82
83
...
3556