Orbiter History

checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 5 out of 60 is failed (threshold=6) [.............X..X.........XXX...............................] duration: 415.257878ms trace: https://jaeger.dev.storj.io/trace/3304a822f59853ad FAIL (2026-04-04 04:46:45 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 415.309891ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 1.227843777s trace: https://jaeger.dev.storj.io/trace/5cce09de575b8506 FAIL (2026-04-04 04:46:37 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 6881916s) [upload=772064]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 4 04:21:44.460130782 (689h21m42.056901815s) [upload=772064]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 4 04:21:44.500551580 (689h21m42.097322604s) [seg_pos={0 0}, upload=772064]: Begin upload segment... Apr 4 04:21:44.500628241 (689h21m42.097399263s) [seg_pos={0 0}, upload=772064]: Done begin upload segment. Apr 4 04:21:44.500668892 (689h21m42.097439922s) [noise=true , address=193.149.164.143:4704 , piece_id=PZ2W2J22M5MZIYBFEOJCEOIVAJOJGYCHCQUPDSELS764HZG4OF2A, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.500792978 (689h21m42.097564014s) [noise=true , address=45.91.116.13:4158 , piece_id=OX4BJEH3DNMAUXS64QHDHRK75YYEF6DPQJAX7MS3J6K7RUM4GM2Q, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.501687053 (689h21m42.09845809s) [noise=true , address=77.90.31.17:4783 , piece_id=E2XPGXQAVSYEMBWOIPKHGL5MVVGC2FSLN5KHZQC6DBUAT3VUWOKA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.501828492 (689h21m42.098599515s) [noise=true , address=193.149.164.69:4084 , piece_id=BUV3LZ7PDHSGRNKHJSSN2OFQYMIYUELWEPO4PF3DMTBX65S4YPMA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.502343164 (689h21m42.099114198s) [noise=true , address=193.149.164.67:4002 , piece_id=CYJYJ5ZNXT63OT5DBDLW7ML32OIZLGNWMLM4W2F5TPSSJ2QJ2NBA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.502900735 (689h21m42.099671764s) [noise=true , address=77.90.31.18:4823 , piece_id=MI45FJCGXJ2CIL4OOVHEUB7TJXEGH6WELCSTKR6JD753YLF5UN4Q, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.503495842 (689h21m42.100266868s) [noise=true , address=193.149.164.68:4062 , piece_id=IM6B4U2ARVUSBX4276OSQPEPQNMWMFSUUOYAEJ3ZG2GLFUKGGQGQ, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.503796813 (689h21m42.100567839s) [noise=true , address=77.90.31.13:4658 , piece_id=I7GGGCPJIHSQ5RYYOPIXKWAJBOMSRSK3ZDCASMKUPBQPALOJCKXA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.504006258 (689h21m42.100777287s) [noise=true , address=77.90.31.16:4750 , piece_id=TI2D4Z2MYSXWBHX3ZVEMRFLI4LW5RDVCENH43S7GEFR7WGIVCI6A, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.504542000 (689h21m42.101313035s) [noise=true , address=145.79.227.33:4898 , piece_id=64X2NB2BPWLP4C5EC53A6EKFPNQCJI6BGEDY6BFE2RFNYEPCDA5Q, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.504560079 (689h21m42.101331115s) [noise=true , address=45.91.116.12:4229 , piece_id=CDXY4QA2BKVNNJUHGL2JPK3MMHTVDPRN64SAQSMZSVQIG755ZOJQ, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.505047220 (689h21m42.101818243s) [noise=true , address=193.149.164.66:4996 , piece_id=YGOQ6INTPJ3AZW3RSTVH2IWITNFONA7JTW25BIXNBZRRZUN6YRYA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.505183875 (689h21m42.10195491s) [noise=true , address=167.17.72.70:4751 , piece_id=AX7C5IEHMVNPGGAB7WPWUOSFEHVWWA75JGDAH3DAGZVZCVMEZKKQ, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.505619570 (689h21m42.102390603s) [noise=true , address=167.17.72.68:4656 , piece_id=YFAXDPM4TYBLYL4UR3Z6FDCDJXU34J7XRURDGLXZAMUOYYZL56IA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.506074686 (689h21m42.102845722s) [noise=true , address=193.149.164.70:4115 , piece_id=EHMHJHJ2ZQWAJWBFSHOTAXWE7QJQOIICXDKAK33KMOAF7C7KZ2PA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.506877697 (689h21m42.103648727s) [noise=true , address=145.79.227.8:4345 , piece_id=2UWIKM3L3JUKNYWPBVJBEGFOEWS3WVM24AY7C2OVBZWDAXDG4I3A, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.507462874 (689h21m42.104233903s) [noise=true , address=193.149.164.71:4308 , piece_id=YGF5F7J25BEULKSTR3ZDC3HE4EHVJ3IF4HQCNATVWHITGRB344JA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.507975323 (689h21m42.10474639s) [noise=true , address=77.90.31.15:4709 , piece_id=HQLXOXRSN7R5WC5K2W7X4YUBVEZFITOY4VEYKG7J6WIJDEHABWGA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.582999887 (689h21m42.17977091s) [upload=772064]: Next returned nil segment Apr 4 04:21:44.583065715 (689h21m42.179836738s) [upload=772064]: Waiting for error group managing segments... Apr 4 04:21:44.587179313 (689h21m42.183950339s) [noise=true , address=45.91.116.12:4229 , piece_id=CDXY4QA2BKVNNJUHGL2JPK3MMHTVDPRN64SAQSMZSVQIG755ZOJQ, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:44.587315638 (689h21m42.184086655s) [noise=true , address=45.91.116.13:4158 , piece_id=OX4BJEH3DNMAUXS64QHDHRK75YYEF6DPQJAX7MS3J6K7RUM4GM2Q, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:44.592227274 (689h21m42.188998305s) [noise=true , address=145.79.227.8:4345 , piece_id=2UWIKM3L3JUKNYWPBVJBEGFOEWS3WVM24AY7C2OVBZWDAXDG4I3A, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:44.897574490 (689h21m42.494345523s) [noise=true , address=167.17.72.70:4751 , piece_id=AX7C5IEHMVNPGGAB7WPWUOSFEHVWWA75JGDAH3DAGZVZCVMEZKKQ, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.084711646 (689h21m42.681482669s) [noise=true , address=193.149.164.66:4996 , piece_id=YGOQ6INTPJ3AZW3RSTVH2IWITNFONA7JTW25BIXNBZRRZUN6YRYA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.084884121 (689h21m42.681655134s) [noise=true , address=193.149.164.67:4002 , piece_id=CYJYJ5ZNXT63OT5DBDLW7ML32OIZLGNWMLM4W2F5TPSSJ2QJ2NBA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.089152844 (689h21m42.685923867s) [noise=true , address=193.149.164.70:4115 , piece_id=EHMHJHJ2ZQWAJWBFSHOTAXWE7QJQOIICXDKAK33KMOAF7C7KZ2PA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.103385369 (689h21m42.700156382s) [noise=true , address=193.149.164.68:4062 , piece_id=IM6B4U2ARVUSBX4276OSQPEPQNMWMFSUUOYAEJ3ZG2GLFUKGGQGQ, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.127711185 (689h21m42.724482213s) [noise=true , address=193.149.164.69:4084 , piece_id=BUV3LZ7PDHSGRNKHJSSN2OFQYMIYUELWEPO4PF3DMTBX65S4YPMA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.143513369 (689h21m42.740284381s) [noise=true , address=193.149.164.71:4308 , piece_id=YGF5F7J25BEULKSTR3ZDC3HE4EHVJ3IF4HQCNATVWHITGRB344JA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.307124646 (689h21m42.903895674s) [noise=true , address=167.17.72.68:4656 , piece_id=YFAXDPM4TYBLYL4UR3Z6FDCDJXU34J7XRURDGLXZAMUOYYZL56IA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.517599945 (689h21m43.114370966s) [noise=true , address=145.79.227.33:4898 , piece_id=64X2NB2BPWLP4C5EC53A6EKFPNQCJI6BGEDY6BFE2RFNYEPCDA5Q, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:46.061904485 (689h21m43.658675509s) [noise=true , address=193.149.164.143:4704 , piece_id=PZ2W2J22M5MZIYBFEOJCEOIVAJOJGYCHCQUPDSELS764HZG4OF2A, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:22:44.423237599 (689h22m42.020008631s) [noise=true , address=77.90.31.13:4658 , piece_id=I7GGGCPJIHSQ5RYYOPIXKWAJBOMSRSK3ZDCASMKUPBQPALOJCKXA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: ecclient: upload failed (node:129aongkQE9HMNz8vsNNA5SvKbhsHqdt9UrtSTPJJpP9mkFsboX, address:77.90.31.13:4658): protocol: read tcp 159.223.143.39:50834->77.90.31.13:4658: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 4 04:22:44.423364101 (689h22m42.020135144s) [noise=true , address=77.90.31.17:4783 , piece_id=E2XPGXQAVSYEMBWOIPKHGL5MVVGC2FSLN5KHZQC6DBUAT3VUWOKA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: ecclient: upload failed (node:155fvDXZdpQeHetitv17WCVVrP1bGswdoqJECQLSe66kXRhBPo, address:77.90.31.17:4783): protocol: read tcp 159.223.143.39:45990->77.90.31.17:4783: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 4 04:22:44.423435607 (689h22m42.020206643s) [noise=true , address=77.90.31.16:4750 , piece_id=TI2D4Z2MYSXWBHX3ZVEMRFLI4LW5RDVCENH43S7GEFR7WGIVCI6A, seg_pos={0 0}, upload=772064]: Done uploading piece. err: ecclient: upload failed (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE, address:77.90.31.16:4750): protocol: read tcp 159.223.143.39:34564->77.90.31.16:4750: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 4 04:22:44.423560856 (689h22m42.020332015s) [noise=true , address=77.90.31.18:4823 , piece_id=MI45FJCGXJ2CIL4OOVHEUB7TJXEGH6WELCSTKR6JD753YLF5UN4Q, seg_pos={0 0}, upload=772064]: Done uploading piece. err: ecclient: upload failed (node:12uUCUDKZDwj25UcEKQ4WiTuQFMRbWw2oymGAdUdMX8wcUSd7jS, address:77.90.31.18:4823): protocol: read tcp 159.223.143.39:40028->77.90.31.18:4823: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 4 04:22:44.423793037 (689h22m42.020564071s) [noise=true , address=77.90.31.15:4709 , piece_id=HQLXOXRSN7R5WC5K2W7X4YUBVEZFITOY4VEYKG7J6WIJDEHABWGA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: ecclient: upload failed (node:1CrHsLV4vge154tJjMvbo5oxnphfukuXQBKfvTjA43n6CJM2Ah, address:77.90.31.15:4709): protocol: read tcp 159.223.143.39:59234->77.90.31.15:4709: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 4 04:22:44.423864095 (689h22m42.020635106s) [seg_pos={0 0}, upload=772064]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 4 04:22:44.451342916 (689h22m42.048113939s) [upload=772064]: Done uploading segments... Apr 4 04:22:44.451381831 (689h22m42.048152855s) [upload=772064]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1.227895541s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 591.512894ms trace: https://jaeger.dev.storj.io/trace/670ee369f5f2d334 FAIL (2026-04-04 04:46:24 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 591.545712ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 5 out of 60 is failed (threshold=6) [..............X..X.........XXX..............................] duration: 484.532484ms trace: https://jaeger.dev.storj.io/trace/3304a822f59853ad FAIL (2026-04-04 04:46:09 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 484.561153ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 1.279172884s trace: https://jaeger.dev.storj.io/trace/5cce09de575b8506 FAIL (2026-04-04 04:45:31 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 6881916s) [upload=772064]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 4 04:21:44.460130782 (689h21m42.056901815s) [upload=772064]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 4 04:21:44.500551580 (689h21m42.097322604s) [seg_pos={0 0}, upload=772064]: Begin upload segment... Apr 4 04:21:44.500628241 (689h21m42.097399263s) [seg_pos={0 0}, upload=772064]: Done begin upload segment. Apr 4 04:21:44.500668892 (689h21m42.097439922s) [noise=true , address=193.149.164.143:4704 , piece_id=PZ2W2J22M5MZIYBFEOJCEOIVAJOJGYCHCQUPDSELS764HZG4OF2A, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.500792978 (689h21m42.097564014s) [noise=true , address=45.91.116.13:4158 , piece_id=OX4BJEH3DNMAUXS64QHDHRK75YYEF6DPQJAX7MS3J6K7RUM4GM2Q, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.501687053 (689h21m42.09845809s) [noise=true , address=77.90.31.17:4783 , piece_id=E2XPGXQAVSYEMBWOIPKHGL5MVVGC2FSLN5KHZQC6DBUAT3VUWOKA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.501828492 (689h21m42.098599515s) [noise=true , address=193.149.164.69:4084 , piece_id=BUV3LZ7PDHSGRNKHJSSN2OFQYMIYUELWEPO4PF3DMTBX65S4YPMA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.502343164 (689h21m42.099114198s) [noise=true , address=193.149.164.67:4002 , piece_id=CYJYJ5ZNXT63OT5DBDLW7ML32OIZLGNWMLM4W2F5TPSSJ2QJ2NBA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.502900735 (689h21m42.099671764s) [noise=true , address=77.90.31.18:4823 , piece_id=MI45FJCGXJ2CIL4OOVHEUB7TJXEGH6WELCSTKR6JD753YLF5UN4Q, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.503495842 (689h21m42.100266868s) [noise=true , address=193.149.164.68:4062 , piece_id=IM6B4U2ARVUSBX4276OSQPEPQNMWMFSUUOYAEJ3ZG2GLFUKGGQGQ, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.503796813 (689h21m42.100567839s) [noise=true , address=77.90.31.13:4658 , piece_id=I7GGGCPJIHSQ5RYYOPIXKWAJBOMSRSK3ZDCASMKUPBQPALOJCKXA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.504006258 (689h21m42.100777287s) [noise=true , address=77.90.31.16:4750 , piece_id=TI2D4Z2MYSXWBHX3ZVEMRFLI4LW5RDVCENH43S7GEFR7WGIVCI6A, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.504542000 (689h21m42.101313035s) [noise=true , address=145.79.227.33:4898 , piece_id=64X2NB2BPWLP4C5EC53A6EKFPNQCJI6BGEDY6BFE2RFNYEPCDA5Q, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.504560079 (689h21m42.101331115s) [noise=true , address=45.91.116.12:4229 , piece_id=CDXY4QA2BKVNNJUHGL2JPK3MMHTVDPRN64SAQSMZSVQIG755ZOJQ, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.505047220 (689h21m42.101818243s) [noise=true , address=193.149.164.66:4996 , piece_id=YGOQ6INTPJ3AZW3RSTVH2IWITNFONA7JTW25BIXNBZRRZUN6YRYA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.505183875 (689h21m42.10195491s) [noise=true , address=167.17.72.70:4751 , piece_id=AX7C5IEHMVNPGGAB7WPWUOSFEHVWWA75JGDAH3DAGZVZCVMEZKKQ, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.505619570 (689h21m42.102390603s) [noise=true , address=167.17.72.68:4656 , piece_id=YFAXDPM4TYBLYL4UR3Z6FDCDJXU34J7XRURDGLXZAMUOYYZL56IA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.506074686 (689h21m42.102845722s) [noise=true , address=193.149.164.70:4115 , piece_id=EHMHJHJ2ZQWAJWBFSHOTAXWE7QJQOIICXDKAK33KMOAF7C7KZ2PA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.506877697 (689h21m42.103648727s) [noise=true , address=145.79.227.8:4345 , piece_id=2UWIKM3L3JUKNYWPBVJBEGFOEWS3WVM24AY7C2OVBZWDAXDG4I3A, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.507462874 (689h21m42.104233903s) [noise=true , address=193.149.164.71:4308 , piece_id=YGF5F7J25BEULKSTR3ZDC3HE4EHVJ3IF4HQCNATVWHITGRB344JA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.507975323 (689h21m42.10474639s) [noise=true , address=77.90.31.15:4709 , piece_id=HQLXOXRSN7R5WC5K2W7X4YUBVEZFITOY4VEYKG7J6WIJDEHABWGA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.582999887 (689h21m42.17977091s) [upload=772064]: Next returned nil segment Apr 4 04:21:44.583065715 (689h21m42.179836738s) [upload=772064]: Waiting for error group managing segments... Apr 4 04:21:44.587179313 (689h21m42.183950339s) [noise=true , address=45.91.116.12:4229 , piece_id=CDXY4QA2BKVNNJUHGL2JPK3MMHTVDPRN64SAQSMZSVQIG755ZOJQ, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:44.587315638 (689h21m42.184086655s) [noise=true , address=45.91.116.13:4158 , piece_id=OX4BJEH3DNMAUXS64QHDHRK75YYEF6DPQJAX7MS3J6K7RUM4GM2Q, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:44.592227274 (689h21m42.188998305s) [noise=true , address=145.79.227.8:4345 , piece_id=2UWIKM3L3JUKNYWPBVJBEGFOEWS3WVM24AY7C2OVBZWDAXDG4I3A, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:44.897574490 (689h21m42.494345523s) [noise=true , address=167.17.72.70:4751 , piece_id=AX7C5IEHMVNPGGAB7WPWUOSFEHVWWA75JGDAH3DAGZVZCVMEZKKQ, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.084711646 (689h21m42.681482669s) [noise=true , address=193.149.164.66:4996 , piece_id=YGOQ6INTPJ3AZW3RSTVH2IWITNFONA7JTW25BIXNBZRRZUN6YRYA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.084884121 (689h21m42.681655134s) [noise=true , address=193.149.164.67:4002 , piece_id=CYJYJ5ZNXT63OT5DBDLW7ML32OIZLGNWMLM4W2F5TPSSJ2QJ2NBA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.089152844 (689h21m42.685923867s) [noise=true , address=193.149.164.70:4115 , piece_id=EHMHJHJ2ZQWAJWBFSHOTAXWE7QJQOIICXDKAK33KMOAF7C7KZ2PA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.103385369 (689h21m42.700156382s) [noise=true , address=193.149.164.68:4062 , piece_id=IM6B4U2ARVUSBX4276OSQPEPQNMWMFSUUOYAEJ3ZG2GLFUKGGQGQ, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.127711185 (689h21m42.724482213s) [noise=true , address=193.149.164.69:4084 , piece_id=BUV3LZ7PDHSGRNKHJSSN2OFQYMIYUELWEPO4PF3DMTBX65S4YPMA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.143513369 (689h21m42.740284381s) [noise=true , address=193.149.164.71:4308 , piece_id=YGF5F7J25BEULKSTR3ZDC3HE4EHVJ3IF4HQCNATVWHITGRB344JA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.307124646 (689h21m42.903895674s) [noise=true , address=167.17.72.68:4656 , piece_id=YFAXDPM4TYBLYL4UR3Z6FDCDJXU34J7XRURDGLXZAMUOYYZL56IA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.517599945 (689h21m43.114370966s) [noise=true , address=145.79.227.33:4898 , piece_id=64X2NB2BPWLP4C5EC53A6EKFPNQCJI6BGEDY6BFE2RFNYEPCDA5Q, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:46.061904485 (689h21m43.658675509s) [noise=true , address=193.149.164.143:4704 , piece_id=PZ2W2J22M5MZIYBFEOJCEOIVAJOJGYCHCQUPDSELS764HZG4OF2A, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:22:44.423237599 (689h22m42.020008631s) [noise=true , address=77.90.31.13:4658 , piece_id=I7GGGCPJIHSQ5RYYOPIXKWAJBOMSRSK3ZDCASMKUPBQPALOJCKXA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: ecclient: upload failed (node:129aongkQE9HMNz8vsNNA5SvKbhsHqdt9UrtSTPJJpP9mkFsboX, address:77.90.31.13:4658): protocol: read tcp 159.223.143.39:50834->77.90.31.13:4658: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 4 04:22:44.423364101 (689h22m42.020135144s) [noise=true , address=77.90.31.17:4783 , piece_id=E2XPGXQAVSYEMBWOIPKHGL5MVVGC2FSLN5KHZQC6DBUAT3VUWOKA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: ecclient: upload failed (node:155fvDXZdpQeHetitv17WCVVrP1bGswdoqJECQLSe66kXRhBPo, address:77.90.31.17:4783): protocol: read tcp 159.223.143.39:45990->77.90.31.17:4783: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 4 04:22:44.423435607 (689h22m42.020206643s) [noise=true , address=77.90.31.16:4750 , piece_id=TI2D4Z2MYSXWBHX3ZVEMRFLI4LW5RDVCENH43S7GEFR7WGIVCI6A, seg_pos={0 0}, upload=772064]: Done uploading piece. err: ecclient: upload failed (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE, address:77.90.31.16:4750): protocol: read tcp 159.223.143.39:34564->77.90.31.16:4750: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 4 04:22:44.423560856 (689h22m42.020332015s) [noise=true , address=77.90.31.18:4823 , piece_id=MI45FJCGXJ2CIL4OOVHEUB7TJXEGH6WELCSTKR6JD753YLF5UN4Q, seg_pos={0 0}, upload=772064]: Done uploading piece. err: ecclient: upload failed (node:12uUCUDKZDwj25UcEKQ4WiTuQFMRbWw2oymGAdUdMX8wcUSd7jS, address:77.90.31.18:4823): protocol: read tcp 159.223.143.39:40028->77.90.31.18:4823: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 4 04:22:44.423793037 (689h22m42.020564071s) [noise=true , address=77.90.31.15:4709 , piece_id=HQLXOXRSN7R5WC5K2W7X4YUBVEZFITOY4VEYKG7J6WIJDEHABWGA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: ecclient: upload failed (node:1CrHsLV4vge154tJjMvbo5oxnphfukuXQBKfvTjA43n6CJM2Ah, address:77.90.31.15:4709): protocol: read tcp 159.223.143.39:59234->77.90.31.15:4709: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 4 04:22:44.423864095 (689h22m42.020635106s) [seg_pos={0 0}, upload=772064]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 4 04:22:44.451342916 (689h22m42.048113939s) [upload=772064]: Done uploading segments... Apr 4 04:22:44.451381831 (689h22m42.048152855s) [upload=772064]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1.279211407s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 579.90132ms trace: https://jaeger.dev.storj.io/trace/670ee369f5f2d334 FAIL (2026-04-04 04:45:12 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 579.955735ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 5 out of 60 is failed (threshold=6) [...............X..X.........XXX.............................] duration: 583.749265ms trace: https://jaeger.dev.storj.io/trace/3304a822f59853ad FAIL (2026-04-04 04:45:10 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 583.778362ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 774.181078ms trace: https://jaeger.dev.storj.io/trace/670ee369f5f2d334 FAIL (2026-04-04 04:44:47 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 774.294651ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 1.279704698s trace: https://jaeger.dev.storj.io/trace/5cce09de575b8506 FAIL (2026-04-04 04:44:37 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 6881916s) [upload=772064]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 4 04:21:44.460130782 (689h21m42.056901815s) [upload=772064]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 4 04:21:44.500551580 (689h21m42.097322604s) [seg_pos={0 0}, upload=772064]: Begin upload segment... Apr 4 04:21:44.500628241 (689h21m42.097399263s) [seg_pos={0 0}, upload=772064]: Done begin upload segment. Apr 4 04:21:44.500668892 (689h21m42.097439922s) [noise=true , address=193.149.164.143:4704 , piece_id=PZ2W2J22M5MZIYBFEOJCEOIVAJOJGYCHCQUPDSELS764HZG4OF2A, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.500792978 (689h21m42.097564014s) [noise=true , address=45.91.116.13:4158 , piece_id=OX4BJEH3DNMAUXS64QHDHRK75YYEF6DPQJAX7MS3J6K7RUM4GM2Q, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.501687053 (689h21m42.09845809s) [noise=true , address=77.90.31.17:4783 , piece_id=E2XPGXQAVSYEMBWOIPKHGL5MVVGC2FSLN5KHZQC6DBUAT3VUWOKA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.501828492 (689h21m42.098599515s) [noise=true , address=193.149.164.69:4084 , piece_id=BUV3LZ7PDHSGRNKHJSSN2OFQYMIYUELWEPO4PF3DMTBX65S4YPMA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.502343164 (689h21m42.099114198s) [noise=true , address=193.149.164.67:4002 , piece_id=CYJYJ5ZNXT63OT5DBDLW7ML32OIZLGNWMLM4W2F5TPSSJ2QJ2NBA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.502900735 (689h21m42.099671764s) [noise=true , address=77.90.31.18:4823 , piece_id=MI45FJCGXJ2CIL4OOVHEUB7TJXEGH6WELCSTKR6JD753YLF5UN4Q, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.503495842 (689h21m42.100266868s) [noise=true , address=193.149.164.68:4062 , piece_id=IM6B4U2ARVUSBX4276OSQPEPQNMWMFSUUOYAEJ3ZG2GLFUKGGQGQ, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.503796813 (689h21m42.100567839s) [noise=true , address=77.90.31.13:4658 , piece_id=I7GGGCPJIHSQ5RYYOPIXKWAJBOMSRSK3ZDCASMKUPBQPALOJCKXA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.504006258 (689h21m42.100777287s) [noise=true , address=77.90.31.16:4750 , piece_id=TI2D4Z2MYSXWBHX3ZVEMRFLI4LW5RDVCENH43S7GEFR7WGIVCI6A, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.504542000 (689h21m42.101313035s) [noise=true , address=145.79.227.33:4898 , piece_id=64X2NB2BPWLP4C5EC53A6EKFPNQCJI6BGEDY6BFE2RFNYEPCDA5Q, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.504560079 (689h21m42.101331115s) [noise=true , address=45.91.116.12:4229 , piece_id=CDXY4QA2BKVNNJUHGL2JPK3MMHTVDPRN64SAQSMZSVQIG755ZOJQ, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.505047220 (689h21m42.101818243s) [noise=true , address=193.149.164.66:4996 , piece_id=YGOQ6INTPJ3AZW3RSTVH2IWITNFONA7JTW25BIXNBZRRZUN6YRYA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.505183875 (689h21m42.10195491s) [noise=true , address=167.17.72.70:4751 , piece_id=AX7C5IEHMVNPGGAB7WPWUOSFEHVWWA75JGDAH3DAGZVZCVMEZKKQ, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.505619570 (689h21m42.102390603s) [noise=true , address=167.17.72.68:4656 , piece_id=YFAXDPM4TYBLYL4UR3Z6FDCDJXU34J7XRURDGLXZAMUOYYZL56IA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.506074686 (689h21m42.102845722s) [noise=true , address=193.149.164.70:4115 , piece_id=EHMHJHJ2ZQWAJWBFSHOTAXWE7QJQOIICXDKAK33KMOAF7C7KZ2PA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.506877697 (689h21m42.103648727s) [noise=true , address=145.79.227.8:4345 , piece_id=2UWIKM3L3JUKNYWPBVJBEGFOEWS3WVM24AY7C2OVBZWDAXDG4I3A, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.507462874 (689h21m42.104233903s) [noise=true , address=193.149.164.71:4308 , piece_id=YGF5F7J25BEULKSTR3ZDC3HE4EHVJ3IF4HQCNATVWHITGRB344JA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.507975323 (689h21m42.10474639s) [noise=true , address=77.90.31.15:4709 , piece_id=HQLXOXRSN7R5WC5K2W7X4YUBVEZFITOY4VEYKG7J6WIJDEHABWGA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.582999887 (689h21m42.17977091s) [upload=772064]: Next returned nil segment Apr 4 04:21:44.583065715 (689h21m42.179836738s) [upload=772064]: Waiting for error group managing segments... Apr 4 04:21:44.587179313 (689h21m42.183950339s) [noise=true , address=45.91.116.12:4229 , piece_id=CDXY4QA2BKVNNJUHGL2JPK3MMHTVDPRN64SAQSMZSVQIG755ZOJQ, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:44.587315638 (689h21m42.184086655s) [noise=true , address=45.91.116.13:4158 , piece_id=OX4BJEH3DNMAUXS64QHDHRK75YYEF6DPQJAX7MS3J6K7RUM4GM2Q, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:44.592227274 (689h21m42.188998305s) [noise=true , address=145.79.227.8:4345 , piece_id=2UWIKM3L3JUKNYWPBVJBEGFOEWS3WVM24AY7C2OVBZWDAXDG4I3A, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:44.897574490 (689h21m42.494345523s) [noise=true , address=167.17.72.70:4751 , piece_id=AX7C5IEHMVNPGGAB7WPWUOSFEHVWWA75JGDAH3DAGZVZCVMEZKKQ, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.084711646 (689h21m42.681482669s) [noise=true , address=193.149.164.66:4996 , piece_id=YGOQ6INTPJ3AZW3RSTVH2IWITNFONA7JTW25BIXNBZRRZUN6YRYA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.084884121 (689h21m42.681655134s) [noise=true , address=193.149.164.67:4002 , piece_id=CYJYJ5ZNXT63OT5DBDLW7ML32OIZLGNWMLM4W2F5TPSSJ2QJ2NBA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.089152844 (689h21m42.685923867s) [noise=true , address=193.149.164.70:4115 , piece_id=EHMHJHJ2ZQWAJWBFSHOTAXWE7QJQOIICXDKAK33KMOAF7C7KZ2PA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.103385369 (689h21m42.700156382s) [noise=true , address=193.149.164.68:4062 , piece_id=IM6B4U2ARVUSBX4276OSQPEPQNMWMFSUUOYAEJ3ZG2GLFUKGGQGQ, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.127711185 (689h21m42.724482213s) [noise=true , address=193.149.164.69:4084 , piece_id=BUV3LZ7PDHSGRNKHJSSN2OFQYMIYUELWEPO4PF3DMTBX65S4YPMA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.143513369 (689h21m42.740284381s) [noise=true , address=193.149.164.71:4308 , piece_id=YGF5F7J25BEULKSTR3ZDC3HE4EHVJ3IF4HQCNATVWHITGRB344JA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.307124646 (689h21m42.903895674s) [noise=true , address=167.17.72.68:4656 , piece_id=YFAXDPM4TYBLYL4UR3Z6FDCDJXU34J7XRURDGLXZAMUOYYZL56IA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.517599945 (689h21m43.114370966s) [noise=true , address=145.79.227.33:4898 , piece_id=64X2NB2BPWLP4C5EC53A6EKFPNQCJI6BGEDY6BFE2RFNYEPCDA5Q, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:46.061904485 (689h21m43.658675509s) [noise=true , address=193.149.164.143:4704 , piece_id=PZ2W2J22M5MZIYBFEOJCEOIVAJOJGYCHCQUPDSELS764HZG4OF2A, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:22:44.423237599 (689h22m42.020008631s) [noise=true , address=77.90.31.13:4658 , piece_id=I7GGGCPJIHSQ5RYYOPIXKWAJBOMSRSK3ZDCASMKUPBQPALOJCKXA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: ecclient: upload failed (node:129aongkQE9HMNz8vsNNA5SvKbhsHqdt9UrtSTPJJpP9mkFsboX, address:77.90.31.13:4658): protocol: read tcp 159.223.143.39:50834->77.90.31.13:4658: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 4 04:22:44.423364101 (689h22m42.020135144s) [noise=true , address=77.90.31.17:4783 , piece_id=E2XPGXQAVSYEMBWOIPKHGL5MVVGC2FSLN5KHZQC6DBUAT3VUWOKA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: ecclient: upload failed (node:155fvDXZdpQeHetitv17WCVVrP1bGswdoqJECQLSe66kXRhBPo, address:77.90.31.17:4783): protocol: read tcp 159.223.143.39:45990->77.90.31.17:4783: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 4 04:22:44.423435607 (689h22m42.020206643s) [noise=true , address=77.90.31.16:4750 , piece_id=TI2D4Z2MYSXWBHX3ZVEMRFLI4LW5RDVCENH43S7GEFR7WGIVCI6A, seg_pos={0 0}, upload=772064]: Done uploading piece. err: ecclient: upload failed (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE, address:77.90.31.16:4750): protocol: read tcp 159.223.143.39:34564->77.90.31.16:4750: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 4 04:22:44.423560856 (689h22m42.020332015s) [noise=true , address=77.90.31.18:4823 , piece_id=MI45FJCGXJ2CIL4OOVHEUB7TJXEGH6WELCSTKR6JD753YLF5UN4Q, seg_pos={0 0}, upload=772064]: Done uploading piece. err: ecclient: upload failed (node:12uUCUDKZDwj25UcEKQ4WiTuQFMRbWw2oymGAdUdMX8wcUSd7jS, address:77.90.31.18:4823): protocol: read tcp 159.223.143.39:40028->77.90.31.18:4823: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 4 04:22:44.423793037 (689h22m42.020564071s) [noise=true , address=77.90.31.15:4709 , piece_id=HQLXOXRSN7R5WC5K2W7X4YUBVEZFITOY4VEYKG7J6WIJDEHABWGA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: ecclient: upload failed (node:1CrHsLV4vge154tJjMvbo5oxnphfukuXQBKfvTjA43n6CJM2Ah, address:77.90.31.15:4709): protocol: read tcp 159.223.143.39:59234->77.90.31.15:4709: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 4 04:22:44.423864095 (689h22m42.020635106s) [seg_pos={0 0}, upload=772064]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 4 04:22:44.451342916 (689h22m42.048113939s) [upload=772064]: Done uploading segments... Apr 4 04:22:44.451381831 (689h22m42.048152855s) [upload=772064]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1.279724878s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 580.836203ms trace: https://jaeger.dev.storj.io/trace/670ee369f5f2d334 FAIL (2026-04-04 04:44:18 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 580.871144ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 5 out of 60 is failed (threshold=6) [................X..X.........XXX............................] duration: 528.707004ms trace: https://jaeger.dev.storj.io/trace/3304a822f59853ad FAIL (2026-04-04 04:43:55 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 528.770356ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 689.069654ms trace: https://jaeger.dev.storj.io/trace/670ee369f5f2d334 FAIL (2026-04-04 04:43:48 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 689.117175ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 1.204457522s trace: https://jaeger.dev.storj.io/trace/5cce09de575b8506 FAIL (2026-04-04 04:43:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 6881916s) [upload=772064]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 4 04:21:44.460130782 (689h21m42.056901815s) [upload=772064]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 4 04:21:44.500551580 (689h21m42.097322604s) [seg_pos={0 0}, upload=772064]: Begin upload segment... Apr 4 04:21:44.500628241 (689h21m42.097399263s) [seg_pos={0 0}, upload=772064]: Done begin upload segment. Apr 4 04:21:44.500668892 (689h21m42.097439922s) [noise=true , address=193.149.164.143:4704 , piece_id=PZ2W2J22M5MZIYBFEOJCEOIVAJOJGYCHCQUPDSELS764HZG4OF2A, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.500792978 (689h21m42.097564014s) [noise=true , address=45.91.116.13:4158 , piece_id=OX4BJEH3DNMAUXS64QHDHRK75YYEF6DPQJAX7MS3J6K7RUM4GM2Q, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.501687053 (689h21m42.09845809s) [noise=true , address=77.90.31.17:4783 , piece_id=E2XPGXQAVSYEMBWOIPKHGL5MVVGC2FSLN5KHZQC6DBUAT3VUWOKA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.501828492 (689h21m42.098599515s) [noise=true , address=193.149.164.69:4084 , piece_id=BUV3LZ7PDHSGRNKHJSSN2OFQYMIYUELWEPO4PF3DMTBX65S4YPMA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.502343164 (689h21m42.099114198s) [noise=true , address=193.149.164.67:4002 , piece_id=CYJYJ5ZNXT63OT5DBDLW7ML32OIZLGNWMLM4W2F5TPSSJ2QJ2NBA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.502900735 (689h21m42.099671764s) [noise=true , address=77.90.31.18:4823 , piece_id=MI45FJCGXJ2CIL4OOVHEUB7TJXEGH6WELCSTKR6JD753YLF5UN4Q, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.503495842 (689h21m42.100266868s) [noise=true , address=193.149.164.68:4062 , piece_id=IM6B4U2ARVUSBX4276OSQPEPQNMWMFSUUOYAEJ3ZG2GLFUKGGQGQ, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.503796813 (689h21m42.100567839s) [noise=true , address=77.90.31.13:4658 , piece_id=I7GGGCPJIHSQ5RYYOPIXKWAJBOMSRSK3ZDCASMKUPBQPALOJCKXA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.504006258 (689h21m42.100777287s) [noise=true , address=77.90.31.16:4750 , piece_id=TI2D4Z2MYSXWBHX3ZVEMRFLI4LW5RDVCENH43S7GEFR7WGIVCI6A, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.504542000 (689h21m42.101313035s) [noise=true , address=145.79.227.33:4898 , piece_id=64X2NB2BPWLP4C5EC53A6EKFPNQCJI6BGEDY6BFE2RFNYEPCDA5Q, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.504560079 (689h21m42.101331115s) [noise=true , address=45.91.116.12:4229 , piece_id=CDXY4QA2BKVNNJUHGL2JPK3MMHTVDPRN64SAQSMZSVQIG755ZOJQ, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.505047220 (689h21m42.101818243s) [noise=true , address=193.149.164.66:4996 , piece_id=YGOQ6INTPJ3AZW3RSTVH2IWITNFONA7JTW25BIXNBZRRZUN6YRYA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.505183875 (689h21m42.10195491s) [noise=true , address=167.17.72.70:4751 , piece_id=AX7C5IEHMVNPGGAB7WPWUOSFEHVWWA75JGDAH3DAGZVZCVMEZKKQ, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.505619570 (689h21m42.102390603s) [noise=true , address=167.17.72.68:4656 , piece_id=YFAXDPM4TYBLYL4UR3Z6FDCDJXU34J7XRURDGLXZAMUOYYZL56IA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.506074686 (689h21m42.102845722s) [noise=true , address=193.149.164.70:4115 , piece_id=EHMHJHJ2ZQWAJWBFSHOTAXWE7QJQOIICXDKAK33KMOAF7C7KZ2PA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.506877697 (689h21m42.103648727s) [noise=true , address=145.79.227.8:4345 , piece_id=2UWIKM3L3JUKNYWPBVJBEGFOEWS3WVM24AY7C2OVBZWDAXDG4I3A, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.507462874 (689h21m42.104233903s) [noise=true , address=193.149.164.71:4308 , piece_id=YGF5F7J25BEULKSTR3ZDC3HE4EHVJ3IF4HQCNATVWHITGRB344JA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.507975323 (689h21m42.10474639s) [noise=true , address=77.90.31.15:4709 , piece_id=HQLXOXRSN7R5WC5K2W7X4YUBVEZFITOY4VEYKG7J6WIJDEHABWGA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.582999887 (689h21m42.17977091s) [upload=772064]: Next returned nil segment Apr 4 04:21:44.583065715 (689h21m42.179836738s) [upload=772064]: Waiting for error group managing segments... Apr 4 04:21:44.587179313 (689h21m42.183950339s) [noise=true , address=45.91.116.12:4229 , piece_id=CDXY4QA2BKVNNJUHGL2JPK3MMHTVDPRN64SAQSMZSVQIG755ZOJQ, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:44.587315638 (689h21m42.184086655s) [noise=true , address=45.91.116.13:4158 , piece_id=OX4BJEH3DNMAUXS64QHDHRK75YYEF6DPQJAX7MS3J6K7RUM4GM2Q, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:44.592227274 (689h21m42.188998305s) [noise=true , address=145.79.227.8:4345 , piece_id=2UWIKM3L3JUKNYWPBVJBEGFOEWS3WVM24AY7C2OVBZWDAXDG4I3A, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:44.897574490 (689h21m42.494345523s) [noise=true , address=167.17.72.70:4751 , piece_id=AX7C5IEHMVNPGGAB7WPWUOSFEHVWWA75JGDAH3DAGZVZCVMEZKKQ, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.084711646 (689h21m42.681482669s) [noise=true , address=193.149.164.66:4996 , piece_id=YGOQ6INTPJ3AZW3RSTVH2IWITNFONA7JTW25BIXNBZRRZUN6YRYA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.084884121 (689h21m42.681655134s) [noise=true , address=193.149.164.67:4002 , piece_id=CYJYJ5ZNXT63OT5DBDLW7ML32OIZLGNWMLM4W2F5TPSSJ2QJ2NBA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.089152844 (689h21m42.685923867s) [noise=true , address=193.149.164.70:4115 , piece_id=EHMHJHJ2ZQWAJWBFSHOTAXWE7QJQOIICXDKAK33KMOAF7C7KZ2PA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.103385369 (689h21m42.700156382s) [noise=true , address=193.149.164.68:4062 , piece_id=IM6B4U2ARVUSBX4276OSQPEPQNMWMFSUUOYAEJ3ZG2GLFUKGGQGQ, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.127711185 (689h21m42.724482213s) [noise=true , address=193.149.164.69:4084 , piece_id=BUV3LZ7PDHSGRNKHJSSN2OFQYMIYUELWEPO4PF3DMTBX65S4YPMA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.143513369 (689h21m42.740284381s) [noise=true , address=193.149.164.71:4308 , piece_id=YGF5F7J25BEULKSTR3ZDC3HE4EHVJ3IF4HQCNATVWHITGRB344JA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.307124646 (689h21m42.903895674s) [noise=true , address=167.17.72.68:4656 , piece_id=YFAXDPM4TYBLYL4UR3Z6FDCDJXU34J7XRURDGLXZAMUOYYZL56IA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.517599945 (689h21m43.114370966s) [noise=true , address=145.79.227.33:4898 , piece_id=64X2NB2BPWLP4C5EC53A6EKFPNQCJI6BGEDY6BFE2RFNYEPCDA5Q, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:46.061904485 (689h21m43.658675509s) [noise=true , address=193.149.164.143:4704 , piece_id=PZ2W2J22M5MZIYBFEOJCEOIVAJOJGYCHCQUPDSELS764HZG4OF2A, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:22:44.423237599 (689h22m42.020008631s) [noise=true , address=77.90.31.13:4658 , piece_id=I7GGGCPJIHSQ5RYYOPIXKWAJBOMSRSK3ZDCASMKUPBQPALOJCKXA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: ecclient: upload failed (node:129aongkQE9HMNz8vsNNA5SvKbhsHqdt9UrtSTPJJpP9mkFsboX, address:77.90.31.13:4658): protocol: read tcp 159.223.143.39:50834->77.90.31.13:4658: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 4 04:22:44.423364101 (689h22m42.020135144s) [noise=true , address=77.90.31.17:4783 , piece_id=E2XPGXQAVSYEMBWOIPKHGL5MVVGC2FSLN5KHZQC6DBUAT3VUWOKA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: ecclient: upload failed (node:155fvDXZdpQeHetitv17WCVVrP1bGswdoqJECQLSe66kXRhBPo, address:77.90.31.17:4783): protocol: read tcp 159.223.143.39:45990->77.90.31.17:4783: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 4 04:22:44.423435607 (689h22m42.020206643s) [noise=true , address=77.90.31.16:4750 , piece_id=TI2D4Z2MYSXWBHX3ZVEMRFLI4LW5RDVCENH43S7GEFR7WGIVCI6A, seg_pos={0 0}, upload=772064]: Done uploading piece. err: ecclient: upload failed (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE, address:77.90.31.16:4750): protocol: read tcp 159.223.143.39:34564->77.90.31.16:4750: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 4 04:22:44.423560856 (689h22m42.020332015s) [noise=true , address=77.90.31.18:4823 , piece_id=MI45FJCGXJ2CIL4OOVHEUB7TJXEGH6WELCSTKR6JD753YLF5UN4Q, seg_pos={0 0}, upload=772064]: Done uploading piece. err: ecclient: upload failed (node:12uUCUDKZDwj25UcEKQ4WiTuQFMRbWw2oymGAdUdMX8wcUSd7jS, address:77.90.31.18:4823): protocol: read tcp 159.223.143.39:40028->77.90.31.18:4823: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 4 04:22:44.423793037 (689h22m42.020564071s) [noise=true , address=77.90.31.15:4709 , piece_id=HQLXOXRSN7R5WC5K2W7X4YUBVEZFITOY4VEYKG7J6WIJDEHABWGA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: ecclient: upload failed (node:1CrHsLV4vge154tJjMvbo5oxnphfukuXQBKfvTjA43n6CJM2Ah, address:77.90.31.15:4709): protocol: read tcp 159.223.143.39:59234->77.90.31.15:4709: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 4 04:22:44.423864095 (689h22m42.020635106s) [seg_pos={0 0}, upload=772064]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 4 04:22:44.451342916 (689h22m42.048113939s) [upload=772064]: Done uploading segments... Apr 4 04:22:44.451381831 (689h22m42.048152855s) [upload=772064]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1.204487472s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 5 out of 60 is failed (threshold=6) [.................X..X.........XXX...........................] duration: 465.928303ms trace: https://jaeger.dev.storj.io/trace/3304a822f59853ad FAIL (2026-04-04 04:42:42 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 466.085223ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 537.710457ms trace: https://jaeger.dev.storj.io/trace/670ee369f5f2d334 FAIL (2026-04-04 04:42:40 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 537.774084ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 1.414766663s trace: https://jaeger.dev.storj.io/trace/5cce09de575b8506 FAIL (2026-04-04 04:42:29 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 6881916s) [upload=772064]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 4 04:21:44.460130782 (689h21m42.056901815s) [upload=772064]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 4 04:21:44.500551580 (689h21m42.097322604s) [seg_pos={0 0}, upload=772064]: Begin upload segment... Apr 4 04:21:44.500628241 (689h21m42.097399263s) [seg_pos={0 0}, upload=772064]: Done begin upload segment. Apr 4 04:21:44.500668892 (689h21m42.097439922s) [noise=true , address=193.149.164.143:4704 , piece_id=PZ2W2J22M5MZIYBFEOJCEOIVAJOJGYCHCQUPDSELS764HZG4OF2A, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.500792978 (689h21m42.097564014s) [noise=true , address=45.91.116.13:4158 , piece_id=OX4BJEH3DNMAUXS64QHDHRK75YYEF6DPQJAX7MS3J6K7RUM4GM2Q, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.501687053 (689h21m42.09845809s) [noise=true , address=77.90.31.17:4783 , piece_id=E2XPGXQAVSYEMBWOIPKHGL5MVVGC2FSLN5KHZQC6DBUAT3VUWOKA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.501828492 (689h21m42.098599515s) [noise=true , address=193.149.164.69:4084 , piece_id=BUV3LZ7PDHSGRNKHJSSN2OFQYMIYUELWEPO4PF3DMTBX65S4YPMA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.502343164 (689h21m42.099114198s) [noise=true , address=193.149.164.67:4002 , piece_id=CYJYJ5ZNXT63OT5DBDLW7ML32OIZLGNWMLM4W2F5TPSSJ2QJ2NBA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.502900735 (689h21m42.099671764s) [noise=true , address=77.90.31.18:4823 , piece_id=MI45FJCGXJ2CIL4OOVHEUB7TJXEGH6WELCSTKR6JD753YLF5UN4Q, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.503495842 (689h21m42.100266868s) [noise=true , address=193.149.164.68:4062 , piece_id=IM6B4U2ARVUSBX4276OSQPEPQNMWMFSUUOYAEJ3ZG2GLFUKGGQGQ, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.503796813 (689h21m42.100567839s) [noise=true , address=77.90.31.13:4658 , piece_id=I7GGGCPJIHSQ5RYYOPIXKWAJBOMSRSK3ZDCASMKUPBQPALOJCKXA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.504006258 (689h21m42.100777287s) [noise=true , address=77.90.31.16:4750 , piece_id=TI2D4Z2MYSXWBHX3ZVEMRFLI4LW5RDVCENH43S7GEFR7WGIVCI6A, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.504542000 (689h21m42.101313035s) [noise=true , address=145.79.227.33:4898 , piece_id=64X2NB2BPWLP4C5EC53A6EKFPNQCJI6BGEDY6BFE2RFNYEPCDA5Q, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.504560079 (689h21m42.101331115s) [noise=true , address=45.91.116.12:4229 , piece_id=CDXY4QA2BKVNNJUHGL2JPK3MMHTVDPRN64SAQSMZSVQIG755ZOJQ, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.505047220 (689h21m42.101818243s) [noise=true , address=193.149.164.66:4996 , piece_id=YGOQ6INTPJ3AZW3RSTVH2IWITNFONA7JTW25BIXNBZRRZUN6YRYA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.505183875 (689h21m42.10195491s) [noise=true , address=167.17.72.70:4751 , piece_id=AX7C5IEHMVNPGGAB7WPWUOSFEHVWWA75JGDAH3DAGZVZCVMEZKKQ, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.505619570 (689h21m42.102390603s) [noise=true , address=167.17.72.68:4656 , piece_id=YFAXDPM4TYBLYL4UR3Z6FDCDJXU34J7XRURDGLXZAMUOYYZL56IA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.506074686 (689h21m42.102845722s) [noise=true , address=193.149.164.70:4115 , piece_id=EHMHJHJ2ZQWAJWBFSHOTAXWE7QJQOIICXDKAK33KMOAF7C7KZ2PA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.506877697 (689h21m42.103648727s) [noise=true , address=145.79.227.8:4345 , piece_id=2UWIKM3L3JUKNYWPBVJBEGFOEWS3WVM24AY7C2OVBZWDAXDG4I3A, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.507462874 (689h21m42.104233903s) [noise=true , address=193.149.164.71:4308 , piece_id=YGF5F7J25BEULKSTR3ZDC3HE4EHVJ3IF4HQCNATVWHITGRB344JA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.507975323 (689h21m42.10474639s) [noise=true , address=77.90.31.15:4709 , piece_id=HQLXOXRSN7R5WC5K2W7X4YUBVEZFITOY4VEYKG7J6WIJDEHABWGA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.582999887 (689h21m42.17977091s) [upload=772064]: Next returned nil segment Apr 4 04:21:44.583065715 (689h21m42.179836738s) [upload=772064]: Waiting for error group managing segments... Apr 4 04:21:44.587179313 (689h21m42.183950339s) [noise=true , address=45.91.116.12:4229 , piece_id=CDXY4QA2BKVNNJUHGL2JPK3MMHTVDPRN64SAQSMZSVQIG755ZOJQ, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:44.587315638 (689h21m42.184086655s) [noise=true , address=45.91.116.13:4158 , piece_id=OX4BJEH3DNMAUXS64QHDHRK75YYEF6DPQJAX7MS3J6K7RUM4GM2Q, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:44.592227274 (689h21m42.188998305s) [noise=true , address=145.79.227.8:4345 , piece_id=2UWIKM3L3JUKNYWPBVJBEGFOEWS3WVM24AY7C2OVBZWDAXDG4I3A, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:44.897574490 (689h21m42.494345523s) [noise=true , address=167.17.72.70:4751 , piece_id=AX7C5IEHMVNPGGAB7WPWUOSFEHVWWA75JGDAH3DAGZVZCVMEZKKQ, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.084711646 (689h21m42.681482669s) [noise=true , address=193.149.164.66:4996 , piece_id=YGOQ6INTPJ3AZW3RSTVH2IWITNFONA7JTW25BIXNBZRRZUN6YRYA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.084884121 (689h21m42.681655134s) [noise=true , address=193.149.164.67:4002 , piece_id=CYJYJ5ZNXT63OT5DBDLW7ML32OIZLGNWMLM4W2F5TPSSJ2QJ2NBA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.089152844 (689h21m42.685923867s) [noise=true , address=193.149.164.70:4115 , piece_id=EHMHJHJ2ZQWAJWBFSHOTAXWE7QJQOIICXDKAK33KMOAF7C7KZ2PA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.103385369 (689h21m42.700156382s) [noise=true , address=193.149.164.68:4062 , piece_id=IM6B4U2ARVUSBX4276OSQPEPQNMWMFSUUOYAEJ3ZG2GLFUKGGQGQ, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.127711185 (689h21m42.724482213s) [noise=true , address=193.149.164.69:4084 , piece_id=BUV3LZ7PDHSGRNKHJSSN2OFQYMIYUELWEPO4PF3DMTBX65S4YPMA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.143513369 (689h21m42.740284381s) [noise=true , address=193.149.164.71:4308 , piece_id=YGF5F7J25BEULKSTR3ZDC3HE4EHVJ3IF4HQCNATVWHITGRB344JA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.307124646 (689h21m42.903895674s) [noise=true , address=167.17.72.68:4656 , piece_id=YFAXDPM4TYBLYL4UR3Z6FDCDJXU34J7XRURDGLXZAMUOYYZL56IA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.517599945 (689h21m43.114370966s) [noise=true , address=145.79.227.33:4898 , piece_id=64X2NB2BPWLP4C5EC53A6EKFPNQCJI6BGEDY6BFE2RFNYEPCDA5Q, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:46.061904485 (689h21m43.658675509s) [noise=true , address=193.149.164.143:4704 , piece_id=PZ2W2J22M5MZIYBFEOJCEOIVAJOJGYCHCQUPDSELS764HZG4OF2A, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:22:44.423237599 (689h22m42.020008631s) [noise=true , address=77.90.31.13:4658 , piece_id=I7GGGCPJIHSQ5RYYOPIXKWAJBOMSRSK3ZDCASMKUPBQPALOJCKXA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: ecclient: upload failed (node:129aongkQE9HMNz8vsNNA5SvKbhsHqdt9UrtSTPJJpP9mkFsboX, address:77.90.31.13:4658): protocol: read tcp 159.223.143.39:50834->77.90.31.13:4658: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 4 04:22:44.423364101 (689h22m42.020135144s) [noise=true , address=77.90.31.17:4783 , piece_id=E2XPGXQAVSYEMBWOIPKHGL5MVVGC2FSLN5KHZQC6DBUAT3VUWOKA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: ecclient: upload failed (node:155fvDXZdpQeHetitv17WCVVrP1bGswdoqJECQLSe66kXRhBPo, address:77.90.31.17:4783): protocol: read tcp 159.223.143.39:45990->77.90.31.17:4783: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 4 04:22:44.423435607 (689h22m42.020206643s) [noise=true , address=77.90.31.16:4750 , piece_id=TI2D4Z2MYSXWBHX3ZVEMRFLI4LW5RDVCENH43S7GEFR7WGIVCI6A, seg_pos={0 0}, upload=772064]: Done uploading piece. err: ecclient: upload failed (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE, address:77.90.31.16:4750): protocol: read tcp 159.223.143.39:34564->77.90.31.16:4750: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 4 04:22:44.423560856 (689h22m42.020332015s) [noise=true , address=77.90.31.18:4823 , piece_id=MI45FJCGXJ2CIL4OOVHEUB7TJXEGH6WELCSTKR6JD753YLF5UN4Q, seg_pos={0 0}, upload=772064]: Done uploading piece. err: ecclient: upload failed (node:12uUCUDKZDwj25UcEKQ4WiTuQFMRbWw2oymGAdUdMX8wcUSd7jS, address:77.90.31.18:4823): protocol: read tcp 159.223.143.39:40028->77.90.31.18:4823: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 4 04:22:44.423793037 (689h22m42.020564071s) [noise=true , address=77.90.31.15:4709 , piece_id=HQLXOXRSN7R5WC5K2W7X4YUBVEZFITOY4VEYKG7J6WIJDEHABWGA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: ecclient: upload failed (node:1CrHsLV4vge154tJjMvbo5oxnphfukuXQBKfvTjA43n6CJM2Ah, address:77.90.31.15:4709): protocol: read tcp 159.223.143.39:59234->77.90.31.15:4709: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 4 04:22:44.423864095 (689h22m42.020635106s) [seg_pos={0 0}, upload=772064]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 4 04:22:44.451342916 (689h22m42.048113939s) [upload=772064]: Done uploading segments... Apr 4 04:22:44.451381831 (689h22m42.048152855s) [upload=772064]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1.414809233s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 1.381396385s trace: https://jaeger.dev.storj.io/trace/5cce09de575b8506 FAIL (2026-04-04 04:41:55 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 6881916s) [upload=772064]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 4 04:21:44.460130782 (689h21m42.056901815s) [upload=772064]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 4 04:21:44.500551580 (689h21m42.097322604s) [seg_pos={0 0}, upload=772064]: Begin upload segment... Apr 4 04:21:44.500628241 (689h21m42.097399263s) [seg_pos={0 0}, upload=772064]: Done begin upload segment. Apr 4 04:21:44.500668892 (689h21m42.097439922s) [noise=true , address=193.149.164.143:4704 , piece_id=PZ2W2J22M5MZIYBFEOJCEOIVAJOJGYCHCQUPDSELS764HZG4OF2A, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.500792978 (689h21m42.097564014s) [noise=true , address=45.91.116.13:4158 , piece_id=OX4BJEH3DNMAUXS64QHDHRK75YYEF6DPQJAX7MS3J6K7RUM4GM2Q, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.501687053 (689h21m42.09845809s) [noise=true , address=77.90.31.17:4783 , piece_id=E2XPGXQAVSYEMBWOIPKHGL5MVVGC2FSLN5KHZQC6DBUAT3VUWOKA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.501828492 (689h21m42.098599515s) [noise=true , address=193.149.164.69:4084 , piece_id=BUV3LZ7PDHSGRNKHJSSN2OFQYMIYUELWEPO4PF3DMTBX65S4YPMA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.502343164 (689h21m42.099114198s) [noise=true , address=193.149.164.67:4002 , piece_id=CYJYJ5ZNXT63OT5DBDLW7ML32OIZLGNWMLM4W2F5TPSSJ2QJ2NBA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.502900735 (689h21m42.099671764s) [noise=true , address=77.90.31.18:4823 , piece_id=MI45FJCGXJ2CIL4OOVHEUB7TJXEGH6WELCSTKR6JD753YLF5UN4Q, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.503495842 (689h21m42.100266868s) [noise=true , address=193.149.164.68:4062 , piece_id=IM6B4U2ARVUSBX4276OSQPEPQNMWMFSUUOYAEJ3ZG2GLFUKGGQGQ, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.503796813 (689h21m42.100567839s) [noise=true , address=77.90.31.13:4658 , piece_id=I7GGGCPJIHSQ5RYYOPIXKWAJBOMSRSK3ZDCASMKUPBQPALOJCKXA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.504006258 (689h21m42.100777287s) [noise=true , address=77.90.31.16:4750 , piece_id=TI2D4Z2MYSXWBHX3ZVEMRFLI4LW5RDVCENH43S7GEFR7WGIVCI6A, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.504542000 (689h21m42.101313035s) [noise=true , address=145.79.227.33:4898 , piece_id=64X2NB2BPWLP4C5EC53A6EKFPNQCJI6BGEDY6BFE2RFNYEPCDA5Q, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.504560079 (689h21m42.101331115s) [noise=true , address=45.91.116.12:4229 , piece_id=CDXY4QA2BKVNNJUHGL2JPK3MMHTVDPRN64SAQSMZSVQIG755ZOJQ, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.505047220 (689h21m42.101818243s) [noise=true , address=193.149.164.66:4996 , piece_id=YGOQ6INTPJ3AZW3RSTVH2IWITNFONA7JTW25BIXNBZRRZUN6YRYA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.505183875 (689h21m42.10195491s) [noise=true , address=167.17.72.70:4751 , piece_id=AX7C5IEHMVNPGGAB7WPWUOSFEHVWWA75JGDAH3DAGZVZCVMEZKKQ, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.505619570 (689h21m42.102390603s) [noise=true , address=167.17.72.68:4656 , piece_id=YFAXDPM4TYBLYL4UR3Z6FDCDJXU34J7XRURDGLXZAMUOYYZL56IA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.506074686 (689h21m42.102845722s) [noise=true , address=193.149.164.70:4115 , piece_id=EHMHJHJ2ZQWAJWBFSHOTAXWE7QJQOIICXDKAK33KMOAF7C7KZ2PA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.506877697 (689h21m42.103648727s) [noise=true , address=145.79.227.8:4345 , piece_id=2UWIKM3L3JUKNYWPBVJBEGFOEWS3WVM24AY7C2OVBZWDAXDG4I3A, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.507462874 (689h21m42.104233903s) [noise=true , address=193.149.164.71:4308 , piece_id=YGF5F7J25BEULKSTR3ZDC3HE4EHVJ3IF4HQCNATVWHITGRB344JA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.507975323 (689h21m42.10474639s) [noise=true , address=77.90.31.15:4709 , piece_id=HQLXOXRSN7R5WC5K2W7X4YUBVEZFITOY4VEYKG7J6WIJDEHABWGA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.582999887 (689h21m42.17977091s) [upload=772064]: Next returned nil segment Apr 4 04:21:44.583065715 (689h21m42.179836738s) [upload=772064]: Waiting for error group managing segments... Apr 4 04:21:44.587179313 (689h21m42.183950339s) [noise=true , address=45.91.116.12:4229 , piece_id=CDXY4QA2BKVNNJUHGL2JPK3MMHTVDPRN64SAQSMZSVQIG755ZOJQ, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:44.587315638 (689h21m42.184086655s) [noise=true , address=45.91.116.13:4158 , piece_id=OX4BJEH3DNMAUXS64QHDHRK75YYEF6DPQJAX7MS3J6K7RUM4GM2Q, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:44.592227274 (689h21m42.188998305s) [noise=true , address=145.79.227.8:4345 , piece_id=2UWIKM3L3JUKNYWPBVJBEGFOEWS3WVM24AY7C2OVBZWDAXDG4I3A, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:44.897574490 (689h21m42.494345523s) [noise=true , address=167.17.72.70:4751 , piece_id=AX7C5IEHMVNPGGAB7WPWUOSFEHVWWA75JGDAH3DAGZVZCVMEZKKQ, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.084711646 (689h21m42.681482669s) [noise=true , address=193.149.164.66:4996 , piece_id=YGOQ6INTPJ3AZW3RSTVH2IWITNFONA7JTW25BIXNBZRRZUN6YRYA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.084884121 (689h21m42.681655134s) [noise=true , address=193.149.164.67:4002 , piece_id=CYJYJ5ZNXT63OT5DBDLW7ML32OIZLGNWMLM4W2F5TPSSJ2QJ2NBA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.089152844 (689h21m42.685923867s) [noise=true , address=193.149.164.70:4115 , piece_id=EHMHJHJ2ZQWAJWBFSHOTAXWE7QJQOIICXDKAK33KMOAF7C7KZ2PA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.103385369 (689h21m42.700156382s) [noise=true , address=193.149.164.68:4062 , piece_id=IM6B4U2ARVUSBX4276OSQPEPQNMWMFSUUOYAEJ3ZG2GLFUKGGQGQ, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.127711185 (689h21m42.724482213s) [noise=true , address=193.149.164.69:4084 , piece_id=BUV3LZ7PDHSGRNKHJSSN2OFQYMIYUELWEPO4PF3DMTBX65S4YPMA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.143513369 (689h21m42.740284381s) [noise=true , address=193.149.164.71:4308 , piece_id=YGF5F7J25BEULKSTR3ZDC3HE4EHVJ3IF4HQCNATVWHITGRB344JA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.307124646 (689h21m42.903895674s) [noise=true , address=167.17.72.68:4656 , piece_id=YFAXDPM4TYBLYL4UR3Z6FDCDJXU34J7XRURDGLXZAMUOYYZL56IA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.517599945 (689h21m43.114370966s) [noise=true , address=145.79.227.33:4898 , piece_id=64X2NB2BPWLP4C5EC53A6EKFPNQCJI6BGEDY6BFE2RFNYEPCDA5Q, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:46.061904485 (689h21m43.658675509s) [noise=true , address=193.149.164.143:4704 , piece_id=PZ2W2J22M5MZIYBFEOJCEOIVAJOJGYCHCQUPDSELS764HZG4OF2A, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:22:44.423237599 (689h22m42.020008631s) [noise=true , address=77.90.31.13:4658 , piece_id=I7GGGCPJIHSQ5RYYOPIXKWAJBOMSRSK3ZDCASMKUPBQPALOJCKXA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: ecclient: upload failed (node:129aongkQE9HMNz8vsNNA5SvKbhsHqdt9UrtSTPJJpP9mkFsboX, address:77.90.31.13:4658): protocol: read tcp 159.223.143.39:50834->77.90.31.13:4658: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 4 04:22:44.423364101 (689h22m42.020135144s) [noise=true , address=77.90.31.17:4783 , piece_id=E2XPGXQAVSYEMBWOIPKHGL5MVVGC2FSLN5KHZQC6DBUAT3VUWOKA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: ecclient: upload failed (node:155fvDXZdpQeHetitv17WCVVrP1bGswdoqJECQLSe66kXRhBPo, address:77.90.31.17:4783): protocol: read tcp 159.223.143.39:45990->77.90.31.17:4783: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 4 04:22:44.423435607 (689h22m42.020206643s) [noise=true , address=77.90.31.16:4750 , piece_id=TI2D4Z2MYSXWBHX3ZVEMRFLI4LW5RDVCENH43S7GEFR7WGIVCI6A, seg_pos={0 0}, upload=772064]: Done uploading piece. err: ecclient: upload failed (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE, address:77.90.31.16:4750): protocol: read tcp 159.223.143.39:34564->77.90.31.16:4750: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 4 04:22:44.423560856 (689h22m42.020332015s) [noise=true , address=77.90.31.18:4823 , piece_id=MI45FJCGXJ2CIL4OOVHEUB7TJXEGH6WELCSTKR6JD753YLF5UN4Q, seg_pos={0 0}, upload=772064]: Done uploading piece. err: ecclient: upload failed (node:12uUCUDKZDwj25UcEKQ4WiTuQFMRbWw2oymGAdUdMX8wcUSd7jS, address:77.90.31.18:4823): protocol: read tcp 159.223.143.39:40028->77.90.31.18:4823: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 4 04:22:44.423793037 (689h22m42.020564071s) [noise=true , address=77.90.31.15:4709 , piece_id=HQLXOXRSN7R5WC5K2W7X4YUBVEZFITOY4VEYKG7J6WIJDEHABWGA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: ecclient: upload failed (node:1CrHsLV4vge154tJjMvbo5oxnphfukuXQBKfvTjA43n6CJM2Ah, address:77.90.31.15:4709): protocol: read tcp 159.223.143.39:59234->77.90.31.15:4709: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 4 04:22:44.423864095 (689h22m42.020635106s) [seg_pos={0 0}, upload=772064]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 4 04:22:44.451342916 (689h22m42.048113939s) [upload=772064]: Done uploading segments... Apr 4 04:22:44.451381831 (689h22m42.048152855s) [upload=772064]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1.381428533s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 5 out of 60 is failed (threshold=6) [..................X..X.........XXX..........................] duration: 485.89042ms trace: https://jaeger.dev.storj.io/trace/3304a822f59853ad FAIL (2026-04-04 04:41:51 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 485.920572ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 1.280618957s trace: https://jaeger.dev.storj.io/trace/5cce09de575b8506 FAIL (2026-04-04 04:41:30 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 6881916s) [upload=772064]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 4 04:21:44.460130782 (689h21m42.056901815s) [upload=772064]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 4 04:21:44.500551580 (689h21m42.097322604s) [seg_pos={0 0}, upload=772064]: Begin upload segment... Apr 4 04:21:44.500628241 (689h21m42.097399263s) [seg_pos={0 0}, upload=772064]: Done begin upload segment. Apr 4 04:21:44.500668892 (689h21m42.097439922s) [noise=true , address=193.149.164.143:4704 , piece_id=PZ2W2J22M5MZIYBFEOJCEOIVAJOJGYCHCQUPDSELS764HZG4OF2A, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.500792978 (689h21m42.097564014s) [noise=true , address=45.91.116.13:4158 , piece_id=OX4BJEH3DNMAUXS64QHDHRK75YYEF6DPQJAX7MS3J6K7RUM4GM2Q, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.501687053 (689h21m42.09845809s) [noise=true , address=77.90.31.17:4783 , piece_id=E2XPGXQAVSYEMBWOIPKHGL5MVVGC2FSLN5KHZQC6DBUAT3VUWOKA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.501828492 (689h21m42.098599515s) [noise=true , address=193.149.164.69:4084 , piece_id=BUV3LZ7PDHSGRNKHJSSN2OFQYMIYUELWEPO4PF3DMTBX65S4YPMA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.502343164 (689h21m42.099114198s) [noise=true , address=193.149.164.67:4002 , piece_id=CYJYJ5ZNXT63OT5DBDLW7ML32OIZLGNWMLM4W2F5TPSSJ2QJ2NBA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.502900735 (689h21m42.099671764s) [noise=true , address=77.90.31.18:4823 , piece_id=MI45FJCGXJ2CIL4OOVHEUB7TJXEGH6WELCSTKR6JD753YLF5UN4Q, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.503495842 (689h21m42.100266868s) [noise=true , address=193.149.164.68:4062 , piece_id=IM6B4U2ARVUSBX4276OSQPEPQNMWMFSUUOYAEJ3ZG2GLFUKGGQGQ, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.503796813 (689h21m42.100567839s) [noise=true , address=77.90.31.13:4658 , piece_id=I7GGGCPJIHSQ5RYYOPIXKWAJBOMSRSK3ZDCASMKUPBQPALOJCKXA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.504006258 (689h21m42.100777287s) [noise=true , address=77.90.31.16:4750 , piece_id=TI2D4Z2MYSXWBHX3ZVEMRFLI4LW5RDVCENH43S7GEFR7WGIVCI6A, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.504542000 (689h21m42.101313035s) [noise=true , address=145.79.227.33:4898 , piece_id=64X2NB2BPWLP4C5EC53A6EKFPNQCJI6BGEDY6BFE2RFNYEPCDA5Q, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.504560079 (689h21m42.101331115s) [noise=true , address=45.91.116.12:4229 , piece_id=CDXY4QA2BKVNNJUHGL2JPK3MMHTVDPRN64SAQSMZSVQIG755ZOJQ, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.505047220 (689h21m42.101818243s) [noise=true , address=193.149.164.66:4996 , piece_id=YGOQ6INTPJ3AZW3RSTVH2IWITNFONA7JTW25BIXNBZRRZUN6YRYA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.505183875 (689h21m42.10195491s) [noise=true , address=167.17.72.70:4751 , piece_id=AX7C5IEHMVNPGGAB7WPWUOSFEHVWWA75JGDAH3DAGZVZCVMEZKKQ, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.505619570 (689h21m42.102390603s) [noise=true , address=167.17.72.68:4656 , piece_id=YFAXDPM4TYBLYL4UR3Z6FDCDJXU34J7XRURDGLXZAMUOYYZL56IA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.506074686 (689h21m42.102845722s) [noise=true , address=193.149.164.70:4115 , piece_id=EHMHJHJ2ZQWAJWBFSHOTAXWE7QJQOIICXDKAK33KMOAF7C7KZ2PA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.506877697 (689h21m42.103648727s) [noise=true , address=145.79.227.8:4345 , piece_id=2UWIKM3L3JUKNYWPBVJBEGFOEWS3WVM24AY7C2OVBZWDAXDG4I3A, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.507462874 (689h21m42.104233903s) [noise=true , address=193.149.164.71:4308 , piece_id=YGF5F7J25BEULKSTR3ZDC3HE4EHVJ3IF4HQCNATVWHITGRB344JA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.507975323 (689h21m42.10474639s) [noise=true , address=77.90.31.15:4709 , piece_id=HQLXOXRSN7R5WC5K2W7X4YUBVEZFITOY4VEYKG7J6WIJDEHABWGA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.582999887 (689h21m42.17977091s) [upload=772064]: Next returned nil segment Apr 4 04:21:44.583065715 (689h21m42.179836738s) [upload=772064]: Waiting for error group managing segments... Apr 4 04:21:44.587179313 (689h21m42.183950339s) [noise=true , address=45.91.116.12:4229 , piece_id=CDXY4QA2BKVNNJUHGL2JPK3MMHTVDPRN64SAQSMZSVQIG755ZOJQ, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:44.587315638 (689h21m42.184086655s) [noise=true , address=45.91.116.13:4158 , piece_id=OX4BJEH3DNMAUXS64QHDHRK75YYEF6DPQJAX7MS3J6K7RUM4GM2Q, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:44.592227274 (689h21m42.188998305s) [noise=true , address=145.79.227.8:4345 , piece_id=2UWIKM3L3JUKNYWPBVJBEGFOEWS3WVM24AY7C2OVBZWDAXDG4I3A, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:44.897574490 (689h21m42.494345523s) [noise=true , address=167.17.72.70:4751 , piece_id=AX7C5IEHMVNPGGAB7WPWUOSFEHVWWA75JGDAH3DAGZVZCVMEZKKQ, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.084711646 (689h21m42.681482669s) [noise=true , address=193.149.164.66:4996 , piece_id=YGOQ6INTPJ3AZW3RSTVH2IWITNFONA7JTW25BIXNBZRRZUN6YRYA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.084884121 (689h21m42.681655134s) [noise=true , address=193.149.164.67:4002 , piece_id=CYJYJ5ZNXT63OT5DBDLW7ML32OIZLGNWMLM4W2F5TPSSJ2QJ2NBA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.089152844 (689h21m42.685923867s) [noise=true , address=193.149.164.70:4115 , piece_id=EHMHJHJ2ZQWAJWBFSHOTAXWE7QJQOIICXDKAK33KMOAF7C7KZ2PA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.103385369 (689h21m42.700156382s) [noise=true , address=193.149.164.68:4062 , piece_id=IM6B4U2ARVUSBX4276OSQPEPQNMWMFSUUOYAEJ3ZG2GLFUKGGQGQ, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.127711185 (689h21m42.724482213s) [noise=true , address=193.149.164.69:4084 , piece_id=BUV3LZ7PDHSGRNKHJSSN2OFQYMIYUELWEPO4PF3DMTBX65S4YPMA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.143513369 (689h21m42.740284381s) [noise=true , address=193.149.164.71:4308 , piece_id=YGF5F7J25BEULKSTR3ZDC3HE4EHVJ3IF4HQCNATVWHITGRB344JA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.307124646 (689h21m42.903895674s) [noise=true , address=167.17.72.68:4656 , piece_id=YFAXDPM4TYBLYL4UR3Z6FDCDJXU34J7XRURDGLXZAMUOYYZL56IA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.517599945 (689h21m43.114370966s) [noise=true , address=145.79.227.33:4898 , piece_id=64X2NB2BPWLP4C5EC53A6EKFPNQCJI6BGEDY6BFE2RFNYEPCDA5Q, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:46.061904485 (689h21m43.658675509s) [noise=true , address=193.149.164.143:4704 , piece_id=PZ2W2J22M5MZIYBFEOJCEOIVAJOJGYCHCQUPDSELS764HZG4OF2A, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:22:44.423237599 (689h22m42.020008631s) [noise=true , address=77.90.31.13:4658 , piece_id=I7GGGCPJIHSQ5RYYOPIXKWAJBOMSRSK3ZDCASMKUPBQPALOJCKXA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: ecclient: upload failed (node:129aongkQE9HMNz8vsNNA5SvKbhsHqdt9UrtSTPJJpP9mkFsboX, address:77.90.31.13:4658): protocol: read tcp 159.223.143.39:50834->77.90.31.13:4658: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 4 04:22:44.423364101 (689h22m42.020135144s) [noise=true , address=77.90.31.17:4783 , piece_id=E2XPGXQAVSYEMBWOIPKHGL5MVVGC2FSLN5KHZQC6DBUAT3VUWOKA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: ecclient: upload failed (node:155fvDXZdpQeHetitv17WCVVrP1bGswdoqJECQLSe66kXRhBPo, address:77.90.31.17:4783): protocol: read tcp 159.223.143.39:45990->77.90.31.17:4783: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 4 04:22:44.423435607 (689h22m42.020206643s) [noise=true , address=77.90.31.16:4750 , piece_id=TI2D4Z2MYSXWBHX3ZVEMRFLI4LW5RDVCENH43S7GEFR7WGIVCI6A, seg_pos={0 0}, upload=772064]: Done uploading piece. err: ecclient: upload failed (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE, address:77.90.31.16:4750): protocol: read tcp 159.223.143.39:34564->77.90.31.16:4750: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 4 04:22:44.423560856 (689h22m42.020332015s) [noise=true , address=77.90.31.18:4823 , piece_id=MI45FJCGXJ2CIL4OOVHEUB7TJXEGH6WELCSTKR6JD753YLF5UN4Q, seg_pos={0 0}, upload=772064]: Done uploading piece. err: ecclient: upload failed (node:12uUCUDKZDwj25UcEKQ4WiTuQFMRbWw2oymGAdUdMX8wcUSd7jS, address:77.90.31.18:4823): protocol: read tcp 159.223.143.39:40028->77.90.31.18:4823: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 4 04:22:44.423793037 (689h22m42.020564071s) [noise=true , address=77.90.31.15:4709 , piece_id=HQLXOXRSN7R5WC5K2W7X4YUBVEZFITOY4VEYKG7J6WIJDEHABWGA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: ecclient: upload failed (node:1CrHsLV4vge154tJjMvbo5oxnphfukuXQBKfvTjA43n6CJM2Ah, address:77.90.31.15:4709): protocol: read tcp 159.223.143.39:59234->77.90.31.15:4709: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 4 04:22:44.423864095 (689h22m42.020635106s) [seg_pos={0 0}, upload=772064]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 4 04:22:44.451342916 (689h22m42.048113939s) [upload=772064]: Done uploading segments... Apr 4 04:22:44.451381831 (689h22m42.048152855s) [upload=772064]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1.280670484s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 720.640442ms trace: https://jaeger.dev.storj.io/trace/670ee369f5f2d334 FAIL (2026-04-04 04:41:28 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 720.684229ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 5 out of 60 is failed (threshold=6) [...................X..X.........XXX.........................] duration: 442.17142ms trace: https://jaeger.dev.storj.io/trace/3304a822f59853ad FAIL (2026-04-04 04:41:21 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 442.197096ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 5 out of 60 is failed (threshold=6) [....................X..X.........XXX........................] duration: 455.367396ms trace: https://jaeger.dev.storj.io/trace/3304a822f59853ad FAIL (2026-04-04 04:40:38 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 455.42777ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 1.352769821s trace: https://jaeger.dev.storj.io/trace/5cce09de575b8506 FAIL (2026-04-04 04:40:29 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 6881916s) [upload=772064]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 4 04:21:44.460130782 (689h21m42.056901815s) [upload=772064]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 4 04:21:44.500551580 (689h21m42.097322604s) [seg_pos={0 0}, upload=772064]: Begin upload segment... Apr 4 04:21:44.500628241 (689h21m42.097399263s) [seg_pos={0 0}, upload=772064]: Done begin upload segment. Apr 4 04:21:44.500668892 (689h21m42.097439922s) [noise=true , address=193.149.164.143:4704 , piece_id=PZ2W2J22M5MZIYBFEOJCEOIVAJOJGYCHCQUPDSELS764HZG4OF2A, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.500792978 (689h21m42.097564014s) [noise=true , address=45.91.116.13:4158 , piece_id=OX4BJEH3DNMAUXS64QHDHRK75YYEF6DPQJAX7MS3J6K7RUM4GM2Q, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.501687053 (689h21m42.09845809s) [noise=true , address=77.90.31.17:4783 , piece_id=E2XPGXQAVSYEMBWOIPKHGL5MVVGC2FSLN5KHZQC6DBUAT3VUWOKA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.501828492 (689h21m42.098599515s) [noise=true , address=193.149.164.69:4084 , piece_id=BUV3LZ7PDHSGRNKHJSSN2OFQYMIYUELWEPO4PF3DMTBX65S4YPMA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.502343164 (689h21m42.099114198s) [noise=true , address=193.149.164.67:4002 , piece_id=CYJYJ5ZNXT63OT5DBDLW7ML32OIZLGNWMLM4W2F5TPSSJ2QJ2NBA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.502900735 (689h21m42.099671764s) [noise=true , address=77.90.31.18:4823 , piece_id=MI45FJCGXJ2CIL4OOVHEUB7TJXEGH6WELCSTKR6JD753YLF5UN4Q, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.503495842 (689h21m42.100266868s) [noise=true , address=193.149.164.68:4062 , piece_id=IM6B4U2ARVUSBX4276OSQPEPQNMWMFSUUOYAEJ3ZG2GLFUKGGQGQ, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.503796813 (689h21m42.100567839s) [noise=true , address=77.90.31.13:4658 , piece_id=I7GGGCPJIHSQ5RYYOPIXKWAJBOMSRSK3ZDCASMKUPBQPALOJCKXA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.504006258 (689h21m42.100777287s) [noise=true , address=77.90.31.16:4750 , piece_id=TI2D4Z2MYSXWBHX3ZVEMRFLI4LW5RDVCENH43S7GEFR7WGIVCI6A, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.504542000 (689h21m42.101313035s) [noise=true , address=145.79.227.33:4898 , piece_id=64X2NB2BPWLP4C5EC53A6EKFPNQCJI6BGEDY6BFE2RFNYEPCDA5Q, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.504560079 (689h21m42.101331115s) [noise=true , address=45.91.116.12:4229 , piece_id=CDXY4QA2BKVNNJUHGL2JPK3MMHTVDPRN64SAQSMZSVQIG755ZOJQ, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.505047220 (689h21m42.101818243s) [noise=true , address=193.149.164.66:4996 , piece_id=YGOQ6INTPJ3AZW3RSTVH2IWITNFONA7JTW25BIXNBZRRZUN6YRYA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.505183875 (689h21m42.10195491s) [noise=true , address=167.17.72.70:4751 , piece_id=AX7C5IEHMVNPGGAB7WPWUOSFEHVWWA75JGDAH3DAGZVZCVMEZKKQ, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.505619570 (689h21m42.102390603s) [noise=true , address=167.17.72.68:4656 , piece_id=YFAXDPM4TYBLYL4UR3Z6FDCDJXU34J7XRURDGLXZAMUOYYZL56IA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.506074686 (689h21m42.102845722s) [noise=true , address=193.149.164.70:4115 , piece_id=EHMHJHJ2ZQWAJWBFSHOTAXWE7QJQOIICXDKAK33KMOAF7C7KZ2PA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.506877697 (689h21m42.103648727s) [noise=true , address=145.79.227.8:4345 , piece_id=2UWIKM3L3JUKNYWPBVJBEGFOEWS3WVM24AY7C2OVBZWDAXDG4I3A, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.507462874 (689h21m42.104233903s) [noise=true , address=193.149.164.71:4308 , piece_id=YGF5F7J25BEULKSTR3ZDC3HE4EHVJ3IF4HQCNATVWHITGRB344JA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.507975323 (689h21m42.10474639s) [noise=true , address=77.90.31.15:4709 , piece_id=HQLXOXRSN7R5WC5K2W7X4YUBVEZFITOY4VEYKG7J6WIJDEHABWGA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.582999887 (689h21m42.17977091s) [upload=772064]: Next returned nil segment Apr 4 04:21:44.583065715 (689h21m42.179836738s) [upload=772064]: Waiting for error group managing segments... Apr 4 04:21:44.587179313 (689h21m42.183950339s) [noise=true , address=45.91.116.12:4229 , piece_id=CDXY4QA2BKVNNJUHGL2JPK3MMHTVDPRN64SAQSMZSVQIG755ZOJQ, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:44.587315638 (689h21m42.184086655s) [noise=true , address=45.91.116.13:4158 , piece_id=OX4BJEH3DNMAUXS64QHDHRK75YYEF6DPQJAX7MS3J6K7RUM4GM2Q, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:44.592227274 (689h21m42.188998305s) [noise=true , address=145.79.227.8:4345 , piece_id=2UWIKM3L3JUKNYWPBVJBEGFOEWS3WVM24AY7C2OVBZWDAXDG4I3A, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:44.897574490 (689h21m42.494345523s) [noise=true , address=167.17.72.70:4751 , piece_id=AX7C5IEHMVNPGGAB7WPWUOSFEHVWWA75JGDAH3DAGZVZCVMEZKKQ, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.084711646 (689h21m42.681482669s) [noise=true , address=193.149.164.66:4996 , piece_id=YGOQ6INTPJ3AZW3RSTVH2IWITNFONA7JTW25BIXNBZRRZUN6YRYA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.084884121 (689h21m42.681655134s) [noise=true , address=193.149.164.67:4002 , piece_id=CYJYJ5ZNXT63OT5DBDLW7ML32OIZLGNWMLM4W2F5TPSSJ2QJ2NBA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.089152844 (689h21m42.685923867s) [noise=true , address=193.149.164.70:4115 , piece_id=EHMHJHJ2ZQWAJWBFSHOTAXWE7QJQOIICXDKAK33KMOAF7C7KZ2PA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.103385369 (689h21m42.700156382s) [noise=true , address=193.149.164.68:4062 , piece_id=IM6B4U2ARVUSBX4276OSQPEPQNMWMFSUUOYAEJ3ZG2GLFUKGGQGQ, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.127711185 (689h21m42.724482213s) [noise=true , address=193.149.164.69:4084 , piece_id=BUV3LZ7PDHSGRNKHJSSN2OFQYMIYUELWEPO4PF3DMTBX65S4YPMA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.143513369 (689h21m42.740284381s) [noise=true , address=193.149.164.71:4308 , piece_id=YGF5F7J25BEULKSTR3ZDC3HE4EHVJ3IF4HQCNATVWHITGRB344JA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.307124646 (689h21m42.903895674s) [noise=true , address=167.17.72.68:4656 , piece_id=YFAXDPM4TYBLYL4UR3Z6FDCDJXU34J7XRURDGLXZAMUOYYZL56IA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.517599945 (689h21m43.114370966s) [noise=true , address=145.79.227.33:4898 , piece_id=64X2NB2BPWLP4C5EC53A6EKFPNQCJI6BGEDY6BFE2RFNYEPCDA5Q, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:46.061904485 (689h21m43.658675509s) [noise=true , address=193.149.164.143:4704 , piece_id=PZ2W2J22M5MZIYBFEOJCEOIVAJOJGYCHCQUPDSELS764HZG4OF2A, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:22:44.423237599 (689h22m42.020008631s) [noise=true , address=77.90.31.13:4658 , piece_id=I7GGGCPJIHSQ5RYYOPIXKWAJBOMSRSK3ZDCASMKUPBQPALOJCKXA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: ecclient: upload failed (node:129aongkQE9HMNz8vsNNA5SvKbhsHqdt9UrtSTPJJpP9mkFsboX, address:77.90.31.13:4658): protocol: read tcp 159.223.143.39:50834->77.90.31.13:4658: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 4 04:22:44.423364101 (689h22m42.020135144s) [noise=true , address=77.90.31.17:4783 , piece_id=E2XPGXQAVSYEMBWOIPKHGL5MVVGC2FSLN5KHZQC6DBUAT3VUWOKA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: ecclient: upload failed (node:155fvDXZdpQeHetitv17WCVVrP1bGswdoqJECQLSe66kXRhBPo, address:77.90.31.17:4783): protocol: read tcp 159.223.143.39:45990->77.90.31.17:4783: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 4 04:22:44.423435607 (689h22m42.020206643s) [noise=true , address=77.90.31.16:4750 , piece_id=TI2D4Z2MYSXWBHX3ZVEMRFLI4LW5RDVCENH43S7GEFR7WGIVCI6A, seg_pos={0 0}, upload=772064]: Done uploading piece. err: ecclient: upload failed (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE, address:77.90.31.16:4750): protocol: read tcp 159.223.143.39:34564->77.90.31.16:4750: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 4 04:22:44.423560856 (689h22m42.020332015s) [noise=true , address=77.90.31.18:4823 , piece_id=MI45FJCGXJ2CIL4OOVHEUB7TJXEGH6WELCSTKR6JD753YLF5UN4Q, seg_pos={0 0}, upload=772064]: Done uploading piece. err: ecclient: upload failed (node:12uUCUDKZDwj25UcEKQ4WiTuQFMRbWw2oymGAdUdMX8wcUSd7jS, address:77.90.31.18:4823): protocol: read tcp 159.223.143.39:40028->77.90.31.18:4823: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 4 04:22:44.423793037 (689h22m42.020564071s) [noise=true , address=77.90.31.15:4709 , piece_id=HQLXOXRSN7R5WC5K2W7X4YUBVEZFITOY4VEYKG7J6WIJDEHABWGA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: ecclient: upload failed (node:1CrHsLV4vge154tJjMvbo5oxnphfukuXQBKfvTjA43n6CJM2Ah, address:77.90.31.15:4709): protocol: read tcp 159.223.143.39:59234->77.90.31.15:4709: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 4 04:22:44.423864095 (689h22m42.020635106s) [seg_pos={0 0}, upload=772064]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 4 04:22:44.451342916 (689h22m42.048113939s) [upload=772064]: Done uploading segments... Apr 4 04:22:44.451381831 (689h22m42.048152855s) [upload=772064]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1.352801006s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 742.714747ms trace: https://jaeger.dev.storj.io/trace/670ee369f5f2d334 FAIL (2026-04-04 04:40:26 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 742.81826ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 890.396161ms trace: https://jaeger.dev.storj.io/trace/670ee369f5f2d334 FAIL (2026-04-04 04:40:04 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 890.523239ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 5 out of 60 is failed (threshold=6) [.....................X..X.........XXX.......................] duration: 459.389271ms trace: https://jaeger.dev.storj.io/trace/3304a822f59853ad FAIL (2026-04-04 04:39:54 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 459.488733ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 5 out of 60 is failed (threshold=6) [......................X..X.........XXX......................] duration: 463.942027ms trace: https://jaeger.dev.storj.io/trace/3304a822f59853ad FAIL (2026-04-04 04:39:19 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 463.967041ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 785.613753ms trace: https://jaeger.dev.storj.io/trace/670ee369f5f2d334 FAIL (2026-04-04 04:39:16 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 785.697856ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 1.372759817s trace: https://jaeger.dev.storj.io/trace/5cce09de575b8506 FAIL (2026-04-04 04:39:14 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 6881916s) [upload=772064]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 4 04:21:44.460130782 (689h21m42.056901815s) [upload=772064]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 4 04:21:44.500551580 (689h21m42.097322604s) [seg_pos={0 0}, upload=772064]: Begin upload segment... Apr 4 04:21:44.500628241 (689h21m42.097399263s) [seg_pos={0 0}, upload=772064]: Done begin upload segment. Apr 4 04:21:44.500668892 (689h21m42.097439922s) [noise=true , address=193.149.164.143:4704 , piece_id=PZ2W2J22M5MZIYBFEOJCEOIVAJOJGYCHCQUPDSELS764HZG4OF2A, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.500792978 (689h21m42.097564014s) [noise=true , address=45.91.116.13:4158 , piece_id=OX4BJEH3DNMAUXS64QHDHRK75YYEF6DPQJAX7MS3J6K7RUM4GM2Q, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.501687053 (689h21m42.09845809s) [noise=true , address=77.90.31.17:4783 , piece_id=E2XPGXQAVSYEMBWOIPKHGL5MVVGC2FSLN5KHZQC6DBUAT3VUWOKA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.501828492 (689h21m42.098599515s) [noise=true , address=193.149.164.69:4084 , piece_id=BUV3LZ7PDHSGRNKHJSSN2OFQYMIYUELWEPO4PF3DMTBX65S4YPMA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.502343164 (689h21m42.099114198s) [noise=true , address=193.149.164.67:4002 , piece_id=CYJYJ5ZNXT63OT5DBDLW7ML32OIZLGNWMLM4W2F5TPSSJ2QJ2NBA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.502900735 (689h21m42.099671764s) [noise=true , address=77.90.31.18:4823 , piece_id=MI45FJCGXJ2CIL4OOVHEUB7TJXEGH6WELCSTKR6JD753YLF5UN4Q, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.503495842 (689h21m42.100266868s) [noise=true , address=193.149.164.68:4062 , piece_id=IM6B4U2ARVUSBX4276OSQPEPQNMWMFSUUOYAEJ3ZG2GLFUKGGQGQ, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.503796813 (689h21m42.100567839s) [noise=true , address=77.90.31.13:4658 , piece_id=I7GGGCPJIHSQ5RYYOPIXKWAJBOMSRSK3ZDCASMKUPBQPALOJCKXA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.504006258 (689h21m42.100777287s) [noise=true , address=77.90.31.16:4750 , piece_id=TI2D4Z2MYSXWBHX3ZVEMRFLI4LW5RDVCENH43S7GEFR7WGIVCI6A, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.504542000 (689h21m42.101313035s) [noise=true , address=145.79.227.33:4898 , piece_id=64X2NB2BPWLP4C5EC53A6EKFPNQCJI6BGEDY6BFE2RFNYEPCDA5Q, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.504560079 (689h21m42.101331115s) [noise=true , address=45.91.116.12:4229 , piece_id=CDXY4QA2BKVNNJUHGL2JPK3MMHTVDPRN64SAQSMZSVQIG755ZOJQ, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.505047220 (689h21m42.101818243s) [noise=true , address=193.149.164.66:4996 , piece_id=YGOQ6INTPJ3AZW3RSTVH2IWITNFONA7JTW25BIXNBZRRZUN6YRYA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.505183875 (689h21m42.10195491s) [noise=true , address=167.17.72.70:4751 , piece_id=AX7C5IEHMVNPGGAB7WPWUOSFEHVWWA75JGDAH3DAGZVZCVMEZKKQ, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.505619570 (689h21m42.102390603s) [noise=true , address=167.17.72.68:4656 , piece_id=YFAXDPM4TYBLYL4UR3Z6FDCDJXU34J7XRURDGLXZAMUOYYZL56IA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.506074686 (689h21m42.102845722s) [noise=true , address=193.149.164.70:4115 , piece_id=EHMHJHJ2ZQWAJWBFSHOTAXWE7QJQOIICXDKAK33KMOAF7C7KZ2PA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.506877697 (689h21m42.103648727s) [noise=true , address=145.79.227.8:4345 , piece_id=2UWIKM3L3JUKNYWPBVJBEGFOEWS3WVM24AY7C2OVBZWDAXDG4I3A, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.507462874 (689h21m42.104233903s) [noise=true , address=193.149.164.71:4308 , piece_id=YGF5F7J25BEULKSTR3ZDC3HE4EHVJ3IF4HQCNATVWHITGRB344JA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.507975323 (689h21m42.10474639s) [noise=true , address=77.90.31.15:4709 , piece_id=HQLXOXRSN7R5WC5K2W7X4YUBVEZFITOY4VEYKG7J6WIJDEHABWGA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.582999887 (689h21m42.17977091s) [upload=772064]: Next returned nil segment Apr 4 04:21:44.583065715 (689h21m42.179836738s) [upload=772064]: Waiting for error group managing segments... Apr 4 04:21:44.587179313 (689h21m42.183950339s) [noise=true , address=45.91.116.12:4229 , piece_id=CDXY4QA2BKVNNJUHGL2JPK3MMHTVDPRN64SAQSMZSVQIG755ZOJQ, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:44.587315638 (689h21m42.184086655s) [noise=true , address=45.91.116.13:4158 , piece_id=OX4BJEH3DNMAUXS64QHDHRK75YYEF6DPQJAX7MS3J6K7RUM4GM2Q, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:44.592227274 (689h21m42.188998305s) [noise=true , address=145.79.227.8:4345 , piece_id=2UWIKM3L3JUKNYWPBVJBEGFOEWS3WVM24AY7C2OVBZWDAXDG4I3A, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:44.897574490 (689h21m42.494345523s) [noise=true , address=167.17.72.70:4751 , piece_id=AX7C5IEHMVNPGGAB7WPWUOSFEHVWWA75JGDAH3DAGZVZCVMEZKKQ, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.084711646 (689h21m42.681482669s) [noise=true , address=193.149.164.66:4996 , piece_id=YGOQ6INTPJ3AZW3RSTVH2IWITNFONA7JTW25BIXNBZRRZUN6YRYA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.084884121 (689h21m42.681655134s) [noise=true , address=193.149.164.67:4002 , piece_id=CYJYJ5ZNXT63OT5DBDLW7ML32OIZLGNWMLM4W2F5TPSSJ2QJ2NBA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.089152844 (689h21m42.685923867s) [noise=true , address=193.149.164.70:4115 , piece_id=EHMHJHJ2ZQWAJWBFSHOTAXWE7QJQOIICXDKAK33KMOAF7C7KZ2PA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.103385369 (689h21m42.700156382s) [noise=true , address=193.149.164.68:4062 , piece_id=IM6B4U2ARVUSBX4276OSQPEPQNMWMFSUUOYAEJ3ZG2GLFUKGGQGQ, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.127711185 (689h21m42.724482213s) [noise=true , address=193.149.164.69:4084 , piece_id=BUV3LZ7PDHSGRNKHJSSN2OFQYMIYUELWEPO4PF3DMTBX65S4YPMA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.143513369 (689h21m42.740284381s) [noise=true , address=193.149.164.71:4308 , piece_id=YGF5F7J25BEULKSTR3ZDC3HE4EHVJ3IF4HQCNATVWHITGRB344JA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.307124646 (689h21m42.903895674s) [noise=true , address=167.17.72.68:4656 , piece_id=YFAXDPM4TYBLYL4UR3Z6FDCDJXU34J7XRURDGLXZAMUOYYZL56IA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.517599945 (689h21m43.114370966s) [noise=true , address=145.79.227.33:4898 , piece_id=64X2NB2BPWLP4C5EC53A6EKFPNQCJI6BGEDY6BFE2RFNYEPCDA5Q, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:46.061904485 (689h21m43.658675509s) [noise=true , address=193.149.164.143:4704 , piece_id=PZ2W2J22M5MZIYBFEOJCEOIVAJOJGYCHCQUPDSELS764HZG4OF2A, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:22:44.423237599 (689h22m42.020008631s) [noise=true , address=77.90.31.13:4658 , piece_id=I7GGGCPJIHSQ5RYYOPIXKWAJBOMSRSK3ZDCASMKUPBQPALOJCKXA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: ecclient: upload failed (node:129aongkQE9HMNz8vsNNA5SvKbhsHqdt9UrtSTPJJpP9mkFsboX, address:77.90.31.13:4658): protocol: read tcp 159.223.143.39:50834->77.90.31.13:4658: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 4 04:22:44.423364101 (689h22m42.020135144s) [noise=true , address=77.90.31.17:4783 , piece_id=E2XPGXQAVSYEMBWOIPKHGL5MVVGC2FSLN5KHZQC6DBUAT3VUWOKA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: ecclient: upload failed (node:155fvDXZdpQeHetitv17WCVVrP1bGswdoqJECQLSe66kXRhBPo, address:77.90.31.17:4783): protocol: read tcp 159.223.143.39:45990->77.90.31.17:4783: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 4 04:22:44.423435607 (689h22m42.020206643s) [noise=true , address=77.90.31.16:4750 , piece_id=TI2D4Z2MYSXWBHX3ZVEMRFLI4LW5RDVCENH43S7GEFR7WGIVCI6A, seg_pos={0 0}, upload=772064]: Done uploading piece. err: ecclient: upload failed (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE, address:77.90.31.16:4750): protocol: read tcp 159.223.143.39:34564->77.90.31.16:4750: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 4 04:22:44.423560856 (689h22m42.020332015s) [noise=true , address=77.90.31.18:4823 , piece_id=MI45FJCGXJ2CIL4OOVHEUB7TJXEGH6WELCSTKR6JD753YLF5UN4Q, seg_pos={0 0}, upload=772064]: Done uploading piece. err: ecclient: upload failed (node:12uUCUDKZDwj25UcEKQ4WiTuQFMRbWw2oymGAdUdMX8wcUSd7jS, address:77.90.31.18:4823): protocol: read tcp 159.223.143.39:40028->77.90.31.18:4823: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 4 04:22:44.423793037 (689h22m42.020564071s) [noise=true , address=77.90.31.15:4709 , piece_id=HQLXOXRSN7R5WC5K2W7X4YUBVEZFITOY4VEYKG7J6WIJDEHABWGA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: ecclient: upload failed (node:1CrHsLV4vge154tJjMvbo5oxnphfukuXQBKfvTjA43n6CJM2Ah, address:77.90.31.15:4709): protocol: read tcp 159.223.143.39:59234->77.90.31.15:4709: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 4 04:22:44.423864095 (689h22m42.020635106s) [seg_pos={0 0}, upload=772064]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 4 04:22:44.451342916 (689h22m42.048113939s) [upload=772064]: Done uploading segments... Apr 4 04:22:44.451381831 (689h22m42.048152855s) [upload=772064]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1.372790078s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 622.017844ms trace: https://jaeger.dev.storj.io/trace/670ee369f5f2d334 FAIL (2026-04-04 04:38:45 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 622.318355ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 1.341983879s trace: https://jaeger.dev.storj.io/trace/5cce09de575b8506 FAIL (2026-04-04 04:38:18 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 6881916s) [upload=772064]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 4 04:21:44.460130782 (689h21m42.056901815s) [upload=772064]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 4 04:21:44.500551580 (689h21m42.097322604s) [seg_pos={0 0}, upload=772064]: Begin upload segment... Apr 4 04:21:44.500628241 (689h21m42.097399263s) [seg_pos={0 0}, upload=772064]: Done begin upload segment. Apr 4 04:21:44.500668892 (689h21m42.097439922s) [noise=true , address=193.149.164.143:4704 , piece_id=PZ2W2J22M5MZIYBFEOJCEOIVAJOJGYCHCQUPDSELS764HZG4OF2A, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.500792978 (689h21m42.097564014s) [noise=true , address=45.91.116.13:4158 , piece_id=OX4BJEH3DNMAUXS64QHDHRK75YYEF6DPQJAX7MS3J6K7RUM4GM2Q, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.501687053 (689h21m42.09845809s) [noise=true , address=77.90.31.17:4783 , piece_id=E2XPGXQAVSYEMBWOIPKHGL5MVVGC2FSLN5KHZQC6DBUAT3VUWOKA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.501828492 (689h21m42.098599515s) [noise=true , address=193.149.164.69:4084 , piece_id=BUV3LZ7PDHSGRNKHJSSN2OFQYMIYUELWEPO4PF3DMTBX65S4YPMA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.502343164 (689h21m42.099114198s) [noise=true , address=193.149.164.67:4002 , piece_id=CYJYJ5ZNXT63OT5DBDLW7ML32OIZLGNWMLM4W2F5TPSSJ2QJ2NBA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.502900735 (689h21m42.099671764s) [noise=true , address=77.90.31.18:4823 , piece_id=MI45FJCGXJ2CIL4OOVHEUB7TJXEGH6WELCSTKR6JD753YLF5UN4Q, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.503495842 (689h21m42.100266868s) [noise=true , address=193.149.164.68:4062 , piece_id=IM6B4U2ARVUSBX4276OSQPEPQNMWMFSUUOYAEJ3ZG2GLFUKGGQGQ, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.503796813 (689h21m42.100567839s) [noise=true , address=77.90.31.13:4658 , piece_id=I7GGGCPJIHSQ5RYYOPIXKWAJBOMSRSK3ZDCASMKUPBQPALOJCKXA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.504006258 (689h21m42.100777287s) [noise=true , address=77.90.31.16:4750 , piece_id=TI2D4Z2MYSXWBHX3ZVEMRFLI4LW5RDVCENH43S7GEFR7WGIVCI6A, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.504542000 (689h21m42.101313035s) [noise=true , address=145.79.227.33:4898 , piece_id=64X2NB2BPWLP4C5EC53A6EKFPNQCJI6BGEDY6BFE2RFNYEPCDA5Q, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.504560079 (689h21m42.101331115s) [noise=true , address=45.91.116.12:4229 , piece_id=CDXY4QA2BKVNNJUHGL2JPK3MMHTVDPRN64SAQSMZSVQIG755ZOJQ, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.505047220 (689h21m42.101818243s) [noise=true , address=193.149.164.66:4996 , piece_id=YGOQ6INTPJ3AZW3RSTVH2IWITNFONA7JTW25BIXNBZRRZUN6YRYA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.505183875 (689h21m42.10195491s) [noise=true , address=167.17.72.70:4751 , piece_id=AX7C5IEHMVNPGGAB7WPWUOSFEHVWWA75JGDAH3DAGZVZCVMEZKKQ, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.505619570 (689h21m42.102390603s) [noise=true , address=167.17.72.68:4656 , piece_id=YFAXDPM4TYBLYL4UR3Z6FDCDJXU34J7XRURDGLXZAMUOYYZL56IA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.506074686 (689h21m42.102845722s) [noise=true , address=193.149.164.70:4115 , piece_id=EHMHJHJ2ZQWAJWBFSHOTAXWE7QJQOIICXDKAK33KMOAF7C7KZ2PA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.506877697 (689h21m42.103648727s) [noise=true , address=145.79.227.8:4345 , piece_id=2UWIKM3L3JUKNYWPBVJBEGFOEWS3WVM24AY7C2OVBZWDAXDG4I3A, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.507462874 (689h21m42.104233903s) [noise=true , address=193.149.164.71:4308 , piece_id=YGF5F7J25BEULKSTR3ZDC3HE4EHVJ3IF4HQCNATVWHITGRB344JA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.507975323 (689h21m42.10474639s) [noise=true , address=77.90.31.15:4709 , piece_id=HQLXOXRSN7R5WC5K2W7X4YUBVEZFITOY4VEYKG7J6WIJDEHABWGA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.582999887 (689h21m42.17977091s) [upload=772064]: Next returned nil segment Apr 4 04:21:44.583065715 (689h21m42.179836738s) [upload=772064]: Waiting for error group managing segments... Apr 4 04:21:44.587179313 (689h21m42.183950339s) [noise=true , address=45.91.116.12:4229 , piece_id=CDXY4QA2BKVNNJUHGL2JPK3MMHTVDPRN64SAQSMZSVQIG755ZOJQ, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:44.587315638 (689h21m42.184086655s) [noise=true , address=45.91.116.13:4158 , piece_id=OX4BJEH3DNMAUXS64QHDHRK75YYEF6DPQJAX7MS3J6K7RUM4GM2Q, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:44.592227274 (689h21m42.188998305s) [noise=true , address=145.79.227.8:4345 , piece_id=2UWIKM3L3JUKNYWPBVJBEGFOEWS3WVM24AY7C2OVBZWDAXDG4I3A, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:44.897574490 (689h21m42.494345523s) [noise=true , address=167.17.72.70:4751 , piece_id=AX7C5IEHMVNPGGAB7WPWUOSFEHVWWA75JGDAH3DAGZVZCVMEZKKQ, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.084711646 (689h21m42.681482669s) [noise=true , address=193.149.164.66:4996 , piece_id=YGOQ6INTPJ3AZW3RSTVH2IWITNFONA7JTW25BIXNBZRRZUN6YRYA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.084884121 (689h21m42.681655134s) [noise=true , address=193.149.164.67:4002 , piece_id=CYJYJ5ZNXT63OT5DBDLW7ML32OIZLGNWMLM4W2F5TPSSJ2QJ2NBA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.089152844 (689h21m42.685923867s) [noise=true , address=193.149.164.70:4115 , piece_id=EHMHJHJ2ZQWAJWBFSHOTAXWE7QJQOIICXDKAK33KMOAF7C7KZ2PA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.103385369 (689h21m42.700156382s) [noise=true , address=193.149.164.68:4062 , piece_id=IM6B4U2ARVUSBX4276OSQPEPQNMWMFSUUOYAEJ3ZG2GLFUKGGQGQ, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.127711185 (689h21m42.724482213s) [noise=true , address=193.149.164.69:4084 , piece_id=BUV3LZ7PDHSGRNKHJSSN2OFQYMIYUELWEPO4PF3DMTBX65S4YPMA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.143513369 (689h21m42.740284381s) [noise=true , address=193.149.164.71:4308 , piece_id=YGF5F7J25BEULKSTR3ZDC3HE4EHVJ3IF4HQCNATVWHITGRB344JA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.307124646 (689h21m42.903895674s) [noise=true , address=167.17.72.68:4656 , piece_id=YFAXDPM4TYBLYL4UR3Z6FDCDJXU34J7XRURDGLXZAMUOYYZL56IA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.517599945 (689h21m43.114370966s) [noise=true , address=145.79.227.33:4898 , piece_id=64X2NB2BPWLP4C5EC53A6EKFPNQCJI6BGEDY6BFE2RFNYEPCDA5Q, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:46.061904485 (689h21m43.658675509s) [noise=true , address=193.149.164.143:4704 , piece_id=PZ2W2J22M5MZIYBFEOJCEOIVAJOJGYCHCQUPDSELS764HZG4OF2A, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:22:44.423237599 (689h22m42.020008631s) [noise=true , address=77.90.31.13:4658 , piece_id=I7GGGCPJIHSQ5RYYOPIXKWAJBOMSRSK3ZDCASMKUPBQPALOJCKXA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: ecclient: upload failed (node:129aongkQE9HMNz8vsNNA5SvKbhsHqdt9UrtSTPJJpP9mkFsboX, address:77.90.31.13:4658): protocol: read tcp 159.223.143.39:50834->77.90.31.13:4658: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 4 04:22:44.423364101 (689h22m42.020135144s) [noise=true , address=77.90.31.17:4783 , piece_id=E2XPGXQAVSYEMBWOIPKHGL5MVVGC2FSLN5KHZQC6DBUAT3VUWOKA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: ecclient: upload failed (node:155fvDXZdpQeHetitv17WCVVrP1bGswdoqJECQLSe66kXRhBPo, address:77.90.31.17:4783): protocol: read tcp 159.223.143.39:45990->77.90.31.17:4783: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 4 04:22:44.423435607 (689h22m42.020206643s) [noise=true , address=77.90.31.16:4750 , piece_id=TI2D4Z2MYSXWBHX3ZVEMRFLI4LW5RDVCENH43S7GEFR7WGIVCI6A, seg_pos={0 0}, upload=772064]: Done uploading piece. err: ecclient: upload failed (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE, address:77.90.31.16:4750): protocol: read tcp 159.223.143.39:34564->77.90.31.16:4750: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 4 04:22:44.423560856 (689h22m42.020332015s) [noise=true , address=77.90.31.18:4823 , piece_id=MI45FJCGXJ2CIL4OOVHEUB7TJXEGH6WELCSTKR6JD753YLF5UN4Q, seg_pos={0 0}, upload=772064]: Done uploading piece. err: ecclient: upload failed (node:12uUCUDKZDwj25UcEKQ4WiTuQFMRbWw2oymGAdUdMX8wcUSd7jS, address:77.90.31.18:4823): protocol: read tcp 159.223.143.39:40028->77.90.31.18:4823: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 4 04:22:44.423793037 (689h22m42.020564071s) [noise=true , address=77.90.31.15:4709 , piece_id=HQLXOXRSN7R5WC5K2W7X4YUBVEZFITOY4VEYKG7J6WIJDEHABWGA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: ecclient: upload failed (node:1CrHsLV4vge154tJjMvbo5oxnphfukuXQBKfvTjA43n6CJM2Ah, address:77.90.31.15:4709): protocol: read tcp 159.223.143.39:59234->77.90.31.15:4709: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 4 04:22:44.423864095 (689h22m42.020635106s) [seg_pos={0 0}, upload=772064]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 4 04:22:44.451342916 (689h22m42.048113939s) [upload=772064]: Done uploading segments... Apr 4 04:22:44.451381831 (689h22m42.048152855s) [upload=772064]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1.34202318s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 5 out of 60 is failed (threshold=6) [.......................X..X.........XXX.....................] duration: 439.269933ms trace: https://jaeger.dev.storj.io/trace/3304a822f59853ad FAIL (2026-04-04 04:38:11 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 439.310212ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 5 out of 60 is failed (threshold=6) [........................X..X.........XXX....................] duration: 585.6116ms trace: https://jaeger.dev.storj.io/trace/3304a822f59853ad FAIL (2026-04-04 04:37:49 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 585.641354ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 509.731256ms trace: https://jaeger.dev.storj.io/trace/670ee369f5f2d334 FAIL (2026-04-04 04:37:36 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 509.876001ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 5 out of 60 is failed (threshold=6) [.........................X..X.........XXX...................] duration: 580.807238ms trace: https://jaeger.dev.storj.io/trace/3304a822f59853ad FAIL (2026-04-04 04:37:15 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 580.82782ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 1.352024205s trace: https://jaeger.dev.storj.io/trace/5cce09de575b8506 FAIL (2026-04-04 04:37:06 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 6881916s) [upload=772064]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 4 04:21:44.460130782 (689h21m42.056901815s) [upload=772064]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 4 04:21:44.500551580 (689h21m42.097322604s) [seg_pos={0 0}, upload=772064]: Begin upload segment... Apr 4 04:21:44.500628241 (689h21m42.097399263s) [seg_pos={0 0}, upload=772064]: Done begin upload segment. Apr 4 04:21:44.500668892 (689h21m42.097439922s) [noise=true , address=193.149.164.143:4704 , piece_id=PZ2W2J22M5MZIYBFEOJCEOIVAJOJGYCHCQUPDSELS764HZG4OF2A, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.500792978 (689h21m42.097564014s) [noise=true , address=45.91.116.13:4158 , piece_id=OX4BJEH3DNMAUXS64QHDHRK75YYEF6DPQJAX7MS3J6K7RUM4GM2Q, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.501687053 (689h21m42.09845809s) [noise=true , address=77.90.31.17:4783 , piece_id=E2XPGXQAVSYEMBWOIPKHGL5MVVGC2FSLN5KHZQC6DBUAT3VUWOKA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.501828492 (689h21m42.098599515s) [noise=true , address=193.149.164.69:4084 , piece_id=BUV3LZ7PDHSGRNKHJSSN2OFQYMIYUELWEPO4PF3DMTBX65S4YPMA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.502343164 (689h21m42.099114198s) [noise=true , address=193.149.164.67:4002 , piece_id=CYJYJ5ZNXT63OT5DBDLW7ML32OIZLGNWMLM4W2F5TPSSJ2QJ2NBA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.502900735 (689h21m42.099671764s) [noise=true , address=77.90.31.18:4823 , piece_id=MI45FJCGXJ2CIL4OOVHEUB7TJXEGH6WELCSTKR6JD753YLF5UN4Q, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.503495842 (689h21m42.100266868s) [noise=true , address=193.149.164.68:4062 , piece_id=IM6B4U2ARVUSBX4276OSQPEPQNMWMFSUUOYAEJ3ZG2GLFUKGGQGQ, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.503796813 (689h21m42.100567839s) [noise=true , address=77.90.31.13:4658 , piece_id=I7GGGCPJIHSQ5RYYOPIXKWAJBOMSRSK3ZDCASMKUPBQPALOJCKXA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.504006258 (689h21m42.100777287s) [noise=true , address=77.90.31.16:4750 , piece_id=TI2D4Z2MYSXWBHX3ZVEMRFLI4LW5RDVCENH43S7GEFR7WGIVCI6A, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.504542000 (689h21m42.101313035s) [noise=true , address=145.79.227.33:4898 , piece_id=64X2NB2BPWLP4C5EC53A6EKFPNQCJI6BGEDY6BFE2RFNYEPCDA5Q, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.504560079 (689h21m42.101331115s) [noise=true , address=45.91.116.12:4229 , piece_id=CDXY4QA2BKVNNJUHGL2JPK3MMHTVDPRN64SAQSMZSVQIG755ZOJQ, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.505047220 (689h21m42.101818243s) [noise=true , address=193.149.164.66:4996 , piece_id=YGOQ6INTPJ3AZW3RSTVH2IWITNFONA7JTW25BIXNBZRRZUN6YRYA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.505183875 (689h21m42.10195491s) [noise=true , address=167.17.72.70:4751 , piece_id=AX7C5IEHMVNPGGAB7WPWUOSFEHVWWA75JGDAH3DAGZVZCVMEZKKQ, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.505619570 (689h21m42.102390603s) [noise=true , address=167.17.72.68:4656 , piece_id=YFAXDPM4TYBLYL4UR3Z6FDCDJXU34J7XRURDGLXZAMUOYYZL56IA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.506074686 (689h21m42.102845722s) [noise=true , address=193.149.164.70:4115 , piece_id=EHMHJHJ2ZQWAJWBFSHOTAXWE7QJQOIICXDKAK33KMOAF7C7KZ2PA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.506877697 (689h21m42.103648727s) [noise=true , address=145.79.227.8:4345 , piece_id=2UWIKM3L3JUKNYWPBVJBEGFOEWS3WVM24AY7C2OVBZWDAXDG4I3A, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.507462874 (689h21m42.104233903s) [noise=true , address=193.149.164.71:4308 , piece_id=YGF5F7J25BEULKSTR3ZDC3HE4EHVJ3IF4HQCNATVWHITGRB344JA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.507975323 (689h21m42.10474639s) [noise=true , address=77.90.31.15:4709 , piece_id=HQLXOXRSN7R5WC5K2W7X4YUBVEZFITOY4VEYKG7J6WIJDEHABWGA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.582999887 (689h21m42.17977091s) [upload=772064]: Next returned nil segment Apr 4 04:21:44.583065715 (689h21m42.179836738s) [upload=772064]: Waiting for error group managing segments... Apr 4 04:21:44.587179313 (689h21m42.183950339s) [noise=true , address=45.91.116.12:4229 , piece_id=CDXY4QA2BKVNNJUHGL2JPK3MMHTVDPRN64SAQSMZSVQIG755ZOJQ, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:44.587315638 (689h21m42.184086655s) [noise=true , address=45.91.116.13:4158 , piece_id=OX4BJEH3DNMAUXS64QHDHRK75YYEF6DPQJAX7MS3J6K7RUM4GM2Q, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:44.592227274 (689h21m42.188998305s) [noise=true , address=145.79.227.8:4345 , piece_id=2UWIKM3L3JUKNYWPBVJBEGFOEWS3WVM24AY7C2OVBZWDAXDG4I3A, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:44.897574490 (689h21m42.494345523s) [noise=true , address=167.17.72.70:4751 , piece_id=AX7C5IEHMVNPGGAB7WPWUOSFEHVWWA75JGDAH3DAGZVZCVMEZKKQ, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.084711646 (689h21m42.681482669s) [noise=true , address=193.149.164.66:4996 , piece_id=YGOQ6INTPJ3AZW3RSTVH2IWITNFONA7JTW25BIXNBZRRZUN6YRYA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.084884121 (689h21m42.681655134s) [noise=true , address=193.149.164.67:4002 , piece_id=CYJYJ5ZNXT63OT5DBDLW7ML32OIZLGNWMLM4W2F5TPSSJ2QJ2NBA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.089152844 (689h21m42.685923867s) [noise=true , address=193.149.164.70:4115 , piece_id=EHMHJHJ2ZQWAJWBFSHOTAXWE7QJQOIICXDKAK33KMOAF7C7KZ2PA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.103385369 (689h21m42.700156382s) [noise=true , address=193.149.164.68:4062 , piece_id=IM6B4U2ARVUSBX4276OSQPEPQNMWMFSUUOYAEJ3ZG2GLFUKGGQGQ, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.127711185 (689h21m42.724482213s) [noise=true , address=193.149.164.69:4084 , piece_id=BUV3LZ7PDHSGRNKHJSSN2OFQYMIYUELWEPO4PF3DMTBX65S4YPMA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.143513369 (689h21m42.740284381s) [noise=true , address=193.149.164.71:4308 , piece_id=YGF5F7J25BEULKSTR3ZDC3HE4EHVJ3IF4HQCNATVWHITGRB344JA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.307124646 (689h21m42.903895674s) [noise=true , address=167.17.72.68:4656 , piece_id=YFAXDPM4TYBLYL4UR3Z6FDCDJXU34J7XRURDGLXZAMUOYYZL56IA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.517599945 (689h21m43.114370966s) [noise=true , address=145.79.227.33:4898 , piece_id=64X2NB2BPWLP4C5EC53A6EKFPNQCJI6BGEDY6BFE2RFNYEPCDA5Q, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:46.061904485 (689h21m43.658675509s) [noise=true , address=193.149.164.143:4704 , piece_id=PZ2W2J22M5MZIYBFEOJCEOIVAJOJGYCHCQUPDSELS764HZG4OF2A, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:22:44.423237599 (689h22m42.020008631s) [noise=true , address=77.90.31.13:4658 , piece_id=I7GGGCPJIHSQ5RYYOPIXKWAJBOMSRSK3ZDCASMKUPBQPALOJCKXA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: ecclient: upload failed (node:129aongkQE9HMNz8vsNNA5SvKbhsHqdt9UrtSTPJJpP9mkFsboX, address:77.90.31.13:4658): protocol: read tcp 159.223.143.39:50834->77.90.31.13:4658: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 4 04:22:44.423364101 (689h22m42.020135144s) [noise=true , address=77.90.31.17:4783 , piece_id=E2XPGXQAVSYEMBWOIPKHGL5MVVGC2FSLN5KHZQC6DBUAT3VUWOKA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: ecclient: upload failed (node:155fvDXZdpQeHetitv17WCVVrP1bGswdoqJECQLSe66kXRhBPo, address:77.90.31.17:4783): protocol: read tcp 159.223.143.39:45990->77.90.31.17:4783: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 4 04:22:44.423435607 (689h22m42.020206643s) [noise=true , address=77.90.31.16:4750 , piece_id=TI2D4Z2MYSXWBHX3ZVEMRFLI4LW5RDVCENH43S7GEFR7WGIVCI6A, seg_pos={0 0}, upload=772064]: Done uploading piece. err: ecclient: upload failed (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE, address:77.90.31.16:4750): protocol: read tcp 159.223.143.39:34564->77.90.31.16:4750: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 4 04:22:44.423560856 (689h22m42.020332015s) [noise=true , address=77.90.31.18:4823 , piece_id=MI45FJCGXJ2CIL4OOVHEUB7TJXEGH6WELCSTKR6JD753YLF5UN4Q, seg_pos={0 0}, upload=772064]: Done uploading piece. err: ecclient: upload failed (node:12uUCUDKZDwj25UcEKQ4WiTuQFMRbWw2oymGAdUdMX8wcUSd7jS, address:77.90.31.18:4823): protocol: read tcp 159.223.143.39:40028->77.90.31.18:4823: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 4 04:22:44.423793037 (689h22m42.020564071s) [noise=true , address=77.90.31.15:4709 , piece_id=HQLXOXRSN7R5WC5K2W7X4YUBVEZFITOY4VEYKG7J6WIJDEHABWGA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: ecclient: upload failed (node:1CrHsLV4vge154tJjMvbo5oxnphfukuXQBKfvTjA43n6CJM2Ah, address:77.90.31.15:4709): protocol: read tcp 159.223.143.39:59234->77.90.31.15:4709: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 4 04:22:44.423864095 (689h22m42.020635106s) [seg_pos={0 0}, upload=772064]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 4 04:22:44.451342916 (689h22m42.048113939s) [upload=772064]: Done uploading segments... Apr 4 04:22:44.451381831 (689h22m42.048152855s) [upload=772064]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1.352056317s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 693.285505ms trace: https://jaeger.dev.storj.io/trace/670ee369f5f2d334 FAIL (2026-04-04 04:36:40 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 693.411299ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 1.337218721s trace: https://jaeger.dev.storj.io/trace/5cce09de575b8506 FAIL (2026-04-04 04:36:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 6881916s) [upload=772064]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 4 04:21:44.460130782 (689h21m42.056901815s) [upload=772064]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 4 04:21:44.500551580 (689h21m42.097322604s) [seg_pos={0 0}, upload=772064]: Begin upload segment... Apr 4 04:21:44.500628241 (689h21m42.097399263s) [seg_pos={0 0}, upload=772064]: Done begin upload segment. Apr 4 04:21:44.500668892 (689h21m42.097439922s) [noise=true , address=193.149.164.143:4704 , piece_id=PZ2W2J22M5MZIYBFEOJCEOIVAJOJGYCHCQUPDSELS764HZG4OF2A, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.500792978 (689h21m42.097564014s) [noise=true , address=45.91.116.13:4158 , piece_id=OX4BJEH3DNMAUXS64QHDHRK75YYEF6DPQJAX7MS3J6K7RUM4GM2Q, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.501687053 (689h21m42.09845809s) [noise=true , address=77.90.31.17:4783 , piece_id=E2XPGXQAVSYEMBWOIPKHGL5MVVGC2FSLN5KHZQC6DBUAT3VUWOKA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.501828492 (689h21m42.098599515s) [noise=true , address=193.149.164.69:4084 , piece_id=BUV3LZ7PDHSGRNKHJSSN2OFQYMIYUELWEPO4PF3DMTBX65S4YPMA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.502343164 (689h21m42.099114198s) [noise=true , address=193.149.164.67:4002 , piece_id=CYJYJ5ZNXT63OT5DBDLW7ML32OIZLGNWMLM4W2F5TPSSJ2QJ2NBA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.502900735 (689h21m42.099671764s) [noise=true , address=77.90.31.18:4823 , piece_id=MI45FJCGXJ2CIL4OOVHEUB7TJXEGH6WELCSTKR6JD753YLF5UN4Q, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.503495842 (689h21m42.100266868s) [noise=true , address=193.149.164.68:4062 , piece_id=IM6B4U2ARVUSBX4276OSQPEPQNMWMFSUUOYAEJ3ZG2GLFUKGGQGQ, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.503796813 (689h21m42.100567839s) [noise=true , address=77.90.31.13:4658 , piece_id=I7GGGCPJIHSQ5RYYOPIXKWAJBOMSRSK3ZDCASMKUPBQPALOJCKXA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.504006258 (689h21m42.100777287s) [noise=true , address=77.90.31.16:4750 , piece_id=TI2D4Z2MYSXWBHX3ZVEMRFLI4LW5RDVCENH43S7GEFR7WGIVCI6A, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.504542000 (689h21m42.101313035s) [noise=true , address=145.79.227.33:4898 , piece_id=64X2NB2BPWLP4C5EC53A6EKFPNQCJI6BGEDY6BFE2RFNYEPCDA5Q, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.504560079 (689h21m42.101331115s) [noise=true , address=45.91.116.12:4229 , piece_id=CDXY4QA2BKVNNJUHGL2JPK3MMHTVDPRN64SAQSMZSVQIG755ZOJQ, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.505047220 (689h21m42.101818243s) [noise=true , address=193.149.164.66:4996 , piece_id=YGOQ6INTPJ3AZW3RSTVH2IWITNFONA7JTW25BIXNBZRRZUN6YRYA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.505183875 (689h21m42.10195491s) [noise=true , address=167.17.72.70:4751 , piece_id=AX7C5IEHMVNPGGAB7WPWUOSFEHVWWA75JGDAH3DAGZVZCVMEZKKQ, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.505619570 (689h21m42.102390603s) [noise=true , address=167.17.72.68:4656 , piece_id=YFAXDPM4TYBLYL4UR3Z6FDCDJXU34J7XRURDGLXZAMUOYYZL56IA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.506074686 (689h21m42.102845722s) [noise=true , address=193.149.164.70:4115 , piece_id=EHMHJHJ2ZQWAJWBFSHOTAXWE7QJQOIICXDKAK33KMOAF7C7KZ2PA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.506877697 (689h21m42.103648727s) [noise=true , address=145.79.227.8:4345 , piece_id=2UWIKM3L3JUKNYWPBVJBEGFOEWS3WVM24AY7C2OVBZWDAXDG4I3A, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.507462874 (689h21m42.104233903s) [noise=true , address=193.149.164.71:4308 , piece_id=YGF5F7J25BEULKSTR3ZDC3HE4EHVJ3IF4HQCNATVWHITGRB344JA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.507975323 (689h21m42.10474639s) [noise=true , address=77.90.31.15:4709 , piece_id=HQLXOXRSN7R5WC5K2W7X4YUBVEZFITOY4VEYKG7J6WIJDEHABWGA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.582999887 (689h21m42.17977091s) [upload=772064]: Next returned nil segment Apr 4 04:21:44.583065715 (689h21m42.179836738s) [upload=772064]: Waiting for error group managing segments... Apr 4 04:21:44.587179313 (689h21m42.183950339s) [noise=true , address=45.91.116.12:4229 , piece_id=CDXY4QA2BKVNNJUHGL2JPK3MMHTVDPRN64SAQSMZSVQIG755ZOJQ, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:44.587315638 (689h21m42.184086655s) [noise=true , address=45.91.116.13:4158 , piece_id=OX4BJEH3DNMAUXS64QHDHRK75YYEF6DPQJAX7MS3J6K7RUM4GM2Q, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:44.592227274 (689h21m42.188998305s) [noise=true , address=145.79.227.8:4345 , piece_id=2UWIKM3L3JUKNYWPBVJBEGFOEWS3WVM24AY7C2OVBZWDAXDG4I3A, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:44.897574490 (689h21m42.494345523s) [noise=true , address=167.17.72.70:4751 , piece_id=AX7C5IEHMVNPGGAB7WPWUOSFEHVWWA75JGDAH3DAGZVZCVMEZKKQ, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.084711646 (689h21m42.681482669s) [noise=true , address=193.149.164.66:4996 , piece_id=YGOQ6INTPJ3AZW3RSTVH2IWITNFONA7JTW25BIXNBZRRZUN6YRYA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.084884121 (689h21m42.681655134s) [noise=true , address=193.149.164.67:4002 , piece_id=CYJYJ5ZNXT63OT5DBDLW7ML32OIZLGNWMLM4W2F5TPSSJ2QJ2NBA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.089152844 (689h21m42.685923867s) [noise=true , address=193.149.164.70:4115 , piece_id=EHMHJHJ2ZQWAJWBFSHOTAXWE7QJQOIICXDKAK33KMOAF7C7KZ2PA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.103385369 (689h21m42.700156382s) [noise=true , address=193.149.164.68:4062 , piece_id=IM6B4U2ARVUSBX4276OSQPEPQNMWMFSUUOYAEJ3ZG2GLFUKGGQGQ, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.127711185 (689h21m42.724482213s) [noise=true , address=193.149.164.69:4084 , piece_id=BUV3LZ7PDHSGRNKHJSSN2OFQYMIYUELWEPO4PF3DMTBX65S4YPMA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.143513369 (689h21m42.740284381s) [noise=true , address=193.149.164.71:4308 , piece_id=YGF5F7J25BEULKSTR3ZDC3HE4EHVJ3IF4HQCNATVWHITGRB344JA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.307124646 (689h21m42.903895674s) [noise=true , address=167.17.72.68:4656 , piece_id=YFAXDPM4TYBLYL4UR3Z6FDCDJXU34J7XRURDGLXZAMUOYYZL56IA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.517599945 (689h21m43.114370966s) [noise=true , address=145.79.227.33:4898 , piece_id=64X2NB2BPWLP4C5EC53A6EKFPNQCJI6BGEDY6BFE2RFNYEPCDA5Q, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:46.061904485 (689h21m43.658675509s) [noise=true , address=193.149.164.143:4704 , piece_id=PZ2W2J22M5MZIYBFEOJCEOIVAJOJGYCHCQUPDSELS764HZG4OF2A, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:22:44.423237599 (689h22m42.020008631s) [noise=true , address=77.90.31.13:4658 , piece_id=I7GGGCPJIHSQ5RYYOPIXKWAJBOMSRSK3ZDCASMKUPBQPALOJCKXA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: ecclient: upload failed (node:129aongkQE9HMNz8vsNNA5SvKbhsHqdt9UrtSTPJJpP9mkFsboX, address:77.90.31.13:4658): protocol: read tcp 159.223.143.39:50834->77.90.31.13:4658: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 4 04:22:44.423364101 (689h22m42.020135144s) [noise=true , address=77.90.31.17:4783 , piece_id=E2XPGXQAVSYEMBWOIPKHGL5MVVGC2FSLN5KHZQC6DBUAT3VUWOKA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: ecclient: upload failed (node:155fvDXZdpQeHetitv17WCVVrP1bGswdoqJECQLSe66kXRhBPo, address:77.90.31.17:4783): protocol: read tcp 159.223.143.39:45990->77.90.31.17:4783: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 4 04:22:44.423435607 (689h22m42.020206643s) [noise=true , address=77.90.31.16:4750 , piece_id=TI2D4Z2MYSXWBHX3ZVEMRFLI4LW5RDVCENH43S7GEFR7WGIVCI6A, seg_pos={0 0}, upload=772064]: Done uploading piece. err: ecclient: upload failed (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE, address:77.90.31.16:4750): protocol: read tcp 159.223.143.39:34564->77.90.31.16:4750: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 4 04:22:44.423560856 (689h22m42.020332015s) [noise=true , address=77.90.31.18:4823 , piece_id=MI45FJCGXJ2CIL4OOVHEUB7TJXEGH6WELCSTKR6JD753YLF5UN4Q, seg_pos={0 0}, upload=772064]: Done uploading piece. err: ecclient: upload failed (node:12uUCUDKZDwj25UcEKQ4WiTuQFMRbWw2oymGAdUdMX8wcUSd7jS, address:77.90.31.18:4823): protocol: read tcp 159.223.143.39:40028->77.90.31.18:4823: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 4 04:22:44.423793037 (689h22m42.020564071s) [noise=true , address=77.90.31.15:4709 , piece_id=HQLXOXRSN7R5WC5K2W7X4YUBVEZFITOY4VEYKG7J6WIJDEHABWGA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: ecclient: upload failed (node:1CrHsLV4vge154tJjMvbo5oxnphfukuXQBKfvTjA43n6CJM2Ah, address:77.90.31.15:4709): protocol: read tcp 159.223.143.39:59234->77.90.31.15:4709: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 4 04:22:44.423864095 (689h22m42.020635106s) [seg_pos={0 0}, upload=772064]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 4 04:22:44.451342916 (689h22m42.048113939s) [upload=772064]: Done uploading segments... Apr 4 04:22:44.451381831 (689h22m42.048152855s) [upload=772064]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1.337254685s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 5 out of 60 is failed (threshold=6) [..........................X..X.........XXX..................] duration: 678.475006ms trace: https://jaeger.dev.storj.io/trace/3304a822f59853ad FAIL (2026-04-04 04:36:30 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 678.503104ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 595.711887ms trace: https://jaeger.dev.storj.io/trace/670ee369f5f2d334 FAIL (2026-04-04 04:36:07 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 595.744859ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 1.396343448s trace: https://jaeger.dev.storj.io/trace/5cce09de575b8506 FAIL (2026-04-04 04:36:02 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 6881916s) [upload=772064]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 4 04:21:44.460130782 (689h21m42.056901815s) [upload=772064]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 4 04:21:44.500551580 (689h21m42.097322604s) [seg_pos={0 0}, upload=772064]: Begin upload segment... Apr 4 04:21:44.500628241 (689h21m42.097399263s) [seg_pos={0 0}, upload=772064]: Done begin upload segment. Apr 4 04:21:44.500668892 (689h21m42.097439922s) [noise=true , address=193.149.164.143:4704 , piece_id=PZ2W2J22M5MZIYBFEOJCEOIVAJOJGYCHCQUPDSELS764HZG4OF2A, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.500792978 (689h21m42.097564014s) [noise=true , address=45.91.116.13:4158 , piece_id=OX4BJEH3DNMAUXS64QHDHRK75YYEF6DPQJAX7MS3J6K7RUM4GM2Q, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.501687053 (689h21m42.09845809s) [noise=true , address=77.90.31.17:4783 , piece_id=E2XPGXQAVSYEMBWOIPKHGL5MVVGC2FSLN5KHZQC6DBUAT3VUWOKA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.501828492 (689h21m42.098599515s) [noise=true , address=193.149.164.69:4084 , piece_id=BUV3LZ7PDHSGRNKHJSSN2OFQYMIYUELWEPO4PF3DMTBX65S4YPMA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.502343164 (689h21m42.099114198s) [noise=true , address=193.149.164.67:4002 , piece_id=CYJYJ5ZNXT63OT5DBDLW7ML32OIZLGNWMLM4W2F5TPSSJ2QJ2NBA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.502900735 (689h21m42.099671764s) [noise=true , address=77.90.31.18:4823 , piece_id=MI45FJCGXJ2CIL4OOVHEUB7TJXEGH6WELCSTKR6JD753YLF5UN4Q, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.503495842 (689h21m42.100266868s) [noise=true , address=193.149.164.68:4062 , piece_id=IM6B4U2ARVUSBX4276OSQPEPQNMWMFSUUOYAEJ3ZG2GLFUKGGQGQ, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.503796813 (689h21m42.100567839s) [noise=true , address=77.90.31.13:4658 , piece_id=I7GGGCPJIHSQ5RYYOPIXKWAJBOMSRSK3ZDCASMKUPBQPALOJCKXA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.504006258 (689h21m42.100777287s) [noise=true , address=77.90.31.16:4750 , piece_id=TI2D4Z2MYSXWBHX3ZVEMRFLI4LW5RDVCENH43S7GEFR7WGIVCI6A, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.504542000 (689h21m42.101313035s) [noise=true , address=145.79.227.33:4898 , piece_id=64X2NB2BPWLP4C5EC53A6EKFPNQCJI6BGEDY6BFE2RFNYEPCDA5Q, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.504560079 (689h21m42.101331115s) [noise=true , address=45.91.116.12:4229 , piece_id=CDXY4QA2BKVNNJUHGL2JPK3MMHTVDPRN64SAQSMZSVQIG755ZOJQ, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.505047220 (689h21m42.101818243s) [noise=true , address=193.149.164.66:4996 , piece_id=YGOQ6INTPJ3AZW3RSTVH2IWITNFONA7JTW25BIXNBZRRZUN6YRYA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.505183875 (689h21m42.10195491s) [noise=true , address=167.17.72.70:4751 , piece_id=AX7C5IEHMVNPGGAB7WPWUOSFEHVWWA75JGDAH3DAGZVZCVMEZKKQ, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.505619570 (689h21m42.102390603s) [noise=true , address=167.17.72.68:4656 , piece_id=YFAXDPM4TYBLYL4UR3Z6FDCDJXU34J7XRURDGLXZAMUOYYZL56IA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.506074686 (689h21m42.102845722s) [noise=true , address=193.149.164.70:4115 , piece_id=EHMHJHJ2ZQWAJWBFSHOTAXWE7QJQOIICXDKAK33KMOAF7C7KZ2PA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.506877697 (689h21m42.103648727s) [noise=true , address=145.79.227.8:4345 , piece_id=2UWIKM3L3JUKNYWPBVJBEGFOEWS3WVM24AY7C2OVBZWDAXDG4I3A, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.507462874 (689h21m42.104233903s) [noise=true , address=193.149.164.71:4308 , piece_id=YGF5F7J25BEULKSTR3ZDC3HE4EHVJ3IF4HQCNATVWHITGRB344JA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.507975323 (689h21m42.10474639s) [noise=true , address=77.90.31.15:4709 , piece_id=HQLXOXRSN7R5WC5K2W7X4YUBVEZFITOY4VEYKG7J6WIJDEHABWGA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.582999887 (689h21m42.17977091s) [upload=772064]: Next returned nil segment Apr 4 04:21:44.583065715 (689h21m42.179836738s) [upload=772064]: Waiting for error group managing segments... Apr 4 04:21:44.587179313 (689h21m42.183950339s) [noise=true , address=45.91.116.12:4229 , piece_id=CDXY4QA2BKVNNJUHGL2JPK3MMHTVDPRN64SAQSMZSVQIG755ZOJQ, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:44.587315638 (689h21m42.184086655s) [noise=true , address=45.91.116.13:4158 , piece_id=OX4BJEH3DNMAUXS64QHDHRK75YYEF6DPQJAX7MS3J6K7RUM4GM2Q, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:44.592227274 (689h21m42.188998305s) [noise=true , address=145.79.227.8:4345 , piece_id=2UWIKM3L3JUKNYWPBVJBEGFOEWS3WVM24AY7C2OVBZWDAXDG4I3A, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:44.897574490 (689h21m42.494345523s) [noise=true , address=167.17.72.70:4751 , piece_id=AX7C5IEHMVNPGGAB7WPWUOSFEHVWWA75JGDAH3DAGZVZCVMEZKKQ, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.084711646 (689h21m42.681482669s) [noise=true , address=193.149.164.66:4996 , piece_id=YGOQ6INTPJ3AZW3RSTVH2IWITNFONA7JTW25BIXNBZRRZUN6YRYA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.084884121 (689h21m42.681655134s) [noise=true , address=193.149.164.67:4002 , piece_id=CYJYJ5ZNXT63OT5DBDLW7ML32OIZLGNWMLM4W2F5TPSSJ2QJ2NBA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.089152844 (689h21m42.685923867s) [noise=true , address=193.149.164.70:4115 , piece_id=EHMHJHJ2ZQWAJWBFSHOTAXWE7QJQOIICXDKAK33KMOAF7C7KZ2PA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.103385369 (689h21m42.700156382s) [noise=true , address=193.149.164.68:4062 , piece_id=IM6B4U2ARVUSBX4276OSQPEPQNMWMFSUUOYAEJ3ZG2GLFUKGGQGQ, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.127711185 (689h21m42.724482213s) [noise=true , address=193.149.164.69:4084 , piece_id=BUV3LZ7PDHSGRNKHJSSN2OFQYMIYUELWEPO4PF3DMTBX65S4YPMA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.143513369 (689h21m42.740284381s) [noise=true , address=193.149.164.71:4308 , piece_id=YGF5F7J25BEULKSTR3ZDC3HE4EHVJ3IF4HQCNATVWHITGRB344JA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.307124646 (689h21m42.903895674s) [noise=true , address=167.17.72.68:4656 , piece_id=YFAXDPM4TYBLYL4UR3Z6FDCDJXU34J7XRURDGLXZAMUOYYZL56IA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.517599945 (689h21m43.114370966s) [noise=true , address=145.79.227.33:4898 , piece_id=64X2NB2BPWLP4C5EC53A6EKFPNQCJI6BGEDY6BFE2RFNYEPCDA5Q, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:46.061904485 (689h21m43.658675509s) [noise=true , address=193.149.164.143:4704 , piece_id=PZ2W2J22M5MZIYBFEOJCEOIVAJOJGYCHCQUPDSELS764HZG4OF2A, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:22:44.423237599 (689h22m42.020008631s) [noise=true , address=77.90.31.13:4658 , piece_id=I7GGGCPJIHSQ5RYYOPIXKWAJBOMSRSK3ZDCASMKUPBQPALOJCKXA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: ecclient: upload failed (node:129aongkQE9HMNz8vsNNA5SvKbhsHqdt9UrtSTPJJpP9mkFsboX, address:77.90.31.13:4658): protocol: read tcp 159.223.143.39:50834->77.90.31.13:4658: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 4 04:22:44.423364101 (689h22m42.020135144s) [noise=true , address=77.90.31.17:4783 , piece_id=E2XPGXQAVSYEMBWOIPKHGL5MVVGC2FSLN5KHZQC6DBUAT3VUWOKA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: ecclient: upload failed (node:155fvDXZdpQeHetitv17WCVVrP1bGswdoqJECQLSe66kXRhBPo, address:77.90.31.17:4783): protocol: read tcp 159.223.143.39:45990->77.90.31.17:4783: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 4 04:22:44.423435607 (689h22m42.020206643s) [noise=true , address=77.90.31.16:4750 , piece_id=TI2D4Z2MYSXWBHX3ZVEMRFLI4LW5RDVCENH43S7GEFR7WGIVCI6A, seg_pos={0 0}, upload=772064]: Done uploading piece. err: ecclient: upload failed (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE, address:77.90.31.16:4750): protocol: read tcp 159.223.143.39:34564->77.90.31.16:4750: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 4 04:22:44.423560856 (689h22m42.020332015s) [noise=true , address=77.90.31.18:4823 , piece_id=MI45FJCGXJ2CIL4OOVHEUB7TJXEGH6WELCSTKR6JD753YLF5UN4Q, seg_pos={0 0}, upload=772064]: Done uploading piece. err: ecclient: upload failed (node:12uUCUDKZDwj25UcEKQ4WiTuQFMRbWw2oymGAdUdMX8wcUSd7jS, address:77.90.31.18:4823): protocol: read tcp 159.223.143.39:40028->77.90.31.18:4823: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 4 04:22:44.423793037 (689h22m42.020564071s) [noise=true , address=77.90.31.15:4709 , piece_id=HQLXOXRSN7R5WC5K2W7X4YUBVEZFITOY4VEYKG7J6WIJDEHABWGA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: ecclient: upload failed (node:1CrHsLV4vge154tJjMvbo5oxnphfukuXQBKfvTjA43n6CJM2Ah, address:77.90.31.15:4709): protocol: read tcp 159.223.143.39:59234->77.90.31.15:4709: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 4 04:22:44.423864095 (689h22m42.020635106s) [seg_pos={0 0}, upload=772064]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 4 04:22:44.451342916 (689h22m42.048113939s) [upload=772064]: Done uploading segments... Apr 4 04:22:44.451381831 (689h22m42.048152855s) [upload=772064]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1.396373054s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 1.236360126s trace: https://jaeger.dev.storj.io/trace/5cce09de575b8506 FAIL (2026-04-04 04:35:39 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 6881916s) [upload=772064]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 4 04:21:44.460130782 (689h21m42.056901815s) [upload=772064]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 4 04:21:44.500551580 (689h21m42.097322604s) [seg_pos={0 0}, upload=772064]: Begin upload segment... Apr 4 04:21:44.500628241 (689h21m42.097399263s) [seg_pos={0 0}, upload=772064]: Done begin upload segment. Apr 4 04:21:44.500668892 (689h21m42.097439922s) [noise=true , address=193.149.164.143:4704 , piece_id=PZ2W2J22M5MZIYBFEOJCEOIVAJOJGYCHCQUPDSELS764HZG4OF2A, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.500792978 (689h21m42.097564014s) [noise=true , address=45.91.116.13:4158 , piece_id=OX4BJEH3DNMAUXS64QHDHRK75YYEF6DPQJAX7MS3J6K7RUM4GM2Q, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.501687053 (689h21m42.09845809s) [noise=true , address=77.90.31.17:4783 , piece_id=E2XPGXQAVSYEMBWOIPKHGL5MVVGC2FSLN5KHZQC6DBUAT3VUWOKA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.501828492 (689h21m42.098599515s) [noise=true , address=193.149.164.69:4084 , piece_id=BUV3LZ7PDHSGRNKHJSSN2OFQYMIYUELWEPO4PF3DMTBX65S4YPMA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.502343164 (689h21m42.099114198s) [noise=true , address=193.149.164.67:4002 , piece_id=CYJYJ5ZNXT63OT5DBDLW7ML32OIZLGNWMLM4W2F5TPSSJ2QJ2NBA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.502900735 (689h21m42.099671764s) [noise=true , address=77.90.31.18:4823 , piece_id=MI45FJCGXJ2CIL4OOVHEUB7TJXEGH6WELCSTKR6JD753YLF5UN4Q, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.503495842 (689h21m42.100266868s) [noise=true , address=193.149.164.68:4062 , piece_id=IM6B4U2ARVUSBX4276OSQPEPQNMWMFSUUOYAEJ3ZG2GLFUKGGQGQ, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.503796813 (689h21m42.100567839s) [noise=true , address=77.90.31.13:4658 , piece_id=I7GGGCPJIHSQ5RYYOPIXKWAJBOMSRSK3ZDCASMKUPBQPALOJCKXA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.504006258 (689h21m42.100777287s) [noise=true , address=77.90.31.16:4750 , piece_id=TI2D4Z2MYSXWBHX3ZVEMRFLI4LW5RDVCENH43S7GEFR7WGIVCI6A, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.504542000 (689h21m42.101313035s) [noise=true , address=145.79.227.33:4898 , piece_id=64X2NB2BPWLP4C5EC53A6EKFPNQCJI6BGEDY6BFE2RFNYEPCDA5Q, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.504560079 (689h21m42.101331115s) [noise=true , address=45.91.116.12:4229 , piece_id=CDXY4QA2BKVNNJUHGL2JPK3MMHTVDPRN64SAQSMZSVQIG755ZOJQ, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.505047220 (689h21m42.101818243s) [noise=true , address=193.149.164.66:4996 , piece_id=YGOQ6INTPJ3AZW3RSTVH2IWITNFONA7JTW25BIXNBZRRZUN6YRYA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.505183875 (689h21m42.10195491s) [noise=true , address=167.17.72.70:4751 , piece_id=AX7C5IEHMVNPGGAB7WPWUOSFEHVWWA75JGDAH3DAGZVZCVMEZKKQ, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.505619570 (689h21m42.102390603s) [noise=true , address=167.17.72.68:4656 , piece_id=YFAXDPM4TYBLYL4UR3Z6FDCDJXU34J7XRURDGLXZAMUOYYZL56IA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.506074686 (689h21m42.102845722s) [noise=true , address=193.149.164.70:4115 , piece_id=EHMHJHJ2ZQWAJWBFSHOTAXWE7QJQOIICXDKAK33KMOAF7C7KZ2PA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.506877697 (689h21m42.103648727s) [noise=true , address=145.79.227.8:4345 , piece_id=2UWIKM3L3JUKNYWPBVJBEGFOEWS3WVM24AY7C2OVBZWDAXDG4I3A, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.507462874 (689h21m42.104233903s) [noise=true , address=193.149.164.71:4308 , piece_id=YGF5F7J25BEULKSTR3ZDC3HE4EHVJ3IF4HQCNATVWHITGRB344JA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.507975323 (689h21m42.10474639s) [noise=true , address=77.90.31.15:4709 , piece_id=HQLXOXRSN7R5WC5K2W7X4YUBVEZFITOY4VEYKG7J6WIJDEHABWGA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.582999887 (689h21m42.17977091s) [upload=772064]: Next returned nil segment Apr 4 04:21:44.583065715 (689h21m42.179836738s) [upload=772064]: Waiting for error group managing segments... Apr 4 04:21:44.587179313 (689h21m42.183950339s) [noise=true , address=45.91.116.12:4229 , piece_id=CDXY4QA2BKVNNJUHGL2JPK3MMHTVDPRN64SAQSMZSVQIG755ZOJQ, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:44.587315638 (689h21m42.184086655s) [noise=true , address=45.91.116.13:4158 , piece_id=OX4BJEH3DNMAUXS64QHDHRK75YYEF6DPQJAX7MS3J6K7RUM4GM2Q, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:44.592227274 (689h21m42.188998305s) [noise=true , address=145.79.227.8:4345 , piece_id=2UWIKM3L3JUKNYWPBVJBEGFOEWS3WVM24AY7C2OVBZWDAXDG4I3A, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:44.897574490 (689h21m42.494345523s) [noise=true , address=167.17.72.70:4751 , piece_id=AX7C5IEHMVNPGGAB7WPWUOSFEHVWWA75JGDAH3DAGZVZCVMEZKKQ, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.084711646 (689h21m42.681482669s) [noise=true , address=193.149.164.66:4996 , piece_id=YGOQ6INTPJ3AZW3RSTVH2IWITNFONA7JTW25BIXNBZRRZUN6YRYA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.084884121 (689h21m42.681655134s) [noise=true , address=193.149.164.67:4002 , piece_id=CYJYJ5ZNXT63OT5DBDLW7ML32OIZLGNWMLM4W2F5TPSSJ2QJ2NBA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.089152844 (689h21m42.685923867s) [noise=true , address=193.149.164.70:4115 , piece_id=EHMHJHJ2ZQWAJWBFSHOTAXWE7QJQOIICXDKAK33KMOAF7C7KZ2PA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.103385369 (689h21m42.700156382s) [noise=true , address=193.149.164.68:4062 , piece_id=IM6B4U2ARVUSBX4276OSQPEPQNMWMFSUUOYAEJ3ZG2GLFUKGGQGQ, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.127711185 (689h21m42.724482213s) [noise=true , address=193.149.164.69:4084 , piece_id=BUV3LZ7PDHSGRNKHJSSN2OFQYMIYUELWEPO4PF3DMTBX65S4YPMA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.143513369 (689h21m42.740284381s) [noise=true , address=193.149.164.71:4308 , piece_id=YGF5F7J25BEULKSTR3ZDC3HE4EHVJ3IF4HQCNATVWHITGRB344JA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.307124646 (689h21m42.903895674s) [noise=true , address=167.17.72.68:4656 , piece_id=YFAXDPM4TYBLYL4UR3Z6FDCDJXU34J7XRURDGLXZAMUOYYZL56IA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.517599945 (689h21m43.114370966s) [noise=true , address=145.79.227.33:4898 , piece_id=64X2NB2BPWLP4C5EC53A6EKFPNQCJI6BGEDY6BFE2RFNYEPCDA5Q, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:46.061904485 (689h21m43.658675509s) [noise=true , address=193.149.164.143:4704 , piece_id=PZ2W2J22M5MZIYBFEOJCEOIVAJOJGYCHCQUPDSELS764HZG4OF2A, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:22:44.423237599 (689h22m42.020008631s) [noise=true , address=77.90.31.13:4658 , piece_id=I7GGGCPJIHSQ5RYYOPIXKWAJBOMSRSK3ZDCASMKUPBQPALOJCKXA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: ecclient: upload failed (node:129aongkQE9HMNz8vsNNA5SvKbhsHqdt9UrtSTPJJpP9mkFsboX, address:77.90.31.13:4658): protocol: read tcp 159.223.143.39:50834->77.90.31.13:4658: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 4 04:22:44.423364101 (689h22m42.020135144s) [noise=true , address=77.90.31.17:4783 , piece_id=E2XPGXQAVSYEMBWOIPKHGL5MVVGC2FSLN5KHZQC6DBUAT3VUWOKA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: ecclient: upload failed (node:155fvDXZdpQeHetitv17WCVVrP1bGswdoqJECQLSe66kXRhBPo, address:77.90.31.17:4783): protocol: read tcp 159.223.143.39:45990->77.90.31.17:4783: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 4 04:22:44.423435607 (689h22m42.020206643s) [noise=true , address=77.90.31.16:4750 , piece_id=TI2D4Z2MYSXWBHX3ZVEMRFLI4LW5RDVCENH43S7GEFR7WGIVCI6A, seg_pos={0 0}, upload=772064]: Done uploading piece. err: ecclient: upload failed (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE, address:77.90.31.16:4750): protocol: read tcp 159.223.143.39:34564->77.90.31.16:4750: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 4 04:22:44.423560856 (689h22m42.020332015s) [noise=true , address=77.90.31.18:4823 , piece_id=MI45FJCGXJ2CIL4OOVHEUB7TJXEGH6WELCSTKR6JD753YLF5UN4Q, seg_pos={0 0}, upload=772064]: Done uploading piece. err: ecclient: upload failed (node:12uUCUDKZDwj25UcEKQ4WiTuQFMRbWw2oymGAdUdMX8wcUSd7jS, address:77.90.31.18:4823): protocol: read tcp 159.223.143.39:40028->77.90.31.18:4823: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 4 04:22:44.423793037 (689h22m42.020564071s) [noise=true , address=77.90.31.15:4709 , piece_id=HQLXOXRSN7R5WC5K2W7X4YUBVEZFITOY4VEYKG7J6WIJDEHABWGA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: ecclient: upload failed (node:1CrHsLV4vge154tJjMvbo5oxnphfukuXQBKfvTjA43n6CJM2Ah, address:77.90.31.15:4709): protocol: read tcp 159.223.143.39:59234->77.90.31.15:4709: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 4 04:22:44.423864095 (689h22m42.020635106s) [seg_pos={0 0}, upload=772064]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 4 04:22:44.451342916 (689h22m42.048113939s) [upload=772064]: Done uploading segments... Apr 4 04:22:44.451381831 (689h22m42.048152855s) [upload=772064]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1.236391995s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 5 out of 60 is failed (threshold=6) [...........................X..X.........XXX.................] duration: 652.013408ms trace: https://jaeger.dev.storj.io/trace/3304a822f59853ad FAIL (2026-04-04 04:35:18 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 652.100249ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 636.032609ms trace: https://jaeger.dev.storj.io/trace/670ee369f5f2d334 FAIL (2026-04-04 04:35:01 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 636.057823ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 1.534581541s trace: https://jaeger.dev.storj.io/trace/5cce09de575b8506 FAIL (2026-04-04 04:34:45 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 6881916s) [upload=772064]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 4 04:21:44.460130782 (689h21m42.056901815s) [upload=772064]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 4 04:21:44.500551580 (689h21m42.097322604s) [seg_pos={0 0}, upload=772064]: Begin upload segment... Apr 4 04:21:44.500628241 (689h21m42.097399263s) [seg_pos={0 0}, upload=772064]: Done begin upload segment. Apr 4 04:21:44.500668892 (689h21m42.097439922s) [noise=true , address=193.149.164.143:4704 , piece_id=PZ2W2J22M5MZIYBFEOJCEOIVAJOJGYCHCQUPDSELS764HZG4OF2A, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.500792978 (689h21m42.097564014s) [noise=true , address=45.91.116.13:4158 , piece_id=OX4BJEH3DNMAUXS64QHDHRK75YYEF6DPQJAX7MS3J6K7RUM4GM2Q, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.501687053 (689h21m42.09845809s) [noise=true , address=77.90.31.17:4783 , piece_id=E2XPGXQAVSYEMBWOIPKHGL5MVVGC2FSLN5KHZQC6DBUAT3VUWOKA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.501828492 (689h21m42.098599515s) [noise=true , address=193.149.164.69:4084 , piece_id=BUV3LZ7PDHSGRNKHJSSN2OFQYMIYUELWEPO4PF3DMTBX65S4YPMA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.502343164 (689h21m42.099114198s) [noise=true , address=193.149.164.67:4002 , piece_id=CYJYJ5ZNXT63OT5DBDLW7ML32OIZLGNWMLM4W2F5TPSSJ2QJ2NBA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.502900735 (689h21m42.099671764s) [noise=true , address=77.90.31.18:4823 , piece_id=MI45FJCGXJ2CIL4OOVHEUB7TJXEGH6WELCSTKR6JD753YLF5UN4Q, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.503495842 (689h21m42.100266868s) [noise=true , address=193.149.164.68:4062 , piece_id=IM6B4U2ARVUSBX4276OSQPEPQNMWMFSUUOYAEJ3ZG2GLFUKGGQGQ, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.503796813 (689h21m42.100567839s) [noise=true , address=77.90.31.13:4658 , piece_id=I7GGGCPJIHSQ5RYYOPIXKWAJBOMSRSK3ZDCASMKUPBQPALOJCKXA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.504006258 (689h21m42.100777287s) [noise=true , address=77.90.31.16:4750 , piece_id=TI2D4Z2MYSXWBHX3ZVEMRFLI4LW5RDVCENH43S7GEFR7WGIVCI6A, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.504542000 (689h21m42.101313035s) [noise=true , address=145.79.227.33:4898 , piece_id=64X2NB2BPWLP4C5EC53A6EKFPNQCJI6BGEDY6BFE2RFNYEPCDA5Q, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.504560079 (689h21m42.101331115s) [noise=true , address=45.91.116.12:4229 , piece_id=CDXY4QA2BKVNNJUHGL2JPK3MMHTVDPRN64SAQSMZSVQIG755ZOJQ, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.505047220 (689h21m42.101818243s) [noise=true , address=193.149.164.66:4996 , piece_id=YGOQ6INTPJ3AZW3RSTVH2IWITNFONA7JTW25BIXNBZRRZUN6YRYA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.505183875 (689h21m42.10195491s) [noise=true , address=167.17.72.70:4751 , piece_id=AX7C5IEHMVNPGGAB7WPWUOSFEHVWWA75JGDAH3DAGZVZCVMEZKKQ, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.505619570 (689h21m42.102390603s) [noise=true , address=167.17.72.68:4656 , piece_id=YFAXDPM4TYBLYL4UR3Z6FDCDJXU34J7XRURDGLXZAMUOYYZL56IA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.506074686 (689h21m42.102845722s) [noise=true , address=193.149.164.70:4115 , piece_id=EHMHJHJ2ZQWAJWBFSHOTAXWE7QJQOIICXDKAK33KMOAF7C7KZ2PA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.506877697 (689h21m42.103648727s) [noise=true , address=145.79.227.8:4345 , piece_id=2UWIKM3L3JUKNYWPBVJBEGFOEWS3WVM24AY7C2OVBZWDAXDG4I3A, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.507462874 (689h21m42.104233903s) [noise=true , address=193.149.164.71:4308 , piece_id=YGF5F7J25BEULKSTR3ZDC3HE4EHVJ3IF4HQCNATVWHITGRB344JA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.507975323 (689h21m42.10474639s) [noise=true , address=77.90.31.15:4709 , piece_id=HQLXOXRSN7R5WC5K2W7X4YUBVEZFITOY4VEYKG7J6WIJDEHABWGA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.582999887 (689h21m42.17977091s) [upload=772064]: Next returned nil segment Apr 4 04:21:44.583065715 (689h21m42.179836738s) [upload=772064]: Waiting for error group managing segments... Apr 4 04:21:44.587179313 (689h21m42.183950339s) [noise=true , address=45.91.116.12:4229 , piece_id=CDXY4QA2BKVNNJUHGL2JPK3MMHTVDPRN64SAQSMZSVQIG755ZOJQ, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:44.587315638 (689h21m42.184086655s) [noise=true , address=45.91.116.13:4158 , piece_id=OX4BJEH3DNMAUXS64QHDHRK75YYEF6DPQJAX7MS3J6K7RUM4GM2Q, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:44.592227274 (689h21m42.188998305s) [noise=true , address=145.79.227.8:4345 , piece_id=2UWIKM3L3JUKNYWPBVJBEGFOEWS3WVM24AY7C2OVBZWDAXDG4I3A, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:44.897574490 (689h21m42.494345523s) [noise=true , address=167.17.72.70:4751 , piece_id=AX7C5IEHMVNPGGAB7WPWUOSFEHVWWA75JGDAH3DAGZVZCVMEZKKQ, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.084711646 (689h21m42.681482669s) [noise=true , address=193.149.164.66:4996 , piece_id=YGOQ6INTPJ3AZW3RSTVH2IWITNFONA7JTW25BIXNBZRRZUN6YRYA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.084884121 (689h21m42.681655134s) [noise=true , address=193.149.164.67:4002 , piece_id=CYJYJ5ZNXT63OT5DBDLW7ML32OIZLGNWMLM4W2F5TPSSJ2QJ2NBA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.089152844 (689h21m42.685923867s) [noise=true , address=193.149.164.70:4115 , piece_id=EHMHJHJ2ZQWAJWBFSHOTAXWE7QJQOIICXDKAK33KMOAF7C7KZ2PA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.103385369 (689h21m42.700156382s) [noise=true , address=193.149.164.68:4062 , piece_id=IM6B4U2ARVUSBX4276OSQPEPQNMWMFSUUOYAEJ3ZG2GLFUKGGQGQ, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.127711185 (689h21m42.724482213s) [noise=true , address=193.149.164.69:4084 , piece_id=BUV3LZ7PDHSGRNKHJSSN2OFQYMIYUELWEPO4PF3DMTBX65S4YPMA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.143513369 (689h21m42.740284381s) [noise=true , address=193.149.164.71:4308 , piece_id=YGF5F7J25BEULKSTR3ZDC3HE4EHVJ3IF4HQCNATVWHITGRB344JA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.307124646 (689h21m42.903895674s) [noise=true , address=167.17.72.68:4656 , piece_id=YFAXDPM4TYBLYL4UR3Z6FDCDJXU34J7XRURDGLXZAMUOYYZL56IA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.517599945 (689h21m43.114370966s) [noise=true , address=145.79.227.33:4898 , piece_id=64X2NB2BPWLP4C5EC53A6EKFPNQCJI6BGEDY6BFE2RFNYEPCDA5Q, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:46.061904485 (689h21m43.658675509s) [noise=true , address=193.149.164.143:4704 , piece_id=PZ2W2J22M5MZIYBFEOJCEOIVAJOJGYCHCQUPDSELS764HZG4OF2A, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:22:44.423237599 (689h22m42.020008631s) [noise=true , address=77.90.31.13:4658 , piece_id=I7GGGCPJIHSQ5RYYOPIXKWAJBOMSRSK3ZDCASMKUPBQPALOJCKXA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: ecclient: upload failed (node:129aongkQE9HMNz8vsNNA5SvKbhsHqdt9UrtSTPJJpP9mkFsboX, address:77.90.31.13:4658): protocol: read tcp 159.223.143.39:50834->77.90.31.13:4658: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 4 04:22:44.423364101 (689h22m42.020135144s) [noise=true , address=77.90.31.17:4783 , piece_id=E2XPGXQAVSYEMBWOIPKHGL5MVVGC2FSLN5KHZQC6DBUAT3VUWOKA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: ecclient: upload failed (node:155fvDXZdpQeHetitv17WCVVrP1bGswdoqJECQLSe66kXRhBPo, address:77.90.31.17:4783): protocol: read tcp 159.223.143.39:45990->77.90.31.17:4783: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 4 04:22:44.423435607 (689h22m42.020206643s) [noise=true , address=77.90.31.16:4750 , piece_id=TI2D4Z2MYSXWBHX3ZVEMRFLI4LW5RDVCENH43S7GEFR7WGIVCI6A, seg_pos={0 0}, upload=772064]: Done uploading piece. err: ecclient: upload failed (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE, address:77.90.31.16:4750): protocol: read tcp 159.223.143.39:34564->77.90.31.16:4750: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 4 04:22:44.423560856 (689h22m42.020332015s) [noise=true , address=77.90.31.18:4823 , piece_id=MI45FJCGXJ2CIL4OOVHEUB7TJXEGH6WELCSTKR6JD753YLF5UN4Q, seg_pos={0 0}, upload=772064]: Done uploading piece. err: ecclient: upload failed (node:12uUCUDKZDwj25UcEKQ4WiTuQFMRbWw2oymGAdUdMX8wcUSd7jS, address:77.90.31.18:4823): protocol: read tcp 159.223.143.39:40028->77.90.31.18:4823: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 4 04:22:44.423793037 (689h22m42.020564071s) [noise=true , address=77.90.31.15:4709 , piece_id=HQLXOXRSN7R5WC5K2W7X4YUBVEZFITOY4VEYKG7J6WIJDEHABWGA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: ecclient: upload failed (node:1CrHsLV4vge154tJjMvbo5oxnphfukuXQBKfvTjA43n6CJM2Ah, address:77.90.31.15:4709): protocol: read tcp 159.223.143.39:59234->77.90.31.15:4709: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 4 04:22:44.423864095 (689h22m42.020635106s) [seg_pos={0 0}, upload=772064]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 4 04:22:44.451342916 (689h22m42.048113939s) [upload=772064]: Done uploading segments... Apr 4 04:22:44.451381831 (689h22m42.048152855s) [upload=772064]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1.534614338s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 5 out of 60 is failed (threshold=6) [............................X..X.........XXX................] duration: 570.727217ms trace: https://jaeger.dev.storj.io/trace/3304a822f59853ad FAIL (2026-04-04 04:34:39 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 570.921802ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 565.56167ms trace: https://jaeger.dev.storj.io/trace/670ee369f5f2d334 FAIL (2026-04-04 04:34:39 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 565.597688ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 5 out of 60 is failed (threshold=6) [.............................X..X.........XXX...............] duration: 657.825695ms trace: https://jaeger.dev.storj.io/trace/3304a822f59853ad FAIL (2026-04-04 04:34:16 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 657.870396ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 5 out of 60 is failed (threshold=6) [..............................X..X.........XXX..............] duration: 528.656438ms trace: https://jaeger.dev.storj.io/trace/3304a822f59853ad FAIL (2026-04-04 04:33:40 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 528.683078ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 1.30319526s trace: https://jaeger.dev.storj.io/trace/5cce09de575b8506 FAIL (2026-04-04 04:33:39 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 6881916s) [upload=772064]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 4 04:21:44.460130782 (689h21m42.056901815s) [upload=772064]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 4 04:21:44.500551580 (689h21m42.097322604s) [seg_pos={0 0}, upload=772064]: Begin upload segment... Apr 4 04:21:44.500628241 (689h21m42.097399263s) [seg_pos={0 0}, upload=772064]: Done begin upload segment. Apr 4 04:21:44.500668892 (689h21m42.097439922s) [noise=true , address=193.149.164.143:4704 , piece_id=PZ2W2J22M5MZIYBFEOJCEOIVAJOJGYCHCQUPDSELS764HZG4OF2A, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.500792978 (689h21m42.097564014s) [noise=true , address=45.91.116.13:4158 , piece_id=OX4BJEH3DNMAUXS64QHDHRK75YYEF6DPQJAX7MS3J6K7RUM4GM2Q, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.501687053 (689h21m42.09845809s) [noise=true , address=77.90.31.17:4783 , piece_id=E2XPGXQAVSYEMBWOIPKHGL5MVVGC2FSLN5KHZQC6DBUAT3VUWOKA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.501828492 (689h21m42.098599515s) [noise=true , address=193.149.164.69:4084 , piece_id=BUV3LZ7PDHSGRNKHJSSN2OFQYMIYUELWEPO4PF3DMTBX65S4YPMA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.502343164 (689h21m42.099114198s) [noise=true , address=193.149.164.67:4002 , piece_id=CYJYJ5ZNXT63OT5DBDLW7ML32OIZLGNWMLM4W2F5TPSSJ2QJ2NBA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.502900735 (689h21m42.099671764s) [noise=true , address=77.90.31.18:4823 , piece_id=MI45FJCGXJ2CIL4OOVHEUB7TJXEGH6WELCSTKR6JD753YLF5UN4Q, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.503495842 (689h21m42.100266868s) [noise=true , address=193.149.164.68:4062 , piece_id=IM6B4U2ARVUSBX4276OSQPEPQNMWMFSUUOYAEJ3ZG2GLFUKGGQGQ, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.503796813 (689h21m42.100567839s) [noise=true , address=77.90.31.13:4658 , piece_id=I7GGGCPJIHSQ5RYYOPIXKWAJBOMSRSK3ZDCASMKUPBQPALOJCKXA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.504006258 (689h21m42.100777287s) [noise=true , address=77.90.31.16:4750 , piece_id=TI2D4Z2MYSXWBHX3ZVEMRFLI4LW5RDVCENH43S7GEFR7WGIVCI6A, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.504542000 (689h21m42.101313035s) [noise=true , address=145.79.227.33:4898 , piece_id=64X2NB2BPWLP4C5EC53A6EKFPNQCJI6BGEDY6BFE2RFNYEPCDA5Q, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.504560079 (689h21m42.101331115s) [noise=true , address=45.91.116.12:4229 , piece_id=CDXY4QA2BKVNNJUHGL2JPK3MMHTVDPRN64SAQSMZSVQIG755ZOJQ, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.505047220 (689h21m42.101818243s) [noise=true , address=193.149.164.66:4996 , piece_id=YGOQ6INTPJ3AZW3RSTVH2IWITNFONA7JTW25BIXNBZRRZUN6YRYA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.505183875 (689h21m42.10195491s) [noise=true , address=167.17.72.70:4751 , piece_id=AX7C5IEHMVNPGGAB7WPWUOSFEHVWWA75JGDAH3DAGZVZCVMEZKKQ, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.505619570 (689h21m42.102390603s) [noise=true , address=167.17.72.68:4656 , piece_id=YFAXDPM4TYBLYL4UR3Z6FDCDJXU34J7XRURDGLXZAMUOYYZL56IA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.506074686 (689h21m42.102845722s) [noise=true , address=193.149.164.70:4115 , piece_id=EHMHJHJ2ZQWAJWBFSHOTAXWE7QJQOIICXDKAK33KMOAF7C7KZ2PA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.506877697 (689h21m42.103648727s) [noise=true , address=145.79.227.8:4345 , piece_id=2UWIKM3L3JUKNYWPBVJBEGFOEWS3WVM24AY7C2OVBZWDAXDG4I3A, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.507462874 (689h21m42.104233903s) [noise=true , address=193.149.164.71:4308 , piece_id=YGF5F7J25BEULKSTR3ZDC3HE4EHVJ3IF4HQCNATVWHITGRB344JA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.507975323 (689h21m42.10474639s) [noise=true , address=77.90.31.15:4709 , piece_id=HQLXOXRSN7R5WC5K2W7X4YUBVEZFITOY4VEYKG7J6WIJDEHABWGA, seg_pos={0 0}, upload=772064]: Uploading piece... Apr 4 04:21:44.582999887 (689h21m42.17977091s) [upload=772064]: Next returned nil segment Apr 4 04:21:44.583065715 (689h21m42.179836738s) [upload=772064]: Waiting for error group managing segments... Apr 4 04:21:44.587179313 (689h21m42.183950339s) [noise=true , address=45.91.116.12:4229 , piece_id=CDXY4QA2BKVNNJUHGL2JPK3MMHTVDPRN64SAQSMZSVQIG755ZOJQ, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:44.587315638 (689h21m42.184086655s) [noise=true , address=45.91.116.13:4158 , piece_id=OX4BJEH3DNMAUXS64QHDHRK75YYEF6DPQJAX7MS3J6K7RUM4GM2Q, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:44.592227274 (689h21m42.188998305s) [noise=true , address=145.79.227.8:4345 , piece_id=2UWIKM3L3JUKNYWPBVJBEGFOEWS3WVM24AY7C2OVBZWDAXDG4I3A, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:44.897574490 (689h21m42.494345523s) [noise=true , address=167.17.72.70:4751 , piece_id=AX7C5IEHMVNPGGAB7WPWUOSFEHVWWA75JGDAH3DAGZVZCVMEZKKQ, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.084711646 (689h21m42.681482669s) [noise=true , address=193.149.164.66:4996 , piece_id=YGOQ6INTPJ3AZW3RSTVH2IWITNFONA7JTW25BIXNBZRRZUN6YRYA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.084884121 (689h21m42.681655134s) [noise=true , address=193.149.164.67:4002 , piece_id=CYJYJ5ZNXT63OT5DBDLW7ML32OIZLGNWMLM4W2F5TPSSJ2QJ2NBA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.089152844 (689h21m42.685923867s) [noise=true , address=193.149.164.70:4115 , piece_id=EHMHJHJ2ZQWAJWBFSHOTAXWE7QJQOIICXDKAK33KMOAF7C7KZ2PA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.103385369 (689h21m42.700156382s) [noise=true , address=193.149.164.68:4062 , piece_id=IM6B4U2ARVUSBX4276OSQPEPQNMWMFSUUOYAEJ3ZG2GLFUKGGQGQ, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.127711185 (689h21m42.724482213s) [noise=true , address=193.149.164.69:4084 , piece_id=BUV3LZ7PDHSGRNKHJSSN2OFQYMIYUELWEPO4PF3DMTBX65S4YPMA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.143513369 (689h21m42.740284381s) [noise=true , address=193.149.164.71:4308 , piece_id=YGF5F7J25BEULKSTR3ZDC3HE4EHVJ3IF4HQCNATVWHITGRB344JA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.307124646 (689h21m42.903895674s) [noise=true , address=167.17.72.68:4656 , piece_id=YFAXDPM4TYBLYL4UR3Z6FDCDJXU34J7XRURDGLXZAMUOYYZL56IA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:45.517599945 (689h21m43.114370966s) [noise=true , address=145.79.227.33:4898 , piece_id=64X2NB2BPWLP4C5EC53A6EKFPNQCJI6BGEDY6BFE2RFNYEPCDA5Q, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:21:46.061904485 (689h21m43.658675509s) [noise=true , address=193.149.164.143:4704 , piece_id=PZ2W2J22M5MZIYBFEOJCEOIVAJOJGYCHCQUPDSELS764HZG4OF2A, seg_pos={0 0}, upload=772064]: Done uploading piece. err: <nil> Apr 4 04:22:44.423237599 (689h22m42.020008631s) [noise=true , address=77.90.31.13:4658 , piece_id=I7GGGCPJIHSQ5RYYOPIXKWAJBOMSRSK3ZDCASMKUPBQPALOJCKXA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: ecclient: upload failed (node:129aongkQE9HMNz8vsNNA5SvKbhsHqdt9UrtSTPJJpP9mkFsboX, address:77.90.31.13:4658): protocol: read tcp 159.223.143.39:50834->77.90.31.13:4658: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 4 04:22:44.423364101 (689h22m42.020135144s) [noise=true , address=77.90.31.17:4783 , piece_id=E2XPGXQAVSYEMBWOIPKHGL5MVVGC2FSLN5KHZQC6DBUAT3VUWOKA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: ecclient: upload failed (node:155fvDXZdpQeHetitv17WCVVrP1bGswdoqJECQLSe66kXRhBPo, address:77.90.31.17:4783): protocol: read tcp 159.223.143.39:45990->77.90.31.17:4783: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 4 04:22:44.423435607 (689h22m42.020206643s) [noise=true , address=77.90.31.16:4750 , piece_id=TI2D4Z2MYSXWBHX3ZVEMRFLI4LW5RDVCENH43S7GEFR7WGIVCI6A, seg_pos={0 0}, upload=772064]: Done uploading piece. err: ecclient: upload failed (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE, address:77.90.31.16:4750): protocol: read tcp 159.223.143.39:34564->77.90.31.16:4750: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 4 04:22:44.423560856 (689h22m42.020332015s) [noise=true , address=77.90.31.18:4823 , piece_id=MI45FJCGXJ2CIL4OOVHEUB7TJXEGH6WELCSTKR6JD753YLF5UN4Q, seg_pos={0 0}, upload=772064]: Done uploading piece. err: ecclient: upload failed (node:12uUCUDKZDwj25UcEKQ4WiTuQFMRbWw2oymGAdUdMX8wcUSd7jS, address:77.90.31.18:4823): protocol: read tcp 159.223.143.39:40028->77.90.31.18:4823: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 4 04:22:44.423793037 (689h22m42.020564071s) [noise=true , address=77.90.31.15:4709 , piece_id=HQLXOXRSN7R5WC5K2W7X4YUBVEZFITOY4VEYKG7J6WIJDEHABWGA, seg_pos={0 0}, upload=772064]: Done uploading piece. err: ecclient: upload failed (node:1CrHsLV4vge154tJjMvbo5oxnphfukuXQBKfvTjA43n6CJM2Ah, address:77.90.31.15:4709): protocol: read tcp 159.223.143.39:59234->77.90.31.15:4709: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 4 04:22:44.423864095 (689h22m42.020635106s) [seg_pos={0 0}, upload=772064]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 4 04:22:44.451342916 (689h22m42.048113939s) [upload=772064]: Done uploading segments... Apr 4 04:22:44.451381831 (689h22m42.048152855s) [upload=772064]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1.303225422s core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)