Orbiter History

checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..................................................X.........] duration: 644.517306ms trace: https://jaeger.dev.storj.io/trace/bc851a757dc03f0 FAIL (2026-04-08 05:50:13 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 644.681678ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...................................................X........] duration: 456.920535ms trace: https://jaeger.dev.storj.io/trace/bc851a757dc03f0 FAIL (2026-04-08 05:49:37 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 456.946875ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [....................................................X.......] duration: 429.877763ms trace: https://jaeger.dev.storj.io/trace/bc851a757dc03f0 FAIL (2026-04-08 05:48:56 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 429.918436ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.....................................................X......] duration: 479.476911ms trace: https://jaeger.dev.storj.io/trace/bc851a757dc03f0 FAIL (2026-04-08 05:48:19 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 479.553201ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [......................................................X.....] duration: 689.583854ms trace: https://jaeger.dev.storj.io/trace/bc851a757dc03f0 FAIL (2026-04-08 05:48:04 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 689.617818ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.......................................................X....] duration: 587.895601ms trace: https://jaeger.dev.storj.io/trace/bc851a757dc03f0 FAIL (2026-04-08 05:47:21 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 587.920566ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [........................................................X...] duration: 509.798691ms trace: https://jaeger.dev.storj.io/trace/bc851a757dc03f0 FAIL (2026-04-08 05:46:41 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 509.829ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.........................................................X..] duration: 493.098861ms trace: https://jaeger.dev.storj.io/trace/bc851a757dc03f0 FAIL (2026-04-08 05:45:35 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 493.177054ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..........................................................X.] duration: 541.181944ms trace: https://jaeger.dev.storj.io/trace/bc851a757dc03f0 FAIL (2026-04-08 05:44:22 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 541.210496ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...........................................................X] duration: 5.000361143s trace: https://jaeger.dev.storj.io/trace/bc851a757dc03f0 FAIL (2026-04-08 05:43:31 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 5.000439759s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 598.112367ms trace: https://jaeger.dev.storj.io/trace/105ed3c1d703150f FAIL (2026-04-07 23:33:49 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 598.143806ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 870.289048ms trace: https://jaeger.dev.storj.io/trace/105ed3c1d703150f FAIL (2026-04-07 23:33:34 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 870.318366ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 688.6369ms trace: https://jaeger.dev.storj.io/trace/105ed3c1d703150f FAIL (2026-04-07 23:32:43 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 688.682693ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 625.838466ms trace: https://jaeger.dev.storj.io/trace/105ed3c1d703150f FAIL (2026-04-07 23:31:51 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 625.864684ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 547.055459ms trace: https://jaeger.dev.storj.io/trace/105ed3c1d703150f FAIL (2026-04-07 23:31:14 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 547.090964ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 610.818051ms trace: https://jaeger.dev.storj.io/trace/105ed3c1d703150f FAIL (2026-04-07 23:30:36 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 610.843893ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [X...........................................................] duration: 497.697933ms trace: https://jaeger.dev.storj.io/trace/c38cda4ab5d9e66 FAIL (2026-04-07 23:30:27 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 497.723885ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [XX..........................................................] duration: 671.723128ms trace: https://jaeger.dev.storj.io/trace/c38cda4ab5d9e66 FAIL (2026-04-07 23:30:07 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 671.753496ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 730.736479ms trace: https://jaeger.dev.storj.io/trace/105ed3c1d703150f FAIL (2026-04-07 23:29:44 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 730.836466ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [.XX.........................................................] duration: 520.976244ms trace: https://jaeger.dev.storj.io/trace/c38cda4ab5d9e66 FAIL (2026-04-07 23:29:31 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 521.004571ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 424.628834ms trace: https://jaeger.dev.storj.io/trace/345ead65e237bb8a FAIL (2026-04-07 23:29:09 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Apr 7 22:47:07.617258930 (6h25m33.482550917s) [upload=7193]: Starting upload Apr 7 22:47:07.617551609 (6h25m33.482843597s) [upload=7193]: Done starting upload Apr 7 22:47:07.617632062 (6h25m33.482924041s) [upload=7193]: Uploading segments... Apr 7 22:47:07.618008917 (6h25m33.483300903s) [upload=7193]: Got next segment. Inline: false Apr 7 22:47:07.618053874 (6h25m33.483345851s) [upload=7193]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 7 22:47:07.618060313 (6h25m33.483352291s) [upload=7193]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 7 22:47:07.642491084 (6h25m33.507783078s) [seg_pos={0 0}, upload=7193]: Begin upload segment... Apr 7 22:47:07.642559488 (6h25m33.507851476s) [seg_pos={0 0}, upload=7193]: Done begin upload segment. Apr 7 22:47:07.642600942 (6h25m33.507892924s) [noise=true , address=145.79.227.28:4731 , piece_id=3HXBKETWP5RQHFAAV7HJ42U2ZYHKARZ6VL4FKXLIOVTX6XLDQARA, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.642716832 (6h25m33.508008826s) [noise=true , address=45.91.116.13:4156 , piece_id=WVIPDCZSKQZ445H2DXV4XYYHUCRADMP4IKRXDE5DBZRN7GOGTFTQ, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.643511245 (6h25m33.508803243s) [noise=true , address=145.79.227.21:4479 , piece_id=S4VPPXONV2JMMXQD2IOXNCMHR535A3LNAPOLZOWW3ZURUS3FDEMQ, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.647823672 (6h25m33.513115661s) [noise=true , address=45.91.116.14:4193 , piece_id=MK7EUCX7QG22SIHIJK4Y3ZHZZHGL26Q2MKCPEUHGEOJCDAOMABJA, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.649681259 (6h25m33.514973259s) [noise=true , address=145.79.227.26:4659 , piece_id=TX4THUW4PKEJE6TW5CCFOW6UU6RQBVDVS54BBRXBNSLO7YX7PT6Q, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.651770415 (6h25m33.517062412s) [noise=true , address=45.91.116.7:4012 , piece_id=IQCNCBQDLPFQFFDNY3A5LM6Z6RQMTGZHR2EMPC6UWIPUBOXHF6EA, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.652128139 (6h25m33.517420136s) [noise=true , address=145.79.227.29:4946 , piece_id=EJJUVMBKALX7SGZP6ESMTLHKU77D2KS3B4HJQI2YLJZ6MSNTDXYA, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.654883612 (6h25m33.520175609s) [noise=true , address=77.90.31.12:4589 , piece_id=TIKNQEL4RBVP5UESD6XCE7BEXNXVTH6H4GBXG4M75BLMIHM4HLLA, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.656250455 (6h25m33.521542459s) [noise=true , address=145.79.227.49:4341 , piece_id=DUW4E5LYLP354CKDFIM7GNLKZSUVBHBQ3YCUKIK4L7TLN4AAEFWQ, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.658537126 (6h25m33.523829107s) [noise=true , address=45.91.116.6:4989 , piece_id=KUTDWWYWPUECRYU23MKG7WM5GY2PGJWB5D2MVEPXGLUHE64OYK3Q, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.658850782 (6h25m33.524142772s) [noise=true , address=145.79.227.38:4981 , piece_id=7EUWLCGLHSCSL6O3BZOEXNGGZH5GKKOSHGLV5G54DACNHW45UB2Q, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.660811231 (6h25m33.526103211s) [noise=true , address=145.79.227.35:4482 , piece_id=O7TPLKFRENI4GRXPFKMNSPKKG6DQU4TOJPC32TA6BBQQE3SZR45Q, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.663186199 (6h25m33.528478178s) [noise=true , address=145.79.227.42:4126 , piece_id=KPELBB5WVPUY2OM2ATAFL7J7STOBEQ4KE7FX6IENH5NY3NQLTF3A, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.663504933 (6h25m33.528796913s) [noise=true , address=45.91.116.9:4265 , piece_id=2462DJLAE2J2NKQLZHJRIOV5V5LYT6RKRBDGPTCDJE7P6SOSBA6A, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.665606909 (6h25m33.53089889s) [noise=true , address=145.79.227.24:4586 , piece_id=W37RY4GNPAH4SV2VCOHISDJSKYG6LLHPCXDRDV47BSGRRQI4S5GA, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.672462977 (6h25m33.537754958s) [noise=true , address=145.79.227.36:4522 , piece_id=IZQJYU53GMVJFS7ZLXTHLRBGWT6Q6VNAZFDT4AR7GDJR6XI6QTCQ, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.725910666 (6h25m33.591202643s) [upload=7193]: Next returned nil segment Apr 7 22:47:07.725925988 (6h25m33.591217965s) [upload=7193]: Waiting for error group managing segments... Apr 7 22:47:07.732309055 (6h25m33.597601048s) [noise=true , address=145.79.227.28:4731 , piece_id=3HXBKETWP5RQHFAAV7HJ42U2ZYHKARZ6VL4FKXLIOVTX6XLDQARA, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:07.734558945 (6h25m33.59985094s) [noise=true , address=145.79.227.29:4946 , piece_id=EJJUVMBKALX7SGZP6ESMTLHKU77D2KS3B4HJQI2YLJZ6MSNTDXYA, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:07.735241290 (6h25m33.60053328s) [noise=true , address=145.79.227.38:4981 , piece_id=7EUWLCGLHSCSL6O3BZOEXNGGZH5GKKOSHGLV5G54DACNHW45UB2Q, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:07.735980668 (6h25m33.601272659s) [noise=true , address=145.79.227.26:4659 , piece_id=TX4THUW4PKEJE6TW5CCFOW6UU6RQBVDVS54BBRXBNSLO7YX7PT6Q, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:07.743196595 (6h25m33.608488585s) [noise=true , address=145.79.227.42:4126 , piece_id=KPELBB5WVPUY2OM2ATAFL7J7STOBEQ4KE7FX6IENH5NY3NQLTF3A, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:07.743581931 (6h25m33.608873919s) [noise=true , address=77.90.31.12:4589 , piece_id=TIKNQEL4RBVP5UESD6XCE7BEXNXVTH6H4GBXG4M75BLMIHM4HLLA, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:07.746144247 (6h25m33.611436224s) [noise=true , address=145.79.227.24:4586 , piece_id=W37RY4GNPAH4SV2VCOHISDJSKYG6LLHPCXDRDV47BSGRRQI4S5GA, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:07.752443895 (6h25m33.617735893s) [noise=true , address=145.79.227.21:4479 , piece_id=S4VPPXONV2JMMXQD2IOXNCMHR535A3LNAPOLZOWW3ZURUS3FDEMQ, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:07.752795955 (6h25m33.618087942s) [noise=true , address=145.79.227.49:4341 , piece_id=DUW4E5LYLP354CKDFIM7GNLKZSUVBHBQ3YCUKIK4L7TLN4AAEFWQ, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:07.804368344 (6h25m33.669660332s) [noise=true , address=145.79.227.36:4522 , piece_id=IZQJYU53GMVJFS7ZLXTHLRBGWT6Q6VNAZFDT4AR7GDJR6XI6QTCQ, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:13.449965188 (6h25m39.315257175s) [noise=true , address=145.79.227.35:4482 , piece_id=O7TPLKFRENI4GRXPFKMNSPKKG6DQU4TOJPC32TA6BBQQE3SZR45Q, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:37.583489322 (6h26m3.448781319s) [noise=true , address=45.91.116.6:4989 , piece_id=KUTDWWYWPUECRYU23MKG7WM5GY2PGJWB5D2MVEPXGLUHE64OYK3Q, seg_pos={0 0}, upload=7193]: Done uploading piece. err: ecclient: upload failed (node:12esv4CJj8pQep6GYNbWEEnuhgyPoNUNanWqQDW3kEUwqks9eNs, address:45.91.116.6:4989): protocol: write tcp 159.223.143.39:45124->45.91.116.6:4989: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 7 22:47:37.583514465 (6h26m3.44880646s) [noise=true , address=45.91.116.7:4012 , piece_id=IQCNCBQDLPFQFFDNY3A5LM6Z6RQMTGZHR2EMPC6UWIPUBOXHF6EA, seg_pos={0 0}, upload=7193]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: read tcp 159.223.143.39:59344->45.91.116.7:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 7 22:47:37.583800633 (6h26m3.449092638s) [noise=true , address=45.91.116.14:4193 , piece_id=MK7EUCX7QG22SIHIJK4Y3ZHZZHGL26Q2MKCPEUHGEOJCDAOMABJA, seg_pos={0 0}, upload=7193]: Done uploading piece. err: ecclient: upload failed (node:12sSFZohdEoVVtcSrDCK2Md34TdKW6dxEu51Jm19FNcn7Yc6dRS, address:45.91.116.14:4193): protocol: write tcp 159.223.143.39:51654->45.91.116.14:4193: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 7 22:47:37.583976810 (6h26m3.449268803s) [noise=true , address=45.91.116.13:4156 , piece_id=WVIPDCZSKQZ445H2DXV4XYYHUCRADMP4IKRXDE5DBZRN7GOGTFTQ, seg_pos={0 0}, upload=7193]: Done uploading piece. err: ecclient: upload failed (node:1uZ5svEKyXSyB2kA3DjkHS976ioUcNjFehKCcxQY2pi9iEaAAy, address:45.91.116.13:4156): protocol: rpc: dial tcp 45.91.116.13:4156: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 7 22:47:37.584254316 (6h26m3.449546394s) [noise=true , address=45.91.116.9:4265 , piece_id=2462DJLAE2J2NKQLZHJRIOV5V5LYT6RKRBDGPTCDJE7P6SOSBA6A, seg_pos={0 0}, upload=7193]: Done uploading piece. err: ecclient: upload failed (node:1xJiYuKnpkxVvRVXLmW2ZXR7KL8kQgw9SKHpgoD7uKhdfD2CyP, address:45.91.116.9:4265): protocol: context deadline exceeded; context deadline exceeded Apr 7 22:47:37.584302735 (6h26m3.449594708s) [seg_pos={0 0}, upload=7193]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Apr 7 22:47:37.603734242 (6h26m3.46902623s) [upload=7193]: Done uploading segments... Apr 7 22:47:37.603777446 (6h26m3.469069433s) [upload=7193]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 424.656036ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [..XX........................................................] duration: 709.316063ms trace: https://jaeger.dev.storj.io/trace/c38cda4ab5d9e66 FAIL (2026-04-07 23:29:08 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 709.344197ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 680.890905ms trace: https://jaeger.dev.storj.io/trace/105ed3c1d703150f FAIL (2026-04-07 23:28:52 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 680.918ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [...XX.......................................................] duration: 639.96328ms trace: https://jaeger.dev.storj.io/trace/c38cda4ab5d9e66 FAIL (2026-04-07 23:28:34 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 639.986915ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 477.256531ms trace: https://jaeger.dev.storj.io/trace/345ead65e237bb8a FAIL (2026-04-07 23:28:12 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Apr 7 22:47:07.617258930 (6h25m33.482550917s) [upload=7193]: Starting upload Apr 7 22:47:07.617551609 (6h25m33.482843597s) [upload=7193]: Done starting upload Apr 7 22:47:07.617632062 (6h25m33.482924041s) [upload=7193]: Uploading segments... Apr 7 22:47:07.618008917 (6h25m33.483300903s) [upload=7193]: Got next segment. Inline: false Apr 7 22:47:07.618053874 (6h25m33.483345851s) [upload=7193]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 7 22:47:07.618060313 (6h25m33.483352291s) [upload=7193]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 7 22:47:07.642491084 (6h25m33.507783078s) [seg_pos={0 0}, upload=7193]: Begin upload segment... Apr 7 22:47:07.642559488 (6h25m33.507851476s) [seg_pos={0 0}, upload=7193]: Done begin upload segment. Apr 7 22:47:07.642600942 (6h25m33.507892924s) [noise=true , address=145.79.227.28:4731 , piece_id=3HXBKETWP5RQHFAAV7HJ42U2ZYHKARZ6VL4FKXLIOVTX6XLDQARA, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.642716832 (6h25m33.508008826s) [noise=true , address=45.91.116.13:4156 , piece_id=WVIPDCZSKQZ445H2DXV4XYYHUCRADMP4IKRXDE5DBZRN7GOGTFTQ, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.643511245 (6h25m33.508803243s) [noise=true , address=145.79.227.21:4479 , piece_id=S4VPPXONV2JMMXQD2IOXNCMHR535A3LNAPOLZOWW3ZURUS3FDEMQ, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.647823672 (6h25m33.513115661s) [noise=true , address=45.91.116.14:4193 , piece_id=MK7EUCX7QG22SIHIJK4Y3ZHZZHGL26Q2MKCPEUHGEOJCDAOMABJA, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.649681259 (6h25m33.514973259s) [noise=true , address=145.79.227.26:4659 , piece_id=TX4THUW4PKEJE6TW5CCFOW6UU6RQBVDVS54BBRXBNSLO7YX7PT6Q, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.651770415 (6h25m33.517062412s) [noise=true , address=45.91.116.7:4012 , piece_id=IQCNCBQDLPFQFFDNY3A5LM6Z6RQMTGZHR2EMPC6UWIPUBOXHF6EA, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.652128139 (6h25m33.517420136s) [noise=true , address=145.79.227.29:4946 , piece_id=EJJUVMBKALX7SGZP6ESMTLHKU77D2KS3B4HJQI2YLJZ6MSNTDXYA, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.654883612 (6h25m33.520175609s) [noise=true , address=77.90.31.12:4589 , piece_id=TIKNQEL4RBVP5UESD6XCE7BEXNXVTH6H4GBXG4M75BLMIHM4HLLA, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.656250455 (6h25m33.521542459s) [noise=true , address=145.79.227.49:4341 , piece_id=DUW4E5LYLP354CKDFIM7GNLKZSUVBHBQ3YCUKIK4L7TLN4AAEFWQ, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.658537126 (6h25m33.523829107s) [noise=true , address=45.91.116.6:4989 , piece_id=KUTDWWYWPUECRYU23MKG7WM5GY2PGJWB5D2MVEPXGLUHE64OYK3Q, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.658850782 (6h25m33.524142772s) [noise=true , address=145.79.227.38:4981 , piece_id=7EUWLCGLHSCSL6O3BZOEXNGGZH5GKKOSHGLV5G54DACNHW45UB2Q, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.660811231 (6h25m33.526103211s) [noise=true , address=145.79.227.35:4482 , piece_id=O7TPLKFRENI4GRXPFKMNSPKKG6DQU4TOJPC32TA6BBQQE3SZR45Q, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.663186199 (6h25m33.528478178s) [noise=true , address=145.79.227.42:4126 , piece_id=KPELBB5WVPUY2OM2ATAFL7J7STOBEQ4KE7FX6IENH5NY3NQLTF3A, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.663504933 (6h25m33.528796913s) [noise=true , address=45.91.116.9:4265 , piece_id=2462DJLAE2J2NKQLZHJRIOV5V5LYT6RKRBDGPTCDJE7P6SOSBA6A, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.665606909 (6h25m33.53089889s) [noise=true , address=145.79.227.24:4586 , piece_id=W37RY4GNPAH4SV2VCOHISDJSKYG6LLHPCXDRDV47BSGRRQI4S5GA, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.672462977 (6h25m33.537754958s) [noise=true , address=145.79.227.36:4522 , piece_id=IZQJYU53GMVJFS7ZLXTHLRBGWT6Q6VNAZFDT4AR7GDJR6XI6QTCQ, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.725910666 (6h25m33.591202643s) [upload=7193]: Next returned nil segment Apr 7 22:47:07.725925988 (6h25m33.591217965s) [upload=7193]: Waiting for error group managing segments... Apr 7 22:47:07.732309055 (6h25m33.597601048s) [noise=true , address=145.79.227.28:4731 , piece_id=3HXBKETWP5RQHFAAV7HJ42U2ZYHKARZ6VL4FKXLIOVTX6XLDQARA, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:07.734558945 (6h25m33.59985094s) [noise=true , address=145.79.227.29:4946 , piece_id=EJJUVMBKALX7SGZP6ESMTLHKU77D2KS3B4HJQI2YLJZ6MSNTDXYA, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:07.735241290 (6h25m33.60053328s) [noise=true , address=145.79.227.38:4981 , piece_id=7EUWLCGLHSCSL6O3BZOEXNGGZH5GKKOSHGLV5G54DACNHW45UB2Q, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:07.735980668 (6h25m33.601272659s) [noise=true , address=145.79.227.26:4659 , piece_id=TX4THUW4PKEJE6TW5CCFOW6UU6RQBVDVS54BBRXBNSLO7YX7PT6Q, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:07.743196595 (6h25m33.608488585s) [noise=true , address=145.79.227.42:4126 , piece_id=KPELBB5WVPUY2OM2ATAFL7J7STOBEQ4KE7FX6IENH5NY3NQLTF3A, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:07.743581931 (6h25m33.608873919s) [noise=true , address=77.90.31.12:4589 , piece_id=TIKNQEL4RBVP5UESD6XCE7BEXNXVTH6H4GBXG4M75BLMIHM4HLLA, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:07.746144247 (6h25m33.611436224s) [noise=true , address=145.79.227.24:4586 , piece_id=W37RY4GNPAH4SV2VCOHISDJSKYG6LLHPCXDRDV47BSGRRQI4S5GA, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:07.752443895 (6h25m33.617735893s) [noise=true , address=145.79.227.21:4479 , piece_id=S4VPPXONV2JMMXQD2IOXNCMHR535A3LNAPOLZOWW3ZURUS3FDEMQ, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:07.752795955 (6h25m33.618087942s) [noise=true , address=145.79.227.49:4341 , piece_id=DUW4E5LYLP354CKDFIM7GNLKZSUVBHBQ3YCUKIK4L7TLN4AAEFWQ, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:07.804368344 (6h25m33.669660332s) [noise=true , address=145.79.227.36:4522 , piece_id=IZQJYU53GMVJFS7ZLXTHLRBGWT6Q6VNAZFDT4AR7GDJR6XI6QTCQ, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:13.449965188 (6h25m39.315257175s) [noise=true , address=145.79.227.35:4482 , piece_id=O7TPLKFRENI4GRXPFKMNSPKKG6DQU4TOJPC32TA6BBQQE3SZR45Q, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:37.583489322 (6h26m3.448781319s) [noise=true , address=45.91.116.6:4989 , piece_id=KUTDWWYWPUECRYU23MKG7WM5GY2PGJWB5D2MVEPXGLUHE64OYK3Q, seg_pos={0 0}, upload=7193]: Done uploading piece. err: ecclient: upload failed (node:12esv4CJj8pQep6GYNbWEEnuhgyPoNUNanWqQDW3kEUwqks9eNs, address:45.91.116.6:4989): protocol: write tcp 159.223.143.39:45124->45.91.116.6:4989: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 7 22:47:37.583514465 (6h26m3.44880646s) [noise=true , address=45.91.116.7:4012 , piece_id=IQCNCBQDLPFQFFDNY3A5LM6Z6RQMTGZHR2EMPC6UWIPUBOXHF6EA, seg_pos={0 0}, upload=7193]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: read tcp 159.223.143.39:59344->45.91.116.7:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 7 22:47:37.583800633 (6h26m3.449092638s) [noise=true , address=45.91.116.14:4193 , piece_id=MK7EUCX7QG22SIHIJK4Y3ZHZZHGL26Q2MKCPEUHGEOJCDAOMABJA, seg_pos={0 0}, upload=7193]: Done uploading piece. err: ecclient: upload failed (node:12sSFZohdEoVVtcSrDCK2Md34TdKW6dxEu51Jm19FNcn7Yc6dRS, address:45.91.116.14:4193): protocol: write tcp 159.223.143.39:51654->45.91.116.14:4193: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 7 22:47:37.583976810 (6h26m3.449268803s) [noise=true , address=45.91.116.13:4156 , piece_id=WVIPDCZSKQZ445H2DXV4XYYHUCRADMP4IKRXDE5DBZRN7GOGTFTQ, seg_pos={0 0}, upload=7193]: Done uploading piece. err: ecclient: upload failed (node:1uZ5svEKyXSyB2kA3DjkHS976ioUcNjFehKCcxQY2pi9iEaAAy, address:45.91.116.13:4156): protocol: rpc: dial tcp 45.91.116.13:4156: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 7 22:47:37.584254316 (6h26m3.449546394s) [noise=true , address=45.91.116.9:4265 , piece_id=2462DJLAE2J2NKQLZHJRIOV5V5LYT6RKRBDGPTCDJE7P6SOSBA6A, seg_pos={0 0}, upload=7193]: Done uploading piece. err: ecclient: upload failed (node:1xJiYuKnpkxVvRVXLmW2ZXR7KL8kQgw9SKHpgoD7uKhdfD2CyP, address:45.91.116.9:4265): protocol: context deadline exceeded; context deadline exceeded Apr 7 22:47:37.584302735 (6h26m3.449594708s) [seg_pos={0 0}, upload=7193]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Apr 7 22:47:37.603734242 (6h26m3.46902623s) [upload=7193]: Done uploading segments... Apr 7 22:47:37.603777446 (6h26m3.469069433s) [upload=7193]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 477.283612ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [....XX......................................................] duration: 508.447484ms trace: https://jaeger.dev.storj.io/trace/c38cda4ab5d9e66 FAIL (2026-04-07 23:28:01 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 508.472897ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 564.088674ms trace: https://jaeger.dev.storj.io/trace/105ed3c1d703150f FAIL (2026-04-07 23:27:51 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 564.11717ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 476.164032ms trace: https://jaeger.dev.storj.io/trace/345ead65e237bb8a FAIL (2026-04-07 23:27:26 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Apr 7 22:47:07.617258930 (6h25m33.482550917s) [upload=7193]: Starting upload Apr 7 22:47:07.617551609 (6h25m33.482843597s) [upload=7193]: Done starting upload Apr 7 22:47:07.617632062 (6h25m33.482924041s) [upload=7193]: Uploading segments... Apr 7 22:47:07.618008917 (6h25m33.483300903s) [upload=7193]: Got next segment. Inline: false Apr 7 22:47:07.618053874 (6h25m33.483345851s) [upload=7193]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 7 22:47:07.618060313 (6h25m33.483352291s) [upload=7193]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 7 22:47:07.642491084 (6h25m33.507783078s) [seg_pos={0 0}, upload=7193]: Begin upload segment... Apr 7 22:47:07.642559488 (6h25m33.507851476s) [seg_pos={0 0}, upload=7193]: Done begin upload segment. Apr 7 22:47:07.642600942 (6h25m33.507892924s) [noise=true , address=145.79.227.28:4731 , piece_id=3HXBKETWP5RQHFAAV7HJ42U2ZYHKARZ6VL4FKXLIOVTX6XLDQARA, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.642716832 (6h25m33.508008826s) [noise=true , address=45.91.116.13:4156 , piece_id=WVIPDCZSKQZ445H2DXV4XYYHUCRADMP4IKRXDE5DBZRN7GOGTFTQ, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.643511245 (6h25m33.508803243s) [noise=true , address=145.79.227.21:4479 , piece_id=S4VPPXONV2JMMXQD2IOXNCMHR535A3LNAPOLZOWW3ZURUS3FDEMQ, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.647823672 (6h25m33.513115661s) [noise=true , address=45.91.116.14:4193 , piece_id=MK7EUCX7QG22SIHIJK4Y3ZHZZHGL26Q2MKCPEUHGEOJCDAOMABJA, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.649681259 (6h25m33.514973259s) [noise=true , address=145.79.227.26:4659 , piece_id=TX4THUW4PKEJE6TW5CCFOW6UU6RQBVDVS54BBRXBNSLO7YX7PT6Q, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.651770415 (6h25m33.517062412s) [noise=true , address=45.91.116.7:4012 , piece_id=IQCNCBQDLPFQFFDNY3A5LM6Z6RQMTGZHR2EMPC6UWIPUBOXHF6EA, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.652128139 (6h25m33.517420136s) [noise=true , address=145.79.227.29:4946 , piece_id=EJJUVMBKALX7SGZP6ESMTLHKU77D2KS3B4HJQI2YLJZ6MSNTDXYA, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.654883612 (6h25m33.520175609s) [noise=true , address=77.90.31.12:4589 , piece_id=TIKNQEL4RBVP5UESD6XCE7BEXNXVTH6H4GBXG4M75BLMIHM4HLLA, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.656250455 (6h25m33.521542459s) [noise=true , address=145.79.227.49:4341 , piece_id=DUW4E5LYLP354CKDFIM7GNLKZSUVBHBQ3YCUKIK4L7TLN4AAEFWQ, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.658537126 (6h25m33.523829107s) [noise=true , address=45.91.116.6:4989 , piece_id=KUTDWWYWPUECRYU23MKG7WM5GY2PGJWB5D2MVEPXGLUHE64OYK3Q, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.658850782 (6h25m33.524142772s) [noise=true , address=145.79.227.38:4981 , piece_id=7EUWLCGLHSCSL6O3BZOEXNGGZH5GKKOSHGLV5G54DACNHW45UB2Q, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.660811231 (6h25m33.526103211s) [noise=true , address=145.79.227.35:4482 , piece_id=O7TPLKFRENI4GRXPFKMNSPKKG6DQU4TOJPC32TA6BBQQE3SZR45Q, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.663186199 (6h25m33.528478178s) [noise=true , address=145.79.227.42:4126 , piece_id=KPELBB5WVPUY2OM2ATAFL7J7STOBEQ4KE7FX6IENH5NY3NQLTF3A, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.663504933 (6h25m33.528796913s) [noise=true , address=45.91.116.9:4265 , piece_id=2462DJLAE2J2NKQLZHJRIOV5V5LYT6RKRBDGPTCDJE7P6SOSBA6A, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.665606909 (6h25m33.53089889s) [noise=true , address=145.79.227.24:4586 , piece_id=W37RY4GNPAH4SV2VCOHISDJSKYG6LLHPCXDRDV47BSGRRQI4S5GA, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.672462977 (6h25m33.537754958s) [noise=true , address=145.79.227.36:4522 , piece_id=IZQJYU53GMVJFS7ZLXTHLRBGWT6Q6VNAZFDT4AR7GDJR6XI6QTCQ, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.725910666 (6h25m33.591202643s) [upload=7193]: Next returned nil segment Apr 7 22:47:07.725925988 (6h25m33.591217965s) [upload=7193]: Waiting for error group managing segments... Apr 7 22:47:07.732309055 (6h25m33.597601048s) [noise=true , address=145.79.227.28:4731 , piece_id=3HXBKETWP5RQHFAAV7HJ42U2ZYHKARZ6VL4FKXLIOVTX6XLDQARA, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:07.734558945 (6h25m33.59985094s) [noise=true , address=145.79.227.29:4946 , piece_id=EJJUVMBKALX7SGZP6ESMTLHKU77D2KS3B4HJQI2YLJZ6MSNTDXYA, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:07.735241290 (6h25m33.60053328s) [noise=true , address=145.79.227.38:4981 , piece_id=7EUWLCGLHSCSL6O3BZOEXNGGZH5GKKOSHGLV5G54DACNHW45UB2Q, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:07.735980668 (6h25m33.601272659s) [noise=true , address=145.79.227.26:4659 , piece_id=TX4THUW4PKEJE6TW5CCFOW6UU6RQBVDVS54BBRXBNSLO7YX7PT6Q, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:07.743196595 (6h25m33.608488585s) [noise=true , address=145.79.227.42:4126 , piece_id=KPELBB5WVPUY2OM2ATAFL7J7STOBEQ4KE7FX6IENH5NY3NQLTF3A, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:07.743581931 (6h25m33.608873919s) [noise=true , address=77.90.31.12:4589 , piece_id=TIKNQEL4RBVP5UESD6XCE7BEXNXVTH6H4GBXG4M75BLMIHM4HLLA, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:07.746144247 (6h25m33.611436224s) [noise=true , address=145.79.227.24:4586 , piece_id=W37RY4GNPAH4SV2VCOHISDJSKYG6LLHPCXDRDV47BSGRRQI4S5GA, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:07.752443895 (6h25m33.617735893s) [noise=true , address=145.79.227.21:4479 , piece_id=S4VPPXONV2JMMXQD2IOXNCMHR535A3LNAPOLZOWW3ZURUS3FDEMQ, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:07.752795955 (6h25m33.618087942s) [noise=true , address=145.79.227.49:4341 , piece_id=DUW4E5LYLP354CKDFIM7GNLKZSUVBHBQ3YCUKIK4L7TLN4AAEFWQ, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:07.804368344 (6h25m33.669660332s) [noise=true , address=145.79.227.36:4522 , piece_id=IZQJYU53GMVJFS7ZLXTHLRBGWT6Q6VNAZFDT4AR7GDJR6XI6QTCQ, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:13.449965188 (6h25m39.315257175s) [noise=true , address=145.79.227.35:4482 , piece_id=O7TPLKFRENI4GRXPFKMNSPKKG6DQU4TOJPC32TA6BBQQE3SZR45Q, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:37.583489322 (6h26m3.448781319s) [noise=true , address=45.91.116.6:4989 , piece_id=KUTDWWYWPUECRYU23MKG7WM5GY2PGJWB5D2MVEPXGLUHE64OYK3Q, seg_pos={0 0}, upload=7193]: Done uploading piece. err: ecclient: upload failed (node:12esv4CJj8pQep6GYNbWEEnuhgyPoNUNanWqQDW3kEUwqks9eNs, address:45.91.116.6:4989): protocol: write tcp 159.223.143.39:45124->45.91.116.6:4989: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 7 22:47:37.583514465 (6h26m3.44880646s) [noise=true , address=45.91.116.7:4012 , piece_id=IQCNCBQDLPFQFFDNY3A5LM6Z6RQMTGZHR2EMPC6UWIPUBOXHF6EA, seg_pos={0 0}, upload=7193]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: read tcp 159.223.143.39:59344->45.91.116.7:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 7 22:47:37.583800633 (6h26m3.449092638s) [noise=true , address=45.91.116.14:4193 , piece_id=MK7EUCX7QG22SIHIJK4Y3ZHZZHGL26Q2MKCPEUHGEOJCDAOMABJA, seg_pos={0 0}, upload=7193]: Done uploading piece. err: ecclient: upload failed (node:12sSFZohdEoVVtcSrDCK2Md34TdKW6dxEu51Jm19FNcn7Yc6dRS, address:45.91.116.14:4193): protocol: write tcp 159.223.143.39:51654->45.91.116.14:4193: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 7 22:47:37.583976810 (6h26m3.449268803s) [noise=true , address=45.91.116.13:4156 , piece_id=WVIPDCZSKQZ445H2DXV4XYYHUCRADMP4IKRXDE5DBZRN7GOGTFTQ, seg_pos={0 0}, upload=7193]: Done uploading piece. err: ecclient: upload failed (node:1uZ5svEKyXSyB2kA3DjkHS976ioUcNjFehKCcxQY2pi9iEaAAy, address:45.91.116.13:4156): protocol: rpc: dial tcp 45.91.116.13:4156: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 7 22:47:37.584254316 (6h26m3.449546394s) [noise=true , address=45.91.116.9:4265 , piece_id=2462DJLAE2J2NKQLZHJRIOV5V5LYT6RKRBDGPTCDJE7P6SOSBA6A, seg_pos={0 0}, upload=7193]: Done uploading piece. err: ecclient: upload failed (node:1xJiYuKnpkxVvRVXLmW2ZXR7KL8kQgw9SKHpgoD7uKhdfD2CyP, address:45.91.116.9:4265): protocol: context deadline exceeded; context deadline exceeded Apr 7 22:47:37.584302735 (6h26m3.449594708s) [seg_pos={0 0}, upload=7193]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Apr 7 22:47:37.603734242 (6h26m3.46902623s) [upload=7193]: Done uploading segments... Apr 7 22:47:37.603777446 (6h26m3.469069433s) [upload=7193]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 476.193813ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [.....XX.....................................................] duration: 866.445721ms trace: https://jaeger.dev.storj.io/trace/c38cda4ab5d9e66 FAIL (2026-04-07 23:27:10 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 866.470182ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 554.50824ms trace: https://jaeger.dev.storj.io/trace/105ed3c1d703150f FAIL (2026-04-07 23:26:41 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 554.674217ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 436.773258ms trace: https://jaeger.dev.storj.io/trace/345ead65e237bb8a FAIL (2026-04-07 23:26:33 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Apr 7 22:47:07.617258930 (6h25m33.482550917s) [upload=7193]: Starting upload Apr 7 22:47:07.617551609 (6h25m33.482843597s) [upload=7193]: Done starting upload Apr 7 22:47:07.617632062 (6h25m33.482924041s) [upload=7193]: Uploading segments... Apr 7 22:47:07.618008917 (6h25m33.483300903s) [upload=7193]: Got next segment. Inline: false Apr 7 22:47:07.618053874 (6h25m33.483345851s) [upload=7193]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 7 22:47:07.618060313 (6h25m33.483352291s) [upload=7193]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 7 22:47:07.642491084 (6h25m33.507783078s) [seg_pos={0 0}, upload=7193]: Begin upload segment... Apr 7 22:47:07.642559488 (6h25m33.507851476s) [seg_pos={0 0}, upload=7193]: Done begin upload segment. Apr 7 22:47:07.642600942 (6h25m33.507892924s) [noise=true , address=145.79.227.28:4731 , piece_id=3HXBKETWP5RQHFAAV7HJ42U2ZYHKARZ6VL4FKXLIOVTX6XLDQARA, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.642716832 (6h25m33.508008826s) [noise=true , address=45.91.116.13:4156 , piece_id=WVIPDCZSKQZ445H2DXV4XYYHUCRADMP4IKRXDE5DBZRN7GOGTFTQ, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.643511245 (6h25m33.508803243s) [noise=true , address=145.79.227.21:4479 , piece_id=S4VPPXONV2JMMXQD2IOXNCMHR535A3LNAPOLZOWW3ZURUS3FDEMQ, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.647823672 (6h25m33.513115661s) [noise=true , address=45.91.116.14:4193 , piece_id=MK7EUCX7QG22SIHIJK4Y3ZHZZHGL26Q2MKCPEUHGEOJCDAOMABJA, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.649681259 (6h25m33.514973259s) [noise=true , address=145.79.227.26:4659 , piece_id=TX4THUW4PKEJE6TW5CCFOW6UU6RQBVDVS54BBRXBNSLO7YX7PT6Q, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.651770415 (6h25m33.517062412s) [noise=true , address=45.91.116.7:4012 , piece_id=IQCNCBQDLPFQFFDNY3A5LM6Z6RQMTGZHR2EMPC6UWIPUBOXHF6EA, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.652128139 (6h25m33.517420136s) [noise=true , address=145.79.227.29:4946 , piece_id=EJJUVMBKALX7SGZP6ESMTLHKU77D2KS3B4HJQI2YLJZ6MSNTDXYA, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.654883612 (6h25m33.520175609s) [noise=true , address=77.90.31.12:4589 , piece_id=TIKNQEL4RBVP5UESD6XCE7BEXNXVTH6H4GBXG4M75BLMIHM4HLLA, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.656250455 (6h25m33.521542459s) [noise=true , address=145.79.227.49:4341 , piece_id=DUW4E5LYLP354CKDFIM7GNLKZSUVBHBQ3YCUKIK4L7TLN4AAEFWQ, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.658537126 (6h25m33.523829107s) [noise=true , address=45.91.116.6:4989 , piece_id=KUTDWWYWPUECRYU23MKG7WM5GY2PGJWB5D2MVEPXGLUHE64OYK3Q, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.658850782 (6h25m33.524142772s) [noise=true , address=145.79.227.38:4981 , piece_id=7EUWLCGLHSCSL6O3BZOEXNGGZH5GKKOSHGLV5G54DACNHW45UB2Q, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.660811231 (6h25m33.526103211s) [noise=true , address=145.79.227.35:4482 , piece_id=O7TPLKFRENI4GRXPFKMNSPKKG6DQU4TOJPC32TA6BBQQE3SZR45Q, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.663186199 (6h25m33.528478178s) [noise=true , address=145.79.227.42:4126 , piece_id=KPELBB5WVPUY2OM2ATAFL7J7STOBEQ4KE7FX6IENH5NY3NQLTF3A, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.663504933 (6h25m33.528796913s) [noise=true , address=45.91.116.9:4265 , piece_id=2462DJLAE2J2NKQLZHJRIOV5V5LYT6RKRBDGPTCDJE7P6SOSBA6A, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.665606909 (6h25m33.53089889s) [noise=true , address=145.79.227.24:4586 , piece_id=W37RY4GNPAH4SV2VCOHISDJSKYG6LLHPCXDRDV47BSGRRQI4S5GA, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.672462977 (6h25m33.537754958s) [noise=true , address=145.79.227.36:4522 , piece_id=IZQJYU53GMVJFS7ZLXTHLRBGWT6Q6VNAZFDT4AR7GDJR6XI6QTCQ, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.725910666 (6h25m33.591202643s) [upload=7193]: Next returned nil segment Apr 7 22:47:07.725925988 (6h25m33.591217965s) [upload=7193]: Waiting for error group managing segments... Apr 7 22:47:07.732309055 (6h25m33.597601048s) [noise=true , address=145.79.227.28:4731 , piece_id=3HXBKETWP5RQHFAAV7HJ42U2ZYHKARZ6VL4FKXLIOVTX6XLDQARA, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:07.734558945 (6h25m33.59985094s) [noise=true , address=145.79.227.29:4946 , piece_id=EJJUVMBKALX7SGZP6ESMTLHKU77D2KS3B4HJQI2YLJZ6MSNTDXYA, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:07.735241290 (6h25m33.60053328s) [noise=true , address=145.79.227.38:4981 , piece_id=7EUWLCGLHSCSL6O3BZOEXNGGZH5GKKOSHGLV5G54DACNHW45UB2Q, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:07.735980668 (6h25m33.601272659s) [noise=true , address=145.79.227.26:4659 , piece_id=TX4THUW4PKEJE6TW5CCFOW6UU6RQBVDVS54BBRXBNSLO7YX7PT6Q, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:07.743196595 (6h25m33.608488585s) [noise=true , address=145.79.227.42:4126 , piece_id=KPELBB5WVPUY2OM2ATAFL7J7STOBEQ4KE7FX6IENH5NY3NQLTF3A, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:07.743581931 (6h25m33.608873919s) [noise=true , address=77.90.31.12:4589 , piece_id=TIKNQEL4RBVP5UESD6XCE7BEXNXVTH6H4GBXG4M75BLMIHM4HLLA, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:07.746144247 (6h25m33.611436224s) [noise=true , address=145.79.227.24:4586 , piece_id=W37RY4GNPAH4SV2VCOHISDJSKYG6LLHPCXDRDV47BSGRRQI4S5GA, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:07.752443895 (6h25m33.617735893s) [noise=true , address=145.79.227.21:4479 , piece_id=S4VPPXONV2JMMXQD2IOXNCMHR535A3LNAPOLZOWW3ZURUS3FDEMQ, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:07.752795955 (6h25m33.618087942s) [noise=true , address=145.79.227.49:4341 , piece_id=DUW4E5LYLP354CKDFIM7GNLKZSUVBHBQ3YCUKIK4L7TLN4AAEFWQ, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:07.804368344 (6h25m33.669660332s) [noise=true , address=145.79.227.36:4522 , piece_id=IZQJYU53GMVJFS7ZLXTHLRBGWT6Q6VNAZFDT4AR7GDJR6XI6QTCQ, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:13.449965188 (6h25m39.315257175s) [noise=true , address=145.79.227.35:4482 , piece_id=O7TPLKFRENI4GRXPFKMNSPKKG6DQU4TOJPC32TA6BBQQE3SZR45Q, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:37.583489322 (6h26m3.448781319s) [noise=true , address=45.91.116.6:4989 , piece_id=KUTDWWYWPUECRYU23MKG7WM5GY2PGJWB5D2MVEPXGLUHE64OYK3Q, seg_pos={0 0}, upload=7193]: Done uploading piece. err: ecclient: upload failed (node:12esv4CJj8pQep6GYNbWEEnuhgyPoNUNanWqQDW3kEUwqks9eNs, address:45.91.116.6:4989): protocol: write tcp 159.223.143.39:45124->45.91.116.6:4989: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 7 22:47:37.583514465 (6h26m3.44880646s) [noise=true , address=45.91.116.7:4012 , piece_id=IQCNCBQDLPFQFFDNY3A5LM6Z6RQMTGZHR2EMPC6UWIPUBOXHF6EA, seg_pos={0 0}, upload=7193]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: read tcp 159.223.143.39:59344->45.91.116.7:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 7 22:47:37.583800633 (6h26m3.449092638s) [noise=true , address=45.91.116.14:4193 , piece_id=MK7EUCX7QG22SIHIJK4Y3ZHZZHGL26Q2MKCPEUHGEOJCDAOMABJA, seg_pos={0 0}, upload=7193]: Done uploading piece. err: ecclient: upload failed (node:12sSFZohdEoVVtcSrDCK2Md34TdKW6dxEu51Jm19FNcn7Yc6dRS, address:45.91.116.14:4193): protocol: write tcp 159.223.143.39:51654->45.91.116.14:4193: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 7 22:47:37.583976810 (6h26m3.449268803s) [noise=true , address=45.91.116.13:4156 , piece_id=WVIPDCZSKQZ445H2DXV4XYYHUCRADMP4IKRXDE5DBZRN7GOGTFTQ, seg_pos={0 0}, upload=7193]: Done uploading piece. err: ecclient: upload failed (node:1uZ5svEKyXSyB2kA3DjkHS976ioUcNjFehKCcxQY2pi9iEaAAy, address:45.91.116.13:4156): protocol: rpc: dial tcp 45.91.116.13:4156: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 7 22:47:37.584254316 (6h26m3.449546394s) [noise=true , address=45.91.116.9:4265 , piece_id=2462DJLAE2J2NKQLZHJRIOV5V5LYT6RKRBDGPTCDJE7P6SOSBA6A, seg_pos={0 0}, upload=7193]: Done uploading piece. err: ecclient: upload failed (node:1xJiYuKnpkxVvRVXLmW2ZXR7KL8kQgw9SKHpgoD7uKhdfD2CyP, address:45.91.116.9:4265): protocol: context deadline exceeded; context deadline exceeded Apr 7 22:47:37.584302735 (6h26m3.449594708s) [seg_pos={0 0}, upload=7193]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Apr 7 22:47:37.603734242 (6h26m3.46902623s) [upload=7193]: Done uploading segments... Apr 7 22:47:37.603777446 (6h26m3.469069433s) [upload=7193]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 436.823833ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 804.915539ms trace: https://jaeger.dev.storj.io/trace/105ed3c1d703150f FAIL (2026-04-07 23:26:21 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 804.943906ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [......XX....................................................] duration: 435.017589ms trace: https://jaeger.dev.storj.io/trace/c38cda4ab5d9e66 FAIL (2026-04-07 23:26:07 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 435.038229ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 467.742998ms trace: https://jaeger.dev.storj.io/trace/345ead65e237bb8a FAIL (2026-04-07 23:25:52 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Apr 7 22:47:07.617258930 (6h25m33.482550917s) [upload=7193]: Starting upload Apr 7 22:47:07.617551609 (6h25m33.482843597s) [upload=7193]: Done starting upload Apr 7 22:47:07.617632062 (6h25m33.482924041s) [upload=7193]: Uploading segments... Apr 7 22:47:07.618008917 (6h25m33.483300903s) [upload=7193]: Got next segment. Inline: false Apr 7 22:47:07.618053874 (6h25m33.483345851s) [upload=7193]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 7 22:47:07.618060313 (6h25m33.483352291s) [upload=7193]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 7 22:47:07.642491084 (6h25m33.507783078s) [seg_pos={0 0}, upload=7193]: Begin upload segment... Apr 7 22:47:07.642559488 (6h25m33.507851476s) [seg_pos={0 0}, upload=7193]: Done begin upload segment. Apr 7 22:47:07.642600942 (6h25m33.507892924s) [noise=true , address=145.79.227.28:4731 , piece_id=3HXBKETWP5RQHFAAV7HJ42U2ZYHKARZ6VL4FKXLIOVTX6XLDQARA, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.642716832 (6h25m33.508008826s) [noise=true , address=45.91.116.13:4156 , piece_id=WVIPDCZSKQZ445H2DXV4XYYHUCRADMP4IKRXDE5DBZRN7GOGTFTQ, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.643511245 (6h25m33.508803243s) [noise=true , address=145.79.227.21:4479 , piece_id=S4VPPXONV2JMMXQD2IOXNCMHR535A3LNAPOLZOWW3ZURUS3FDEMQ, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.647823672 (6h25m33.513115661s) [noise=true , address=45.91.116.14:4193 , piece_id=MK7EUCX7QG22SIHIJK4Y3ZHZZHGL26Q2MKCPEUHGEOJCDAOMABJA, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.649681259 (6h25m33.514973259s) [noise=true , address=145.79.227.26:4659 , piece_id=TX4THUW4PKEJE6TW5CCFOW6UU6RQBVDVS54BBRXBNSLO7YX7PT6Q, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.651770415 (6h25m33.517062412s) [noise=true , address=45.91.116.7:4012 , piece_id=IQCNCBQDLPFQFFDNY3A5LM6Z6RQMTGZHR2EMPC6UWIPUBOXHF6EA, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.652128139 (6h25m33.517420136s) [noise=true , address=145.79.227.29:4946 , piece_id=EJJUVMBKALX7SGZP6ESMTLHKU77D2KS3B4HJQI2YLJZ6MSNTDXYA, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.654883612 (6h25m33.520175609s) [noise=true , address=77.90.31.12:4589 , piece_id=TIKNQEL4RBVP5UESD6XCE7BEXNXVTH6H4GBXG4M75BLMIHM4HLLA, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.656250455 (6h25m33.521542459s) [noise=true , address=145.79.227.49:4341 , piece_id=DUW4E5LYLP354CKDFIM7GNLKZSUVBHBQ3YCUKIK4L7TLN4AAEFWQ, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.658537126 (6h25m33.523829107s) [noise=true , address=45.91.116.6:4989 , piece_id=KUTDWWYWPUECRYU23MKG7WM5GY2PGJWB5D2MVEPXGLUHE64OYK3Q, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.658850782 (6h25m33.524142772s) [noise=true , address=145.79.227.38:4981 , piece_id=7EUWLCGLHSCSL6O3BZOEXNGGZH5GKKOSHGLV5G54DACNHW45UB2Q, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.660811231 (6h25m33.526103211s) [noise=true , address=145.79.227.35:4482 , piece_id=O7TPLKFRENI4GRXPFKMNSPKKG6DQU4TOJPC32TA6BBQQE3SZR45Q, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.663186199 (6h25m33.528478178s) [noise=true , address=145.79.227.42:4126 , piece_id=KPELBB5WVPUY2OM2ATAFL7J7STOBEQ4KE7FX6IENH5NY3NQLTF3A, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.663504933 (6h25m33.528796913s) [noise=true , address=45.91.116.9:4265 , piece_id=2462DJLAE2J2NKQLZHJRIOV5V5LYT6RKRBDGPTCDJE7P6SOSBA6A, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.665606909 (6h25m33.53089889s) [noise=true , address=145.79.227.24:4586 , piece_id=W37RY4GNPAH4SV2VCOHISDJSKYG6LLHPCXDRDV47BSGRRQI4S5GA, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.672462977 (6h25m33.537754958s) [noise=true , address=145.79.227.36:4522 , piece_id=IZQJYU53GMVJFS7ZLXTHLRBGWT6Q6VNAZFDT4AR7GDJR6XI6QTCQ, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.725910666 (6h25m33.591202643s) [upload=7193]: Next returned nil segment Apr 7 22:47:07.725925988 (6h25m33.591217965s) [upload=7193]: Waiting for error group managing segments... Apr 7 22:47:07.732309055 (6h25m33.597601048s) [noise=true , address=145.79.227.28:4731 , piece_id=3HXBKETWP5RQHFAAV7HJ42U2ZYHKARZ6VL4FKXLIOVTX6XLDQARA, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:07.734558945 (6h25m33.59985094s) [noise=true , address=145.79.227.29:4946 , piece_id=EJJUVMBKALX7SGZP6ESMTLHKU77D2KS3B4HJQI2YLJZ6MSNTDXYA, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:07.735241290 (6h25m33.60053328s) [noise=true , address=145.79.227.38:4981 , piece_id=7EUWLCGLHSCSL6O3BZOEXNGGZH5GKKOSHGLV5G54DACNHW45UB2Q, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:07.735980668 (6h25m33.601272659s) [noise=true , address=145.79.227.26:4659 , piece_id=TX4THUW4PKEJE6TW5CCFOW6UU6RQBVDVS54BBRXBNSLO7YX7PT6Q, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:07.743196595 (6h25m33.608488585s) [noise=true , address=145.79.227.42:4126 , piece_id=KPELBB5WVPUY2OM2ATAFL7J7STOBEQ4KE7FX6IENH5NY3NQLTF3A, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:07.743581931 (6h25m33.608873919s) [noise=true , address=77.90.31.12:4589 , piece_id=TIKNQEL4RBVP5UESD6XCE7BEXNXVTH6H4GBXG4M75BLMIHM4HLLA, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:07.746144247 (6h25m33.611436224s) [noise=true , address=145.79.227.24:4586 , piece_id=W37RY4GNPAH4SV2VCOHISDJSKYG6LLHPCXDRDV47BSGRRQI4S5GA, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:07.752443895 (6h25m33.617735893s) [noise=true , address=145.79.227.21:4479 , piece_id=S4VPPXONV2JMMXQD2IOXNCMHR535A3LNAPOLZOWW3ZURUS3FDEMQ, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:07.752795955 (6h25m33.618087942s) [noise=true , address=145.79.227.49:4341 , piece_id=DUW4E5LYLP354CKDFIM7GNLKZSUVBHBQ3YCUKIK4L7TLN4AAEFWQ, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:07.804368344 (6h25m33.669660332s) [noise=true , address=145.79.227.36:4522 , piece_id=IZQJYU53GMVJFS7ZLXTHLRBGWT6Q6VNAZFDT4AR7GDJR6XI6QTCQ, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:13.449965188 (6h25m39.315257175s) [noise=true , address=145.79.227.35:4482 , piece_id=O7TPLKFRENI4GRXPFKMNSPKKG6DQU4TOJPC32TA6BBQQE3SZR45Q, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:37.583489322 (6h26m3.448781319s) [noise=true , address=45.91.116.6:4989 , piece_id=KUTDWWYWPUECRYU23MKG7WM5GY2PGJWB5D2MVEPXGLUHE64OYK3Q, seg_pos={0 0}, upload=7193]: Done uploading piece. err: ecclient: upload failed (node:12esv4CJj8pQep6GYNbWEEnuhgyPoNUNanWqQDW3kEUwqks9eNs, address:45.91.116.6:4989): protocol: write tcp 159.223.143.39:45124->45.91.116.6:4989: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 7 22:47:37.583514465 (6h26m3.44880646s) [noise=true , address=45.91.116.7:4012 , piece_id=IQCNCBQDLPFQFFDNY3A5LM6Z6RQMTGZHR2EMPC6UWIPUBOXHF6EA, seg_pos={0 0}, upload=7193]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: read tcp 159.223.143.39:59344->45.91.116.7:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 7 22:47:37.583800633 (6h26m3.449092638s) [noise=true , address=45.91.116.14:4193 , piece_id=MK7EUCX7QG22SIHIJK4Y3ZHZZHGL26Q2MKCPEUHGEOJCDAOMABJA, seg_pos={0 0}, upload=7193]: Done uploading piece. err: ecclient: upload failed (node:12sSFZohdEoVVtcSrDCK2Md34TdKW6dxEu51Jm19FNcn7Yc6dRS, address:45.91.116.14:4193): protocol: write tcp 159.223.143.39:51654->45.91.116.14:4193: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 7 22:47:37.583976810 (6h26m3.449268803s) [noise=true , address=45.91.116.13:4156 , piece_id=WVIPDCZSKQZ445H2DXV4XYYHUCRADMP4IKRXDE5DBZRN7GOGTFTQ, seg_pos={0 0}, upload=7193]: Done uploading piece. err: ecclient: upload failed (node:1uZ5svEKyXSyB2kA3DjkHS976ioUcNjFehKCcxQY2pi9iEaAAy, address:45.91.116.13:4156): protocol: rpc: dial tcp 45.91.116.13:4156: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 7 22:47:37.584254316 (6h26m3.449546394s) [noise=true , address=45.91.116.9:4265 , piece_id=2462DJLAE2J2NKQLZHJRIOV5V5LYT6RKRBDGPTCDJE7P6SOSBA6A, seg_pos={0 0}, upload=7193]: Done uploading piece. err: ecclient: upload failed (node:1xJiYuKnpkxVvRVXLmW2ZXR7KL8kQgw9SKHpgoD7uKhdfD2CyP, address:45.91.116.9:4265): protocol: context deadline exceeded; context deadline exceeded Apr 7 22:47:37.584302735 (6h26m3.449594708s) [seg_pos={0 0}, upload=7193]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Apr 7 22:47:37.603734242 (6h26m3.46902623s) [upload=7193]: Done uploading segments... Apr 7 22:47:37.603777446 (6h26m3.469069433s) [upload=7193]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 467.772113ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [.......XX...................................................] duration: 686.495313ms trace: https://jaeger.dev.storj.io/trace/c38cda4ab5d9e66 FAIL (2026-04-07 23:25:35 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 686.530509ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 648.957026ms trace: https://jaeger.dev.storj.io/trace/105ed3c1d703150f FAIL (2026-04-07 23:25:13 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 648.985782ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [........XX..................................................] duration: 472.721098ms trace: https://jaeger.dev.storj.io/trace/c38cda4ab5d9e66 FAIL (2026-04-07 23:25:10 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 472.784155ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 622.368271ms trace: https://jaeger.dev.storj.io/trace/105ed3c1d703150f FAIL (2026-04-07 23:24:48 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 622.666055ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 417.35529ms trace: https://jaeger.dev.storj.io/trace/345ead65e237bb8a FAIL (2026-04-07 23:24:43 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Apr 7 22:47:07.617258930 (6h25m33.482550917s) [upload=7193]: Starting upload Apr 7 22:47:07.617551609 (6h25m33.482843597s) [upload=7193]: Done starting upload Apr 7 22:47:07.617632062 (6h25m33.482924041s) [upload=7193]: Uploading segments... Apr 7 22:47:07.618008917 (6h25m33.483300903s) [upload=7193]: Got next segment. Inline: false Apr 7 22:47:07.618053874 (6h25m33.483345851s) [upload=7193]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 7 22:47:07.618060313 (6h25m33.483352291s) [upload=7193]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 7 22:47:07.642491084 (6h25m33.507783078s) [seg_pos={0 0}, upload=7193]: Begin upload segment... Apr 7 22:47:07.642559488 (6h25m33.507851476s) [seg_pos={0 0}, upload=7193]: Done begin upload segment. Apr 7 22:47:07.642600942 (6h25m33.507892924s) [noise=true , address=145.79.227.28:4731 , piece_id=3HXBKETWP5RQHFAAV7HJ42U2ZYHKARZ6VL4FKXLIOVTX6XLDQARA, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.642716832 (6h25m33.508008826s) [noise=true , address=45.91.116.13:4156 , piece_id=WVIPDCZSKQZ445H2DXV4XYYHUCRADMP4IKRXDE5DBZRN7GOGTFTQ, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.643511245 (6h25m33.508803243s) [noise=true , address=145.79.227.21:4479 , piece_id=S4VPPXONV2JMMXQD2IOXNCMHR535A3LNAPOLZOWW3ZURUS3FDEMQ, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.647823672 (6h25m33.513115661s) [noise=true , address=45.91.116.14:4193 , piece_id=MK7EUCX7QG22SIHIJK4Y3ZHZZHGL26Q2MKCPEUHGEOJCDAOMABJA, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.649681259 (6h25m33.514973259s) [noise=true , address=145.79.227.26:4659 , piece_id=TX4THUW4PKEJE6TW5CCFOW6UU6RQBVDVS54BBRXBNSLO7YX7PT6Q, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.651770415 (6h25m33.517062412s) [noise=true , address=45.91.116.7:4012 , piece_id=IQCNCBQDLPFQFFDNY3A5LM6Z6RQMTGZHR2EMPC6UWIPUBOXHF6EA, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.652128139 (6h25m33.517420136s) [noise=true , address=145.79.227.29:4946 , piece_id=EJJUVMBKALX7SGZP6ESMTLHKU77D2KS3B4HJQI2YLJZ6MSNTDXYA, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.654883612 (6h25m33.520175609s) [noise=true , address=77.90.31.12:4589 , piece_id=TIKNQEL4RBVP5UESD6XCE7BEXNXVTH6H4GBXG4M75BLMIHM4HLLA, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.656250455 (6h25m33.521542459s) [noise=true , address=145.79.227.49:4341 , piece_id=DUW4E5LYLP354CKDFIM7GNLKZSUVBHBQ3YCUKIK4L7TLN4AAEFWQ, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.658537126 (6h25m33.523829107s) [noise=true , address=45.91.116.6:4989 , piece_id=KUTDWWYWPUECRYU23MKG7WM5GY2PGJWB5D2MVEPXGLUHE64OYK3Q, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.658850782 (6h25m33.524142772s) [noise=true , address=145.79.227.38:4981 , piece_id=7EUWLCGLHSCSL6O3BZOEXNGGZH5GKKOSHGLV5G54DACNHW45UB2Q, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.660811231 (6h25m33.526103211s) [noise=true , address=145.79.227.35:4482 , piece_id=O7TPLKFRENI4GRXPFKMNSPKKG6DQU4TOJPC32TA6BBQQE3SZR45Q, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.663186199 (6h25m33.528478178s) [noise=true , address=145.79.227.42:4126 , piece_id=KPELBB5WVPUY2OM2ATAFL7J7STOBEQ4KE7FX6IENH5NY3NQLTF3A, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.663504933 (6h25m33.528796913s) [noise=true , address=45.91.116.9:4265 , piece_id=2462DJLAE2J2NKQLZHJRIOV5V5LYT6RKRBDGPTCDJE7P6SOSBA6A, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.665606909 (6h25m33.53089889s) [noise=true , address=145.79.227.24:4586 , piece_id=W37RY4GNPAH4SV2VCOHISDJSKYG6LLHPCXDRDV47BSGRRQI4S5GA, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.672462977 (6h25m33.537754958s) [noise=true , address=145.79.227.36:4522 , piece_id=IZQJYU53GMVJFS7ZLXTHLRBGWT6Q6VNAZFDT4AR7GDJR6XI6QTCQ, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.725910666 (6h25m33.591202643s) [upload=7193]: Next returned nil segment Apr 7 22:47:07.725925988 (6h25m33.591217965s) [upload=7193]: Waiting for error group managing segments... Apr 7 22:47:07.732309055 (6h25m33.597601048s) [noise=true , address=145.79.227.28:4731 , piece_id=3HXBKETWP5RQHFAAV7HJ42U2ZYHKARZ6VL4FKXLIOVTX6XLDQARA, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:07.734558945 (6h25m33.59985094s) [noise=true , address=145.79.227.29:4946 , piece_id=EJJUVMBKALX7SGZP6ESMTLHKU77D2KS3B4HJQI2YLJZ6MSNTDXYA, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:07.735241290 (6h25m33.60053328s) [noise=true , address=145.79.227.38:4981 , piece_id=7EUWLCGLHSCSL6O3BZOEXNGGZH5GKKOSHGLV5G54DACNHW45UB2Q, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:07.735980668 (6h25m33.601272659s) [noise=true , address=145.79.227.26:4659 , piece_id=TX4THUW4PKEJE6TW5CCFOW6UU6RQBVDVS54BBRXBNSLO7YX7PT6Q, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:07.743196595 (6h25m33.608488585s) [noise=true , address=145.79.227.42:4126 , piece_id=KPELBB5WVPUY2OM2ATAFL7J7STOBEQ4KE7FX6IENH5NY3NQLTF3A, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:07.743581931 (6h25m33.608873919s) [noise=true , address=77.90.31.12:4589 , piece_id=TIKNQEL4RBVP5UESD6XCE7BEXNXVTH6H4GBXG4M75BLMIHM4HLLA, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:07.746144247 (6h25m33.611436224s) [noise=true , address=145.79.227.24:4586 , piece_id=W37RY4GNPAH4SV2VCOHISDJSKYG6LLHPCXDRDV47BSGRRQI4S5GA, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:07.752443895 (6h25m33.617735893s) [noise=true , address=145.79.227.21:4479 , piece_id=S4VPPXONV2JMMXQD2IOXNCMHR535A3LNAPOLZOWW3ZURUS3FDEMQ, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:07.752795955 (6h25m33.618087942s) [noise=true , address=145.79.227.49:4341 , piece_id=DUW4E5LYLP354CKDFIM7GNLKZSUVBHBQ3YCUKIK4L7TLN4AAEFWQ, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:07.804368344 (6h25m33.669660332s) [noise=true , address=145.79.227.36:4522 , piece_id=IZQJYU53GMVJFS7ZLXTHLRBGWT6Q6VNAZFDT4AR7GDJR6XI6QTCQ, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:13.449965188 (6h25m39.315257175s) [noise=true , address=145.79.227.35:4482 , piece_id=O7TPLKFRENI4GRXPFKMNSPKKG6DQU4TOJPC32TA6BBQQE3SZR45Q, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:37.583489322 (6h26m3.448781319s) [noise=true , address=45.91.116.6:4989 , piece_id=KUTDWWYWPUECRYU23MKG7WM5GY2PGJWB5D2MVEPXGLUHE64OYK3Q, seg_pos={0 0}, upload=7193]: Done uploading piece. err: ecclient: upload failed (node:12esv4CJj8pQep6GYNbWEEnuhgyPoNUNanWqQDW3kEUwqks9eNs, address:45.91.116.6:4989): protocol: write tcp 159.223.143.39:45124->45.91.116.6:4989: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 7 22:47:37.583514465 (6h26m3.44880646s) [noise=true , address=45.91.116.7:4012 , piece_id=IQCNCBQDLPFQFFDNY3A5LM6Z6RQMTGZHR2EMPC6UWIPUBOXHF6EA, seg_pos={0 0}, upload=7193]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: read tcp 159.223.143.39:59344->45.91.116.7:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 7 22:47:37.583800633 (6h26m3.449092638s) [noise=true , address=45.91.116.14:4193 , piece_id=MK7EUCX7QG22SIHIJK4Y3ZHZZHGL26Q2MKCPEUHGEOJCDAOMABJA, seg_pos={0 0}, upload=7193]: Done uploading piece. err: ecclient: upload failed (node:12sSFZohdEoVVtcSrDCK2Md34TdKW6dxEu51Jm19FNcn7Yc6dRS, address:45.91.116.14:4193): protocol: write tcp 159.223.143.39:51654->45.91.116.14:4193: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 7 22:47:37.583976810 (6h26m3.449268803s) [noise=true , address=45.91.116.13:4156 , piece_id=WVIPDCZSKQZ445H2DXV4XYYHUCRADMP4IKRXDE5DBZRN7GOGTFTQ, seg_pos={0 0}, upload=7193]: Done uploading piece. err: ecclient: upload failed (node:1uZ5svEKyXSyB2kA3DjkHS976ioUcNjFehKCcxQY2pi9iEaAAy, address:45.91.116.13:4156): protocol: rpc: dial tcp 45.91.116.13:4156: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 7 22:47:37.584254316 (6h26m3.449546394s) [noise=true , address=45.91.116.9:4265 , piece_id=2462DJLAE2J2NKQLZHJRIOV5V5LYT6RKRBDGPTCDJE7P6SOSBA6A, seg_pos={0 0}, upload=7193]: Done uploading piece. err: ecclient: upload failed (node:1xJiYuKnpkxVvRVXLmW2ZXR7KL8kQgw9SKHpgoD7uKhdfD2CyP, address:45.91.116.9:4265): protocol: context deadline exceeded; context deadline exceeded Apr 7 22:47:37.584302735 (6h26m3.449594708s) [seg_pos={0 0}, upload=7193]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Apr 7 22:47:37.603734242 (6h26m3.46902623s) [upload=7193]: Done uploading segments... Apr 7 22:47:37.603777446 (6h26m3.469069433s) [upload=7193]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 417.382879ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [.........XX.................................................] duration: 500.677436ms trace: https://jaeger.dev.storj.io/trace/c38cda4ab5d9e66 FAIL (2026-04-07 23:24:19 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 500.729711ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 472.399447ms trace: https://jaeger.dev.storj.io/trace/345ead65e237bb8a FAIL (2026-04-07 23:23:52 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Apr 7 22:47:07.617258930 (6h25m33.482550917s) [upload=7193]: Starting upload Apr 7 22:47:07.617551609 (6h25m33.482843597s) [upload=7193]: Done starting upload Apr 7 22:47:07.617632062 (6h25m33.482924041s) [upload=7193]: Uploading segments... Apr 7 22:47:07.618008917 (6h25m33.483300903s) [upload=7193]: Got next segment. Inline: false Apr 7 22:47:07.618053874 (6h25m33.483345851s) [upload=7193]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 7 22:47:07.618060313 (6h25m33.483352291s) [upload=7193]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 7 22:47:07.642491084 (6h25m33.507783078s) [seg_pos={0 0}, upload=7193]: Begin upload segment... Apr 7 22:47:07.642559488 (6h25m33.507851476s) [seg_pos={0 0}, upload=7193]: Done begin upload segment. Apr 7 22:47:07.642600942 (6h25m33.507892924s) [noise=true , address=145.79.227.28:4731 , piece_id=3HXBKETWP5RQHFAAV7HJ42U2ZYHKARZ6VL4FKXLIOVTX6XLDQARA, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.642716832 (6h25m33.508008826s) [noise=true , address=45.91.116.13:4156 , piece_id=WVIPDCZSKQZ445H2DXV4XYYHUCRADMP4IKRXDE5DBZRN7GOGTFTQ, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.643511245 (6h25m33.508803243s) [noise=true , address=145.79.227.21:4479 , piece_id=S4VPPXONV2JMMXQD2IOXNCMHR535A3LNAPOLZOWW3ZURUS3FDEMQ, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.647823672 (6h25m33.513115661s) [noise=true , address=45.91.116.14:4193 , piece_id=MK7EUCX7QG22SIHIJK4Y3ZHZZHGL26Q2MKCPEUHGEOJCDAOMABJA, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.649681259 (6h25m33.514973259s) [noise=true , address=145.79.227.26:4659 , piece_id=TX4THUW4PKEJE6TW5CCFOW6UU6RQBVDVS54BBRXBNSLO7YX7PT6Q, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.651770415 (6h25m33.517062412s) [noise=true , address=45.91.116.7:4012 , piece_id=IQCNCBQDLPFQFFDNY3A5LM6Z6RQMTGZHR2EMPC6UWIPUBOXHF6EA, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.652128139 (6h25m33.517420136s) [noise=true , address=145.79.227.29:4946 , piece_id=EJJUVMBKALX7SGZP6ESMTLHKU77D2KS3B4HJQI2YLJZ6MSNTDXYA, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.654883612 (6h25m33.520175609s) [noise=true , address=77.90.31.12:4589 , piece_id=TIKNQEL4RBVP5UESD6XCE7BEXNXVTH6H4GBXG4M75BLMIHM4HLLA, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.656250455 (6h25m33.521542459s) [noise=true , address=145.79.227.49:4341 , piece_id=DUW4E5LYLP354CKDFIM7GNLKZSUVBHBQ3YCUKIK4L7TLN4AAEFWQ, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.658537126 (6h25m33.523829107s) [noise=true , address=45.91.116.6:4989 , piece_id=KUTDWWYWPUECRYU23MKG7WM5GY2PGJWB5D2MVEPXGLUHE64OYK3Q, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.658850782 (6h25m33.524142772s) [noise=true , address=145.79.227.38:4981 , piece_id=7EUWLCGLHSCSL6O3BZOEXNGGZH5GKKOSHGLV5G54DACNHW45UB2Q, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.660811231 (6h25m33.526103211s) [noise=true , address=145.79.227.35:4482 , piece_id=O7TPLKFRENI4GRXPFKMNSPKKG6DQU4TOJPC32TA6BBQQE3SZR45Q, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.663186199 (6h25m33.528478178s) [noise=true , address=145.79.227.42:4126 , piece_id=KPELBB5WVPUY2OM2ATAFL7J7STOBEQ4KE7FX6IENH5NY3NQLTF3A, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.663504933 (6h25m33.528796913s) [noise=true , address=45.91.116.9:4265 , piece_id=2462DJLAE2J2NKQLZHJRIOV5V5LYT6RKRBDGPTCDJE7P6SOSBA6A, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.665606909 (6h25m33.53089889s) [noise=true , address=145.79.227.24:4586 , piece_id=W37RY4GNPAH4SV2VCOHISDJSKYG6LLHPCXDRDV47BSGRRQI4S5GA, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.672462977 (6h25m33.537754958s) [noise=true , address=145.79.227.36:4522 , piece_id=IZQJYU53GMVJFS7ZLXTHLRBGWT6Q6VNAZFDT4AR7GDJR6XI6QTCQ, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.725910666 (6h25m33.591202643s) [upload=7193]: Next returned nil segment Apr 7 22:47:07.725925988 (6h25m33.591217965s) [upload=7193]: Waiting for error group managing segments... Apr 7 22:47:07.732309055 (6h25m33.597601048s) [noise=true , address=145.79.227.28:4731 , piece_id=3HXBKETWP5RQHFAAV7HJ42U2ZYHKARZ6VL4FKXLIOVTX6XLDQARA, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:07.734558945 (6h25m33.59985094s) [noise=true , address=145.79.227.29:4946 , piece_id=EJJUVMBKALX7SGZP6ESMTLHKU77D2KS3B4HJQI2YLJZ6MSNTDXYA, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:07.735241290 (6h25m33.60053328s) [noise=true , address=145.79.227.38:4981 , piece_id=7EUWLCGLHSCSL6O3BZOEXNGGZH5GKKOSHGLV5G54DACNHW45UB2Q, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:07.735980668 (6h25m33.601272659s) [noise=true , address=145.79.227.26:4659 , piece_id=TX4THUW4PKEJE6TW5CCFOW6UU6RQBVDVS54BBRXBNSLO7YX7PT6Q, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:07.743196595 (6h25m33.608488585s) [noise=true , address=145.79.227.42:4126 , piece_id=KPELBB5WVPUY2OM2ATAFL7J7STOBEQ4KE7FX6IENH5NY3NQLTF3A, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:07.743581931 (6h25m33.608873919s) [noise=true , address=77.90.31.12:4589 , piece_id=TIKNQEL4RBVP5UESD6XCE7BEXNXVTH6H4GBXG4M75BLMIHM4HLLA, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:07.746144247 (6h25m33.611436224s) [noise=true , address=145.79.227.24:4586 , piece_id=W37RY4GNPAH4SV2VCOHISDJSKYG6LLHPCXDRDV47BSGRRQI4S5GA, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:07.752443895 (6h25m33.617735893s) [noise=true , address=145.79.227.21:4479 , piece_id=S4VPPXONV2JMMXQD2IOXNCMHR535A3LNAPOLZOWW3ZURUS3FDEMQ, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:07.752795955 (6h25m33.618087942s) [noise=true , address=145.79.227.49:4341 , piece_id=DUW4E5LYLP354CKDFIM7GNLKZSUVBHBQ3YCUKIK4L7TLN4AAEFWQ, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:07.804368344 (6h25m33.669660332s) [noise=true , address=145.79.227.36:4522 , piece_id=IZQJYU53GMVJFS7ZLXTHLRBGWT6Q6VNAZFDT4AR7GDJR6XI6QTCQ, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:13.449965188 (6h25m39.315257175s) [noise=true , address=145.79.227.35:4482 , piece_id=O7TPLKFRENI4GRXPFKMNSPKKG6DQU4TOJPC32TA6BBQQE3SZR45Q, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:37.583489322 (6h26m3.448781319s) [noise=true , address=45.91.116.6:4989 , piece_id=KUTDWWYWPUECRYU23MKG7WM5GY2PGJWB5D2MVEPXGLUHE64OYK3Q, seg_pos={0 0}, upload=7193]: Done uploading piece. err: ecclient: upload failed (node:12esv4CJj8pQep6GYNbWEEnuhgyPoNUNanWqQDW3kEUwqks9eNs, address:45.91.116.6:4989): protocol: write tcp 159.223.143.39:45124->45.91.116.6:4989: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 7 22:47:37.583514465 (6h26m3.44880646s) [noise=true , address=45.91.116.7:4012 , piece_id=IQCNCBQDLPFQFFDNY3A5LM6Z6RQMTGZHR2EMPC6UWIPUBOXHF6EA, seg_pos={0 0}, upload=7193]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: read tcp 159.223.143.39:59344->45.91.116.7:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 7 22:47:37.583800633 (6h26m3.449092638s) [noise=true , address=45.91.116.14:4193 , piece_id=MK7EUCX7QG22SIHIJK4Y3ZHZZHGL26Q2MKCPEUHGEOJCDAOMABJA, seg_pos={0 0}, upload=7193]: Done uploading piece. err: ecclient: upload failed (node:12sSFZohdEoVVtcSrDCK2Md34TdKW6dxEu51Jm19FNcn7Yc6dRS, address:45.91.116.14:4193): protocol: write tcp 159.223.143.39:51654->45.91.116.14:4193: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 7 22:47:37.583976810 (6h26m3.449268803s) [noise=true , address=45.91.116.13:4156 , piece_id=WVIPDCZSKQZ445H2DXV4XYYHUCRADMP4IKRXDE5DBZRN7GOGTFTQ, seg_pos={0 0}, upload=7193]: Done uploading piece. err: ecclient: upload failed (node:1uZ5svEKyXSyB2kA3DjkHS976ioUcNjFehKCcxQY2pi9iEaAAy, address:45.91.116.13:4156): protocol: rpc: dial tcp 45.91.116.13:4156: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 7 22:47:37.584254316 (6h26m3.449546394s) [noise=true , address=45.91.116.9:4265 , piece_id=2462DJLAE2J2NKQLZHJRIOV5V5LYT6RKRBDGPTCDJE7P6SOSBA6A, seg_pos={0 0}, upload=7193]: Done uploading piece. err: ecclient: upload failed (node:1xJiYuKnpkxVvRVXLmW2ZXR7KL8kQgw9SKHpgoD7uKhdfD2CyP, address:45.91.116.9:4265): protocol: context deadline exceeded; context deadline exceeded Apr 7 22:47:37.584302735 (6h26m3.449594708s) [seg_pos={0 0}, upload=7193]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Apr 7 22:47:37.603734242 (6h26m3.46902623s) [upload=7193]: Done uploading segments... Apr 7 22:47:37.603777446 (6h26m3.469069433s) [upload=7193]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 472.42772ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 775.311995ms trace: https://jaeger.dev.storj.io/trace/105ed3c1d703150f FAIL (2026-04-07 23:23:50 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 775.363228ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 550.063501ms trace: https://jaeger.dev.storj.io/trace/105ed3c1d703150f FAIL (2026-04-07 23:23:29 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 550.206241ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [..........XX................................................] duration: 482.357683ms trace: https://jaeger.dev.storj.io/trace/c38cda4ab5d9e66 FAIL (2026-04-07 23:23:27 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 482.383782ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 415.606193ms trace: https://jaeger.dev.storj.io/trace/345ead65e237bb8a FAIL (2026-04-07 23:23:24 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Apr 7 22:47:07.617258930 (6h25m33.482550917s) [upload=7193]: Starting upload Apr 7 22:47:07.617551609 (6h25m33.482843597s) [upload=7193]: Done starting upload Apr 7 22:47:07.617632062 (6h25m33.482924041s) [upload=7193]: Uploading segments... Apr 7 22:47:07.618008917 (6h25m33.483300903s) [upload=7193]: Got next segment. Inline: false Apr 7 22:47:07.618053874 (6h25m33.483345851s) [upload=7193]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 7 22:47:07.618060313 (6h25m33.483352291s) [upload=7193]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 7 22:47:07.642491084 (6h25m33.507783078s) [seg_pos={0 0}, upload=7193]: Begin upload segment... Apr 7 22:47:07.642559488 (6h25m33.507851476s) [seg_pos={0 0}, upload=7193]: Done begin upload segment. Apr 7 22:47:07.642600942 (6h25m33.507892924s) [noise=true , address=145.79.227.28:4731 , piece_id=3HXBKETWP5RQHFAAV7HJ42U2ZYHKARZ6VL4FKXLIOVTX6XLDQARA, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.642716832 (6h25m33.508008826s) [noise=true , address=45.91.116.13:4156 , piece_id=WVIPDCZSKQZ445H2DXV4XYYHUCRADMP4IKRXDE5DBZRN7GOGTFTQ, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.643511245 (6h25m33.508803243s) [noise=true , address=145.79.227.21:4479 , piece_id=S4VPPXONV2JMMXQD2IOXNCMHR535A3LNAPOLZOWW3ZURUS3FDEMQ, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.647823672 (6h25m33.513115661s) [noise=true , address=45.91.116.14:4193 , piece_id=MK7EUCX7QG22SIHIJK4Y3ZHZZHGL26Q2MKCPEUHGEOJCDAOMABJA, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.649681259 (6h25m33.514973259s) [noise=true , address=145.79.227.26:4659 , piece_id=TX4THUW4PKEJE6TW5CCFOW6UU6RQBVDVS54BBRXBNSLO7YX7PT6Q, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.651770415 (6h25m33.517062412s) [noise=true , address=45.91.116.7:4012 , piece_id=IQCNCBQDLPFQFFDNY3A5LM6Z6RQMTGZHR2EMPC6UWIPUBOXHF6EA, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.652128139 (6h25m33.517420136s) [noise=true , address=145.79.227.29:4946 , piece_id=EJJUVMBKALX7SGZP6ESMTLHKU77D2KS3B4HJQI2YLJZ6MSNTDXYA, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.654883612 (6h25m33.520175609s) [noise=true , address=77.90.31.12:4589 , piece_id=TIKNQEL4RBVP5UESD6XCE7BEXNXVTH6H4GBXG4M75BLMIHM4HLLA, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.656250455 (6h25m33.521542459s) [noise=true , address=145.79.227.49:4341 , piece_id=DUW4E5LYLP354CKDFIM7GNLKZSUVBHBQ3YCUKIK4L7TLN4AAEFWQ, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.658537126 (6h25m33.523829107s) [noise=true , address=45.91.116.6:4989 , piece_id=KUTDWWYWPUECRYU23MKG7WM5GY2PGJWB5D2MVEPXGLUHE64OYK3Q, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.658850782 (6h25m33.524142772s) [noise=true , address=145.79.227.38:4981 , piece_id=7EUWLCGLHSCSL6O3BZOEXNGGZH5GKKOSHGLV5G54DACNHW45UB2Q, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.660811231 (6h25m33.526103211s) [noise=true , address=145.79.227.35:4482 , piece_id=O7TPLKFRENI4GRXPFKMNSPKKG6DQU4TOJPC32TA6BBQQE3SZR45Q, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.663186199 (6h25m33.528478178s) [noise=true , address=145.79.227.42:4126 , piece_id=KPELBB5WVPUY2OM2ATAFL7J7STOBEQ4KE7FX6IENH5NY3NQLTF3A, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.663504933 (6h25m33.528796913s) [noise=true , address=45.91.116.9:4265 , piece_id=2462DJLAE2J2NKQLZHJRIOV5V5LYT6RKRBDGPTCDJE7P6SOSBA6A, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.665606909 (6h25m33.53089889s) [noise=true , address=145.79.227.24:4586 , piece_id=W37RY4GNPAH4SV2VCOHISDJSKYG6LLHPCXDRDV47BSGRRQI4S5GA, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.672462977 (6h25m33.537754958s) [noise=true , address=145.79.227.36:4522 , piece_id=IZQJYU53GMVJFS7ZLXTHLRBGWT6Q6VNAZFDT4AR7GDJR6XI6QTCQ, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.725910666 (6h25m33.591202643s) [upload=7193]: Next returned nil segment Apr 7 22:47:07.725925988 (6h25m33.591217965s) [upload=7193]: Waiting for error group managing segments... Apr 7 22:47:07.732309055 (6h25m33.597601048s) [noise=true , address=145.79.227.28:4731 , piece_id=3HXBKETWP5RQHFAAV7HJ42U2ZYHKARZ6VL4FKXLIOVTX6XLDQARA, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:07.734558945 (6h25m33.59985094s) [noise=true , address=145.79.227.29:4946 , piece_id=EJJUVMBKALX7SGZP6ESMTLHKU77D2KS3B4HJQI2YLJZ6MSNTDXYA, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:07.735241290 (6h25m33.60053328s) [noise=true , address=145.79.227.38:4981 , piece_id=7EUWLCGLHSCSL6O3BZOEXNGGZH5GKKOSHGLV5G54DACNHW45UB2Q, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:07.735980668 (6h25m33.601272659s) [noise=true , address=145.79.227.26:4659 , piece_id=TX4THUW4PKEJE6TW5CCFOW6UU6RQBVDVS54BBRXBNSLO7YX7PT6Q, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:07.743196595 (6h25m33.608488585s) [noise=true , address=145.79.227.42:4126 , piece_id=KPELBB5WVPUY2OM2ATAFL7J7STOBEQ4KE7FX6IENH5NY3NQLTF3A, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:07.743581931 (6h25m33.608873919s) [noise=true , address=77.90.31.12:4589 , piece_id=TIKNQEL4RBVP5UESD6XCE7BEXNXVTH6H4GBXG4M75BLMIHM4HLLA, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:07.746144247 (6h25m33.611436224s) [noise=true , address=145.79.227.24:4586 , piece_id=W37RY4GNPAH4SV2VCOHISDJSKYG6LLHPCXDRDV47BSGRRQI4S5GA, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:07.752443895 (6h25m33.617735893s) [noise=true , address=145.79.227.21:4479 , piece_id=S4VPPXONV2JMMXQD2IOXNCMHR535A3LNAPOLZOWW3ZURUS3FDEMQ, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:07.752795955 (6h25m33.618087942s) [noise=true , address=145.79.227.49:4341 , piece_id=DUW4E5LYLP354CKDFIM7GNLKZSUVBHBQ3YCUKIK4L7TLN4AAEFWQ, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:07.804368344 (6h25m33.669660332s) [noise=true , address=145.79.227.36:4522 , piece_id=IZQJYU53GMVJFS7ZLXTHLRBGWT6Q6VNAZFDT4AR7GDJR6XI6QTCQ, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:13.449965188 (6h25m39.315257175s) [noise=true , address=145.79.227.35:4482 , piece_id=O7TPLKFRENI4GRXPFKMNSPKKG6DQU4TOJPC32TA6BBQQE3SZR45Q, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:37.583489322 (6h26m3.448781319s) [noise=true , address=45.91.116.6:4989 , piece_id=KUTDWWYWPUECRYU23MKG7WM5GY2PGJWB5D2MVEPXGLUHE64OYK3Q, seg_pos={0 0}, upload=7193]: Done uploading piece. err: ecclient: upload failed (node:12esv4CJj8pQep6GYNbWEEnuhgyPoNUNanWqQDW3kEUwqks9eNs, address:45.91.116.6:4989): protocol: write tcp 159.223.143.39:45124->45.91.116.6:4989: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 7 22:47:37.583514465 (6h26m3.44880646s) [noise=true , address=45.91.116.7:4012 , piece_id=IQCNCBQDLPFQFFDNY3A5LM6Z6RQMTGZHR2EMPC6UWIPUBOXHF6EA, seg_pos={0 0}, upload=7193]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: read tcp 159.223.143.39:59344->45.91.116.7:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 7 22:47:37.583800633 (6h26m3.449092638s) [noise=true , address=45.91.116.14:4193 , piece_id=MK7EUCX7QG22SIHIJK4Y3ZHZZHGL26Q2MKCPEUHGEOJCDAOMABJA, seg_pos={0 0}, upload=7193]: Done uploading piece. err: ecclient: upload failed (node:12sSFZohdEoVVtcSrDCK2Md34TdKW6dxEu51Jm19FNcn7Yc6dRS, address:45.91.116.14:4193): protocol: write tcp 159.223.143.39:51654->45.91.116.14:4193: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 7 22:47:37.583976810 (6h26m3.449268803s) [noise=true , address=45.91.116.13:4156 , piece_id=WVIPDCZSKQZ445H2DXV4XYYHUCRADMP4IKRXDE5DBZRN7GOGTFTQ, seg_pos={0 0}, upload=7193]: Done uploading piece. err: ecclient: upload failed (node:1uZ5svEKyXSyB2kA3DjkHS976ioUcNjFehKCcxQY2pi9iEaAAy, address:45.91.116.13:4156): protocol: rpc: dial tcp 45.91.116.13:4156: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 7 22:47:37.584254316 (6h26m3.449546394s) [noise=true , address=45.91.116.9:4265 , piece_id=2462DJLAE2J2NKQLZHJRIOV5V5LYT6RKRBDGPTCDJE7P6SOSBA6A, seg_pos={0 0}, upload=7193]: Done uploading piece. err: ecclient: upload failed (node:1xJiYuKnpkxVvRVXLmW2ZXR7KL8kQgw9SKHpgoD7uKhdfD2CyP, address:45.91.116.9:4265): protocol: context deadline exceeded; context deadline exceeded Apr 7 22:47:37.584302735 (6h26m3.449594708s) [seg_pos={0 0}, upload=7193]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Apr 7 22:47:37.603734242 (6h26m3.46902623s) [upload=7193]: Done uploading segments... Apr 7 22:47:37.603777446 (6h26m3.469069433s) [upload=7193]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 415.657607ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 570.692409ms trace: https://jaeger.dev.storj.io/trace/105ed3c1d703150f FAIL (2026-04-07 23:23:10 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 570.864085ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [...........XX...............................................] duration: 476.152518ms trace: https://jaeger.dev.storj.io/trace/c38cda4ab5d9e66 FAIL (2026-04-07 23:22:56 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 476.180195ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 454.886299ms trace: https://jaeger.dev.storj.io/trace/345ead65e237bb8a FAIL (2026-04-07 23:22:23 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Apr 7 22:47:07.617258930 (6h25m33.482550917s) [upload=7193]: Starting upload Apr 7 22:47:07.617551609 (6h25m33.482843597s) [upload=7193]: Done starting upload Apr 7 22:47:07.617632062 (6h25m33.482924041s) [upload=7193]: Uploading segments... Apr 7 22:47:07.618008917 (6h25m33.483300903s) [upload=7193]: Got next segment. Inline: false Apr 7 22:47:07.618053874 (6h25m33.483345851s) [upload=7193]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 7 22:47:07.618060313 (6h25m33.483352291s) [upload=7193]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 7 22:47:07.642491084 (6h25m33.507783078s) [seg_pos={0 0}, upload=7193]: Begin upload segment... Apr 7 22:47:07.642559488 (6h25m33.507851476s) [seg_pos={0 0}, upload=7193]: Done begin upload segment. Apr 7 22:47:07.642600942 (6h25m33.507892924s) [noise=true , address=145.79.227.28:4731 , piece_id=3HXBKETWP5RQHFAAV7HJ42U2ZYHKARZ6VL4FKXLIOVTX6XLDQARA, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.642716832 (6h25m33.508008826s) [noise=true , address=45.91.116.13:4156 , piece_id=WVIPDCZSKQZ445H2DXV4XYYHUCRADMP4IKRXDE5DBZRN7GOGTFTQ, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.643511245 (6h25m33.508803243s) [noise=true , address=145.79.227.21:4479 , piece_id=S4VPPXONV2JMMXQD2IOXNCMHR535A3LNAPOLZOWW3ZURUS3FDEMQ, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.647823672 (6h25m33.513115661s) [noise=true , address=45.91.116.14:4193 , piece_id=MK7EUCX7QG22SIHIJK4Y3ZHZZHGL26Q2MKCPEUHGEOJCDAOMABJA, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.649681259 (6h25m33.514973259s) [noise=true , address=145.79.227.26:4659 , piece_id=TX4THUW4PKEJE6TW5CCFOW6UU6RQBVDVS54BBRXBNSLO7YX7PT6Q, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.651770415 (6h25m33.517062412s) [noise=true , address=45.91.116.7:4012 , piece_id=IQCNCBQDLPFQFFDNY3A5LM6Z6RQMTGZHR2EMPC6UWIPUBOXHF6EA, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.652128139 (6h25m33.517420136s) [noise=true , address=145.79.227.29:4946 , piece_id=EJJUVMBKALX7SGZP6ESMTLHKU77D2KS3B4HJQI2YLJZ6MSNTDXYA, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.654883612 (6h25m33.520175609s) [noise=true , address=77.90.31.12:4589 , piece_id=TIKNQEL4RBVP5UESD6XCE7BEXNXVTH6H4GBXG4M75BLMIHM4HLLA, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.656250455 (6h25m33.521542459s) [noise=true , address=145.79.227.49:4341 , piece_id=DUW4E5LYLP354CKDFIM7GNLKZSUVBHBQ3YCUKIK4L7TLN4AAEFWQ, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.658537126 (6h25m33.523829107s) [noise=true , address=45.91.116.6:4989 , piece_id=KUTDWWYWPUECRYU23MKG7WM5GY2PGJWB5D2MVEPXGLUHE64OYK3Q, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.658850782 (6h25m33.524142772s) [noise=true , address=145.79.227.38:4981 , piece_id=7EUWLCGLHSCSL6O3BZOEXNGGZH5GKKOSHGLV5G54DACNHW45UB2Q, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.660811231 (6h25m33.526103211s) [noise=true , address=145.79.227.35:4482 , piece_id=O7TPLKFRENI4GRXPFKMNSPKKG6DQU4TOJPC32TA6BBQQE3SZR45Q, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.663186199 (6h25m33.528478178s) [noise=true , address=145.79.227.42:4126 , piece_id=KPELBB5WVPUY2OM2ATAFL7J7STOBEQ4KE7FX6IENH5NY3NQLTF3A, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.663504933 (6h25m33.528796913s) [noise=true , address=45.91.116.9:4265 , piece_id=2462DJLAE2J2NKQLZHJRIOV5V5LYT6RKRBDGPTCDJE7P6SOSBA6A, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.665606909 (6h25m33.53089889s) [noise=true , address=145.79.227.24:4586 , piece_id=W37RY4GNPAH4SV2VCOHISDJSKYG6LLHPCXDRDV47BSGRRQI4S5GA, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.672462977 (6h25m33.537754958s) [noise=true , address=145.79.227.36:4522 , piece_id=IZQJYU53GMVJFS7ZLXTHLRBGWT6Q6VNAZFDT4AR7GDJR6XI6QTCQ, seg_pos={0 0}, upload=7193]: Uploading piece... Apr 7 22:47:07.725910666 (6h25m33.591202643s) [upload=7193]: Next returned nil segment Apr 7 22:47:07.725925988 (6h25m33.591217965s) [upload=7193]: Waiting for error group managing segments... Apr 7 22:47:07.732309055 (6h25m33.597601048s) [noise=true , address=145.79.227.28:4731 , piece_id=3HXBKETWP5RQHFAAV7HJ42U2ZYHKARZ6VL4FKXLIOVTX6XLDQARA, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:07.734558945 (6h25m33.59985094s) [noise=true , address=145.79.227.29:4946 , piece_id=EJJUVMBKALX7SGZP6ESMTLHKU77D2KS3B4HJQI2YLJZ6MSNTDXYA, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:07.735241290 (6h25m33.60053328s) [noise=true , address=145.79.227.38:4981 , piece_id=7EUWLCGLHSCSL6O3BZOEXNGGZH5GKKOSHGLV5G54DACNHW45UB2Q, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:07.735980668 (6h25m33.601272659s) [noise=true , address=145.79.227.26:4659 , piece_id=TX4THUW4PKEJE6TW5CCFOW6UU6RQBVDVS54BBRXBNSLO7YX7PT6Q, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:07.743196595 (6h25m33.608488585s) [noise=true , address=145.79.227.42:4126 , piece_id=KPELBB5WVPUY2OM2ATAFL7J7STOBEQ4KE7FX6IENH5NY3NQLTF3A, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:07.743581931 (6h25m33.608873919s) [noise=true , address=77.90.31.12:4589 , piece_id=TIKNQEL4RBVP5UESD6XCE7BEXNXVTH6H4GBXG4M75BLMIHM4HLLA, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:07.746144247 (6h25m33.611436224s) [noise=true , address=145.79.227.24:4586 , piece_id=W37RY4GNPAH4SV2VCOHISDJSKYG6LLHPCXDRDV47BSGRRQI4S5GA, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:07.752443895 (6h25m33.617735893s) [noise=true , address=145.79.227.21:4479 , piece_id=S4VPPXONV2JMMXQD2IOXNCMHR535A3LNAPOLZOWW3ZURUS3FDEMQ, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:07.752795955 (6h25m33.618087942s) [noise=true , address=145.79.227.49:4341 , piece_id=DUW4E5LYLP354CKDFIM7GNLKZSUVBHBQ3YCUKIK4L7TLN4AAEFWQ, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:07.804368344 (6h25m33.669660332s) [noise=true , address=145.79.227.36:4522 , piece_id=IZQJYU53GMVJFS7ZLXTHLRBGWT6Q6VNAZFDT4AR7GDJR6XI6QTCQ, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:13.449965188 (6h25m39.315257175s) [noise=true , address=145.79.227.35:4482 , piece_id=O7TPLKFRENI4GRXPFKMNSPKKG6DQU4TOJPC32TA6BBQQE3SZR45Q, seg_pos={0 0}, upload=7193]: Done uploading piece. err: <nil> Apr 7 22:47:37.583489322 (6h26m3.448781319s) [noise=true , address=45.91.116.6:4989 , piece_id=KUTDWWYWPUECRYU23MKG7WM5GY2PGJWB5D2MVEPXGLUHE64OYK3Q, seg_pos={0 0}, upload=7193]: Done uploading piece. err: ecclient: upload failed (node:12esv4CJj8pQep6GYNbWEEnuhgyPoNUNanWqQDW3kEUwqks9eNs, address:45.91.116.6:4989): protocol: write tcp 159.223.143.39:45124->45.91.116.6:4989: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 7 22:47:37.583514465 (6h26m3.44880646s) [noise=true , address=45.91.116.7:4012 , piece_id=IQCNCBQDLPFQFFDNY3A5LM6Z6RQMTGZHR2EMPC6UWIPUBOXHF6EA, seg_pos={0 0}, upload=7193]: Done uploading piece. err: ecclient: upload failed (node:1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, address:45.91.116.7:4012): protocol: read tcp 159.223.143.39:59344->45.91.116.7:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 7 22:47:37.583800633 (6h26m3.449092638s) [noise=true , address=45.91.116.14:4193 , piece_id=MK7EUCX7QG22SIHIJK4Y3ZHZZHGL26Q2MKCPEUHGEOJCDAOMABJA, seg_pos={0 0}, upload=7193]: Done uploading piece. err: ecclient: upload failed (node:12sSFZohdEoVVtcSrDCK2Md34TdKW6dxEu51Jm19FNcn7Yc6dRS, address:45.91.116.14:4193): protocol: write tcp 159.223.143.39:51654->45.91.116.14:4193: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 7 22:47:37.583976810 (6h26m3.449268803s) [noise=true , address=45.91.116.13:4156 , piece_id=WVIPDCZSKQZ445H2DXV4XYYHUCRADMP4IKRXDE5DBZRN7GOGTFTQ, seg_pos={0 0}, upload=7193]: Done uploading piece. err: ecclient: upload failed (node:1uZ5svEKyXSyB2kA3DjkHS976ioUcNjFehKCcxQY2pi9iEaAAy, address:45.91.116.13:4156): protocol: rpc: dial tcp 45.91.116.13:4156: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Apr 7 22:47:37.584254316 (6h26m3.449546394s) [noise=true , address=45.91.116.9:4265 , piece_id=2462DJLAE2J2NKQLZHJRIOV5V5LYT6RKRBDGPTCDJE7P6SOSBA6A, seg_pos={0 0}, upload=7193]: Done uploading piece. err: ecclient: upload failed (node:1xJiYuKnpkxVvRVXLmW2ZXR7KL8kQgw9SKHpgoD7uKhdfD2CyP, address:45.91.116.9:4265): protocol: context deadline exceeded; context deadline exceeded Apr 7 22:47:37.584302735 (6h26m3.449594708s) [seg_pos={0 0}, upload=7193]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Apr 7 22:47:37.603734242 (6h26m3.46902623s) [upload=7193]: Done uploading segments... Apr 7 22:47:37.603777446 (6h26m3.469069433s) [upload=7193]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 454.957291ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 605.162799ms trace: https://jaeger.dev.storj.io/trace/105ed3c1d703150f FAIL (2026-04-07 23:22:12 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 605.192119ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [............XX..............................................] duration: 689.878044ms trace: https://jaeger.dev.storj.io/trace/c38cda4ab5d9e66 FAIL (2026-04-07 23:21:41 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 689.902986ms core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)