Orbiter History

checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 15 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXX.............................................] duration: 513.080504ms trace: https://jaeger.dev.storj.io/trace/6cabdd7d47c9cb44 FAIL (2026-03-18 22:28:36 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 513.130446ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 10 out of 60 is failed (threshold=3) [XXXXXX.X..XXX...............................................] duration: 1.870258184s trace: https://jaeger.dev.storj.io/trace/27ed2876d5f3e85d FAIL (2026-03-18 22:28:34 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 1.87035021s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 9 out of 60 is failed (threshold=3) [XXXX.X...XX.X.X.............................................] duration: 323.306266ms trace: https://jaeger.dev.storj.io/trace/47004c0815b844e7 FAIL (2026-03-18 22:28:33 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 18 21:53:36.090826124 (298h53m33.687597155s) [upload=334767]: Starting upload Mar 18 21:53:36.090977797 (298h53m33.687748822s) [upload=334767]: Done starting upload Mar 18 21:53:36.091078093 (298h53m33.687849119s) [upload=334767]: Uploading segments... Mar 18 21:53:36.091268776 (298h53m33.688040083s) [upload=334767]: Got next segment. Inline: true Mar 18 21:53:36.091279074 (298h53m33.688050103s) [upload=334767]: Waiting for error group managing segments... Mar 18 21:53:36.091372978 (298h53m33.688144001s) [upload=334767]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 21:53:36.091379985 (298h53m33.688151008s) [upload=334767]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Mar 18 21:53:36.091384265 (298h53m33.688155287s) [upload=334767]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 21:54:04.718146694 (298h54m2.314917717s) [upload=334767]: Done uploading segments... Mar 18 21:54:04.718169125 (298h54m2.314940155s) [upload=334767]: Upload finished. err: <nil> Duration:* 323.325387ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 9 out of 60 is failed (threshold=3) [XX..XX.X..X..X.XX...........................................] duration: 252.631936ms trace: https://jaeger.dev.storj.io/trace/7c4371cfa8f7834c FAIL (2026-03-18 22:28:08 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 252.662283ms core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 10 out of 60 is failed (threshold=3) [XXXXX.X...XX.X.X............................................] duration: 282.086786ms trace: https://jaeger.dev.storj.io/trace/47004c0815b844e7 FAIL (2026-03-18 22:27:50 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 18 21:53:36.090826124 (298h53m33.687597155s) [upload=334767]: Starting upload Mar 18 21:53:36.090977797 (298h53m33.687748822s) [upload=334767]: Done starting upload Mar 18 21:53:36.091078093 (298h53m33.687849119s) [upload=334767]: Uploading segments... Mar 18 21:53:36.091268776 (298h53m33.688040083s) [upload=334767]: Got next segment. Inline: true Mar 18 21:53:36.091279074 (298h53m33.688050103s) [upload=334767]: Waiting for error group managing segments... Mar 18 21:53:36.091372978 (298h53m33.688144001s) [upload=334767]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 21:53:36.091379985 (298h53m33.688151008s) [upload=334767]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Mar 18 21:53:36.091384265 (298h53m33.688155287s) [upload=334767]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 21:54:04.718146694 (298h54m2.314917717s) [upload=334767]: Done uploading segments... Mar 18 21:54:04.718169125 (298h54m2.314940155s) [upload=334767]: Upload finished. err: <nil> Duration:* 282.115464ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [XX.....XXXX.XXX.............................................] duration: 468.182889ms trace: https://jaeger.dev.storj.io/trace/6dceff78c86f0774 FAIL (2026-03-18 22:27:46 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 18 21:54:17.190275520 (298h54m14.787046534s) [upload=334781]: Starting upload Mar 18 21:54:17.190367052 (298h54m14.787138068s) [upload=334781]: Done starting upload Mar 18 21:54:17.190448332 (298h54m14.787219348s) [upload=334781]: Uploading segments... Mar 18 21:54:17.190705184 (298h54m14.787476207s) [upload=334781]: Got next segment. Inline: false Mar 18 21:54:17.190783980 (298h54m14.787555002s) [upload=334781]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 21:54:17.190804859 (298h54m14.787575882s) [upload=334781]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 18 21:54:25.452304457 (298h54m23.049075487s) [seg_pos={0 0}, upload=334781]: Begin upload segment... Mar 18 21:54:25.452374690 (298h54m23.049145713s) [seg_pos={0 0}, upload=334781]: Done begin upload segment. Mar 18 21:54:25.452463335 (298h54m23.049234366s) [noise=true , address=145.79.227.46:4604 , piece_id=DAGTJ24IRMED5FNGQHRYTXEIXYTDVSRW2IT23MC2UC45SES3PE7Q, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.452479705 (298h54m23.049250739s) [noise=true , address=45.91.116.9:4264 , piece_id=EE3TXU5KSPHKMNNCI3BHIGFKQNYZQRBDO4YT3TTR5U66YJMFT6XQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.453224160 (298h54m23.049995188s) [noise=true , address=79.127.219.46:4892 , piece_id=533AWUNWA66VX4AZF7Y3AANCF65GMESWZD3IJWBC5IOKB4QMVTZA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.454147610 (298h54m23.050918643s) [noise=true , address=145.79.227.48:4307 , piece_id=D6WSRVCL6UO4KQ542EKZWA6IP7FLA3ZUGSBXCSZUMLPFCJ5G5CQA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.454790080 (298h54m23.051561121s) [noise=true , address=145.79.227.2:4116 , piece_id=KEKK3M6M37G7Y7HXPULI2RSX5CFGUNCM3LILYNQOZ2PASM742ZGQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.455678920 (298h54m23.052449952s) [noise=true , address=145.79.227.19:4080 , piece_id=ANOMNO3HLII5GPWIT2RDX6P7WOBTVJXMOJM5PAPDMK7ST2NYJWMA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.456387102 (298h54m23.053158136s) [noise=true , address=145.79.227.37:4569 , piece_id=PTE4YT3YJ5M3NDKXKQOUFTGQ3T5FDAUZB74QHJBVH6HQL53QXLUA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.456987483 (298h54m23.053758515s) [noise=true , address=145.79.227.4:4190 , piece_id=LDID6UGFJ4GWYK635S5OE6334DEULOPVSOLOCLYQCBOFTXRZEKRQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.457980795 (298h54m23.054751829s) [noise=true , address=79.127.219.45:4855 , piece_id=644Z2MAXJ423ZF4JGZ2YLPM6XJRZT7IUJCZRRE7ZJMHB3EW7KB4Q, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.459806473 (298h54m23.056577503s) [noise=true , address=145.79.227.50:4376 , piece_id=BHABIBKA3ZENQUNU2OVBVUJZVWEQRZD435EHA3MEB25KMNBVFQIA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.460305248 (298h54m23.057076281s) [noise=true , address=77.90.31.16:4737 , piece_id=SMQOC6BKSIEIE5FXEYZGOIWRZ6GORC7GCOVEOFZXPQX4FGILVIPA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.461552822 (298h54m23.058323848s) [noise=true , address=79.127.219.41:4926 , piece_id=AVSCKJ5WDK4OBWWUSLUQNOPZI7BGTTDGT4XJYSLQAJPUHRYY4KVQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.463144575 (298h54m23.059915608s) [noise=true , address=145.79.227.36:4518 , piece_id=R5PDKAATCPFI3LOOUX6HJ4OP64NQ72N3RAOZS3GYFH5VSWPFU6OQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.464837828 (298h54m23.06160886s) [noise=true , address=145.79.227.26:4656 , piece_id=DDNFTYMUJMFHTZXFHJTFUURSVOJBQX2KVT6UWLBCZ5TIOVBDVOVQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.466411504 (298h54m23.063182537s) [noise=true , address=145.79.227.6:4261 , piece_id=CYOWHNR7X3Y4YE5SFHPNDBOU5QQ6M6DZIRAO3VX56KQBL3OP2FLQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.469091278 (298h54m23.065862311s) [noise=true , address=45.91.116.11:4118 , piece_id=CVKN5WPP2OTRAHYWJKDLCJV3OZ3I6G5CLYXTSEKJDQD7LRTJ55EA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.586976347 (298h54m23.183747361s) [upload=334781]: Next returned nil segment Mar 18 21:54:25.587000505 (298h54m23.183771534s) [upload=334781]: Waiting for error group managing segments... Mar 18 21:54:25.596319798 (298h54m23.193090825s) [noise=true , address=79.127.219.41:4926 , piece_id=AVSCKJ5WDK4OBWWUSLUQNOPZI7BGTTDGT4XJYSLQAJPUHRYY4KVQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.596985876 (298h54m23.1937569s) [noise=true , address=45.91.116.9:4264 , piece_id=EE3TXU5KSPHKMNNCI3BHIGFKQNYZQRBDO4YT3TTR5U66YJMFT6XQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.598026901 (298h54m23.194797934s) [noise=true , address=145.79.227.26:4656 , piece_id=DDNFTYMUJMFHTZXFHJTFUURSVOJBQX2KVT6UWLBCZ5TIOVBDVOVQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.598566162 (298h54m23.195337186s) [noise=true , address=145.79.227.50:4376 , piece_id=BHABIBKA3ZENQUNU2OVBVUJZVWEQRZD435EHA3MEB25KMNBVFQIA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.598781105 (298h54m23.19555213s) [noise=true , address=77.90.31.16:4737 , piece_id=SMQOC6BKSIEIE5FXEYZGOIWRZ6GORC7GCOVEOFZXPQX4FGILVIPA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.600260475 (298h54m23.197031509s) [noise=true , address=145.79.227.6:4261 , piece_id=CYOWHNR7X3Y4YE5SFHPNDBOU5QQ6M6DZIRAO3VX56KQBL3OP2FLQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.600344398 (298h54m23.197115411s) [noise=true , address=145.79.227.4:4190 , piece_id=LDID6UGFJ4GWYK635S5OE6334DEULOPVSOLOCLYQCBOFTXRZEKRQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.602467676 (298h54m23.19923871s) [noise=true , address=145.79.227.2:4116 , piece_id=KEKK3M6M37G7Y7HXPULI2RSX5CFGUNCM3LILYNQOZ2PASM742ZGQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.602977247 (298h54m23.199748272s) [noise=true , address=145.79.227.48:4307 , piece_id=D6WSRVCL6UO4KQ542EKZWA6IP7FLA3ZUGSBXCSZUMLPFCJ5G5CQA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.603520888 (298h54m23.200291919s) [noise=true , address=79.127.219.45:4855 , piece_id=644Z2MAXJ423ZF4JGZ2YLPM6XJRZT7IUJCZRRE7ZJMHB3EW7KB4Q, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.603907549 (298h54m23.200678573s) [noise=true , address=145.79.227.36:4518 , piece_id=R5PDKAATCPFI3LOOUX6HJ4OP64NQ72N3RAOZS3GYFH5VSWPFU6OQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.604438080 (298h54m23.201209113s) [noise=true , address=45.91.116.11:4118 , piece_id=CVKN5WPP2OTRAHYWJKDLCJV3OZ3I6G5CLYXTSEKJDQD7LRTJ55EA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.618924566 (298h54m23.21569559s) [noise=true , address=79.127.219.46:4892 , piece_id=533AWUNWA66VX4AZF7Y3AANCF65GMESWZD3IJWBC5IOKB4QMVTZA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.619077190 (298h54m23.215848214s) [seg_pos={0 0}, upload=334781]: Satisfied cohort requirements after 13 pieces Mar 18 21:54:25.619546929 (298h54m23.216317961s) [noise=true , address=145.79.227.46:4604 , piece_id=DAGTJ24IRMED5FNGQHRYTXEIXYTDVSRW2IT23MC2UC45SES3PE7Q, seg_pos={0 0}, upload=334781]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy): protocol: expected piece hash; context canceled; EOF Mar 18 21:54:25.619578476 (298h54m23.21634951s) [noise=true , address=145.79.227.46:4604 , piece_id=DAGTJ24IRMED5FNGQHRYTXEIXYTDVSRW2IT23MC2UC45SES3PE7Q, seg_pos={0 0}, upload=334781]: Long tail cancellation due to optimal threshold Mar 18 21:54:25.620166659 (298h54m23.216937744s) [noise=true , address=145.79.227.19:4080 , piece_id=ANOMNO3HLII5GPWIT2RDX6P7WOBTVJXMOJM5PAPDMK7ST2NYJWMA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KK51wudKiZb39Ku7wZhgBYEYmgsZTHfjiNNLmpBU9uuNFapt3): protocol: expected piece hash; context canceled; EOF Mar 18 21:54:25.620178720 (298h54m23.216949746s) [noise=true , address=145.79.227.19:4080 , piece_id=ANOMNO3HLII5GPWIT2RDX6P7WOBTVJXMOJM5PAPDMK7ST2NYJWMA, seg_pos={0 0}, upload=334781]: Long tail cancellation due to optimal threshold Mar 18 21:54:25.620702987 (298h54m23.217474021s) [noise=true , address=145.79.227.37:4569 , piece_id=PTE4YT3YJ5M3NDKXKQOUFTGQ3T5FDAUZB74QHJBVH6HQL53QXLUA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:18TqoejkuHLqZWhEPe3QPH1weVXnWJhaeoi8BGkvjCTLW6t4qT): protocol: expected piece hash; context canceled; EOF Mar 18 21:54:25.620718441 (298h54m23.217489474s) [noise=true , address=145.79.227.37:4569 , piece_id=PTE4YT3YJ5M3NDKXKQOUFTGQ3T5FDAUZB74QHJBVH6HQL53QXLUA, seg_pos={0 0}, upload=334781]: Long tail cancellation due to optimal threshold Mar 18 21:54:25.620748329 (298h54m23.21751937s) [seg_pos={0 0}, upload=334781]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Mar 18 21:54:25.620912612 (298h54m23.21768364s) [upload=334781]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 18 21:54:25.620920854 (298h54m23.217691883s) [upload=334781]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 21:54:38.136688455 (298h54m35.733459485s) [upload=334781]: Done uploading segments... Mar 18 21:54:38.136713048 (298h54m35.73348408s) [upload=334781]: Upload finished. err: <nil> Duration: 468.226557ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 228.697369ms trace: https://jaeger.dev.storj.io/trace/2ab94940f3da3187 FAIL (2026-03-18 22:27:45 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 18 21:41:41.272084084 (298h41m38.868855112s) [upload=334559]: Starting upload Mar 18 21:41:41.272169036 (298h41m38.868940066s) [upload=334559]: Done starting upload Mar 18 21:41:41.272251155 (298h41m38.869022168s) [upload=334559]: Uploading segments... Mar 18 21:41:41.272358183 (298h41m38.869129211s) [upload=334559]: Got next segment. Inline: true Mar 18 21:41:41.272367512 (298h41m38.869138542s) [upload=334559]: Waiting for error group managing segments... Mar 18 21:41:41.272525860 (298h41m38.869296906s) [upload=334559]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 21:41:41.272550748 (298h41m38.869321776s) [upload=334559]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Mar 18 21:41:41.272555523 (298h41m38.86932655s) [upload=334559]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 21:42:30.253452076 (298h42m27.850223099s) [upload=334559]: Done uploading segments... Mar 18 21:42:30.253475139 (298h42m27.850246163s) [upload=334559]: Upload finished. err: <nil> Duration:* 228.726223ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 10 out of 60 is failed (threshold=3) [.XXXXXX.X..XXX..............................................] duration: 1.6874302s trace: https://jaeger.dev.storj.io/trace/27ed2876d5f3e85d FAIL (2026-03-18 22:27:30 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 1.687461339s core uplink us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 16 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXX............................................] duration: 735.453165ms trace: https://jaeger.dev.storj.io/trace/6cabdd7d47c9cb44 FAIL (2026-03-18 22:27:28 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 735.52412ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 213.149146ms trace: https://jaeger.dev.storj.io/trace/2ab94940f3da3187 FAIL (2026-03-18 22:27:21 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 18 21:41:41.272084084 (298h41m38.868855112s) [upload=334559]: Starting upload Mar 18 21:41:41.272169036 (298h41m38.868940066s) [upload=334559]: Done starting upload Mar 18 21:41:41.272251155 (298h41m38.869022168s) [upload=334559]: Uploading segments... Mar 18 21:41:41.272358183 (298h41m38.869129211s) [upload=334559]: Got next segment. Inline: true Mar 18 21:41:41.272367512 (298h41m38.869138542s) [upload=334559]: Waiting for error group managing segments... Mar 18 21:41:41.272525860 (298h41m38.869296906s) [upload=334559]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 21:41:41.272550748 (298h41m38.869321776s) [upload=334559]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Mar 18 21:41:41.272555523 (298h41m38.86932655s) [upload=334559]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 21:42:30.253452076 (298h42m27.850223099s) [upload=334559]: Done uploading segments... Mar 18 21:42:30.253475139 (298h42m27.850246163s) [upload=334559]: Upload finished. err: <nil> Duration:* 213.176219ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 11 out of 60 is failed (threshold=3) [XXXXXX.X...XX.X.X...........................................] duration: 293.012157ms trace: https://jaeger.dev.storj.io/trace/47004c0815b844e7 FAIL (2026-03-18 22:27:13 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 18 21:53:36.090826124 (298h53m33.687597155s) [upload=334767]: Starting upload Mar 18 21:53:36.090977797 (298h53m33.687748822s) [upload=334767]: Done starting upload Mar 18 21:53:36.091078093 (298h53m33.687849119s) [upload=334767]: Uploading segments... Mar 18 21:53:36.091268776 (298h53m33.688040083s) [upload=334767]: Got next segment. Inline: true Mar 18 21:53:36.091279074 (298h53m33.688050103s) [upload=334767]: Waiting for error group managing segments... Mar 18 21:53:36.091372978 (298h53m33.688144001s) [upload=334767]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 21:53:36.091379985 (298h53m33.688151008s) [upload=334767]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Mar 18 21:53:36.091384265 (298h53m33.688155287s) [upload=334767]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 21:54:04.718146694 (298h54m2.314917717s) [upload=334767]: Done uploading segments... Mar 18 21:54:04.718169125 (298h54m2.314940155s) [upload=334767]: Upload finished. err: <nil> Duration:* 293.125585ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 17 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXX...........................................] duration: 510.682711ms trace: https://jaeger.dev.storj.io/trace/6cabdd7d47c9cb44 FAIL (2026-03-18 22:27:08 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 510.742148ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [XXX.....XXXX.XXX............................................] duration: 495.134568ms trace: https://jaeger.dev.storj.io/trace/6dceff78c86f0774 FAIL (2026-03-18 22:27:06 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 18 21:54:17.190275520 (298h54m14.787046534s) [upload=334781]: Starting upload Mar 18 21:54:17.190367052 (298h54m14.787138068s) [upload=334781]: Done starting upload Mar 18 21:54:17.190448332 (298h54m14.787219348s) [upload=334781]: Uploading segments... Mar 18 21:54:17.190705184 (298h54m14.787476207s) [upload=334781]: Got next segment. Inline: false Mar 18 21:54:17.190783980 (298h54m14.787555002s) [upload=334781]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 21:54:17.190804859 (298h54m14.787575882s) [upload=334781]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 18 21:54:25.452304457 (298h54m23.049075487s) [seg_pos={0 0}, upload=334781]: Begin upload segment... Mar 18 21:54:25.452374690 (298h54m23.049145713s) [seg_pos={0 0}, upload=334781]: Done begin upload segment. Mar 18 21:54:25.452463335 (298h54m23.049234366s) [noise=true , address=145.79.227.46:4604 , piece_id=DAGTJ24IRMED5FNGQHRYTXEIXYTDVSRW2IT23MC2UC45SES3PE7Q, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.452479705 (298h54m23.049250739s) [noise=true , address=45.91.116.9:4264 , piece_id=EE3TXU5KSPHKMNNCI3BHIGFKQNYZQRBDO4YT3TTR5U66YJMFT6XQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.453224160 (298h54m23.049995188s) [noise=true , address=79.127.219.46:4892 , piece_id=533AWUNWA66VX4AZF7Y3AANCF65GMESWZD3IJWBC5IOKB4QMVTZA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.454147610 (298h54m23.050918643s) [noise=true , address=145.79.227.48:4307 , piece_id=D6WSRVCL6UO4KQ542EKZWA6IP7FLA3ZUGSBXCSZUMLPFCJ5G5CQA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.454790080 (298h54m23.051561121s) [noise=true , address=145.79.227.2:4116 , piece_id=KEKK3M6M37G7Y7HXPULI2RSX5CFGUNCM3LILYNQOZ2PASM742ZGQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.455678920 (298h54m23.052449952s) [noise=true , address=145.79.227.19:4080 , piece_id=ANOMNO3HLII5GPWIT2RDX6P7WOBTVJXMOJM5PAPDMK7ST2NYJWMA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.456387102 (298h54m23.053158136s) [noise=true , address=145.79.227.37:4569 , piece_id=PTE4YT3YJ5M3NDKXKQOUFTGQ3T5FDAUZB74QHJBVH6HQL53QXLUA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.456987483 (298h54m23.053758515s) [noise=true , address=145.79.227.4:4190 , piece_id=LDID6UGFJ4GWYK635S5OE6334DEULOPVSOLOCLYQCBOFTXRZEKRQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.457980795 (298h54m23.054751829s) [noise=true , address=79.127.219.45:4855 , piece_id=644Z2MAXJ423ZF4JGZ2YLPM6XJRZT7IUJCZRRE7ZJMHB3EW7KB4Q, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.459806473 (298h54m23.056577503s) [noise=true , address=145.79.227.50:4376 , piece_id=BHABIBKA3ZENQUNU2OVBVUJZVWEQRZD435EHA3MEB25KMNBVFQIA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.460305248 (298h54m23.057076281s) [noise=true , address=77.90.31.16:4737 , piece_id=SMQOC6BKSIEIE5FXEYZGOIWRZ6GORC7GCOVEOFZXPQX4FGILVIPA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.461552822 (298h54m23.058323848s) [noise=true , address=79.127.219.41:4926 , piece_id=AVSCKJ5WDK4OBWWUSLUQNOPZI7BGTTDGT4XJYSLQAJPUHRYY4KVQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.463144575 (298h54m23.059915608s) [noise=true , address=145.79.227.36:4518 , piece_id=R5PDKAATCPFI3LOOUX6HJ4OP64NQ72N3RAOZS3GYFH5VSWPFU6OQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.464837828 (298h54m23.06160886s) [noise=true , address=145.79.227.26:4656 , piece_id=DDNFTYMUJMFHTZXFHJTFUURSVOJBQX2KVT6UWLBCZ5TIOVBDVOVQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.466411504 (298h54m23.063182537s) [noise=true , address=145.79.227.6:4261 , piece_id=CYOWHNR7X3Y4YE5SFHPNDBOU5QQ6M6DZIRAO3VX56KQBL3OP2FLQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.469091278 (298h54m23.065862311s) [noise=true , address=45.91.116.11:4118 , piece_id=CVKN5WPP2OTRAHYWJKDLCJV3OZ3I6G5CLYXTSEKJDQD7LRTJ55EA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.586976347 (298h54m23.183747361s) [upload=334781]: Next returned nil segment Mar 18 21:54:25.587000505 (298h54m23.183771534s) [upload=334781]: Waiting for error group managing segments... Mar 18 21:54:25.596319798 (298h54m23.193090825s) [noise=true , address=79.127.219.41:4926 , piece_id=AVSCKJ5WDK4OBWWUSLUQNOPZI7BGTTDGT4XJYSLQAJPUHRYY4KVQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.596985876 (298h54m23.1937569s) [noise=true , address=45.91.116.9:4264 , piece_id=EE3TXU5KSPHKMNNCI3BHIGFKQNYZQRBDO4YT3TTR5U66YJMFT6XQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.598026901 (298h54m23.194797934s) [noise=true , address=145.79.227.26:4656 , piece_id=DDNFTYMUJMFHTZXFHJTFUURSVOJBQX2KVT6UWLBCZ5TIOVBDVOVQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.598566162 (298h54m23.195337186s) [noise=true , address=145.79.227.50:4376 , piece_id=BHABIBKA3ZENQUNU2OVBVUJZVWEQRZD435EHA3MEB25KMNBVFQIA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.598781105 (298h54m23.19555213s) [noise=true , address=77.90.31.16:4737 , piece_id=SMQOC6BKSIEIE5FXEYZGOIWRZ6GORC7GCOVEOFZXPQX4FGILVIPA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.600260475 (298h54m23.197031509s) [noise=true , address=145.79.227.6:4261 , piece_id=CYOWHNR7X3Y4YE5SFHPNDBOU5QQ6M6DZIRAO3VX56KQBL3OP2FLQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.600344398 (298h54m23.197115411s) [noise=true , address=145.79.227.4:4190 , piece_id=LDID6UGFJ4GWYK635S5OE6334DEULOPVSOLOCLYQCBOFTXRZEKRQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.602467676 (298h54m23.19923871s) [noise=true , address=145.79.227.2:4116 , piece_id=KEKK3M6M37G7Y7HXPULI2RSX5CFGUNCM3LILYNQOZ2PASM742ZGQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.602977247 (298h54m23.199748272s) [noise=true , address=145.79.227.48:4307 , piece_id=D6WSRVCL6UO4KQ542EKZWA6IP7FLA3ZUGSBXCSZUMLPFCJ5G5CQA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.603520888 (298h54m23.200291919s) [noise=true , address=79.127.219.45:4855 , piece_id=644Z2MAXJ423ZF4JGZ2YLPM6XJRZT7IUJCZRRE7ZJMHB3EW7KB4Q, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.603907549 (298h54m23.200678573s) [noise=true , address=145.79.227.36:4518 , piece_id=R5PDKAATCPFI3LOOUX6HJ4OP64NQ72N3RAOZS3GYFH5VSWPFU6OQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.604438080 (298h54m23.201209113s) [noise=true , address=45.91.116.11:4118 , piece_id=CVKN5WPP2OTRAHYWJKDLCJV3OZ3I6G5CLYXTSEKJDQD7LRTJ55EA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.618924566 (298h54m23.21569559s) [noise=true , address=79.127.219.46:4892 , piece_id=533AWUNWA66VX4AZF7Y3AANCF65GMESWZD3IJWBC5IOKB4QMVTZA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.619077190 (298h54m23.215848214s) [seg_pos={0 0}, upload=334781]: Satisfied cohort requirements after 13 pieces Mar 18 21:54:25.619546929 (298h54m23.216317961s) [noise=true , address=145.79.227.46:4604 , piece_id=DAGTJ24IRMED5FNGQHRYTXEIXYTDVSRW2IT23MC2UC45SES3PE7Q, seg_pos={0 0}, upload=334781]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy): protocol: expected piece hash; context canceled; EOF Mar 18 21:54:25.619578476 (298h54m23.21634951s) [noise=true , address=145.79.227.46:4604 , piece_id=DAGTJ24IRMED5FNGQHRYTXEIXYTDVSRW2IT23MC2UC45SES3PE7Q, seg_pos={0 0}, upload=334781]: Long tail cancellation due to optimal threshold Mar 18 21:54:25.620166659 (298h54m23.216937744s) [noise=true , address=145.79.227.19:4080 , piece_id=ANOMNO3HLII5GPWIT2RDX6P7WOBTVJXMOJM5PAPDMK7ST2NYJWMA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KK51wudKiZb39Ku7wZhgBYEYmgsZTHfjiNNLmpBU9uuNFapt3): protocol: expected piece hash; context canceled; EOF Mar 18 21:54:25.620178720 (298h54m23.216949746s) [noise=true , address=145.79.227.19:4080 , piece_id=ANOMNO3HLII5GPWIT2RDX6P7WOBTVJXMOJM5PAPDMK7ST2NYJWMA, seg_pos={0 0}, upload=334781]: Long tail cancellation due to optimal threshold Mar 18 21:54:25.620702987 (298h54m23.217474021s) [noise=true , address=145.79.227.37:4569 , piece_id=PTE4YT3YJ5M3NDKXKQOUFTGQ3T5FDAUZB74QHJBVH6HQL53QXLUA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:18TqoejkuHLqZWhEPe3QPH1weVXnWJhaeoi8BGkvjCTLW6t4qT): protocol: expected piece hash; context canceled; EOF Mar 18 21:54:25.620718441 (298h54m23.217489474s) [noise=true , address=145.79.227.37:4569 , piece_id=PTE4YT3YJ5M3NDKXKQOUFTGQ3T5FDAUZB74QHJBVH6HQL53QXLUA, seg_pos={0 0}, upload=334781]: Long tail cancellation due to optimal threshold Mar 18 21:54:25.620748329 (298h54m23.21751937s) [seg_pos={0 0}, upload=334781]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Mar 18 21:54:25.620912612 (298h54m23.21768364s) [upload=334781]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 18 21:54:25.620920854 (298h54m23.217691883s) [upload=334781]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 21:54:38.136688455 (298h54m35.733459485s) [upload=334781]: Done uploading segments... Mar 18 21:54:38.136713048 (298h54m35.73348408s) [upload=334781]: Upload finished. err: <nil> Duration: 495.165194ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 11 out of 60 is failed (threshold=3) [X.XXXXXX.X..XXX.............................................] duration: 1.633209041s trace: https://jaeger.dev.storj.io/trace/27ed2876d5f3e85d FAIL (2026-03-18 22:27:02 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 1.633295952s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 10 out of 60 is failed (threshold=3) [XXX..XX.X..X..X.XX..........................................] duration: 229.551924ms trace: https://jaeger.dev.storj.io/trace/7c4371cfa8f7834c FAIL (2026-03-18 22:26:54 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 229.639758ms core uplink us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 18 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXX..........................................] duration: 549.392398ms trace: https://jaeger.dev.storj.io/trace/6cabdd7d47c9cb44 FAIL (2026-03-18 22:26:45 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 549.434365ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 193.920113ms trace: https://jaeger.dev.storj.io/trace/2ab94940f3da3187 FAIL (2026-03-18 22:26:40 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 18 21:41:41.272084084 (298h41m38.868855112s) [upload=334559]: Starting upload Mar 18 21:41:41.272169036 (298h41m38.868940066s) [upload=334559]: Done starting upload Mar 18 21:41:41.272251155 (298h41m38.869022168s) [upload=334559]: Uploading segments... Mar 18 21:41:41.272358183 (298h41m38.869129211s) [upload=334559]: Got next segment. Inline: true Mar 18 21:41:41.272367512 (298h41m38.869138542s) [upload=334559]: Waiting for error group managing segments... Mar 18 21:41:41.272525860 (298h41m38.869296906s) [upload=334559]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 21:41:41.272550748 (298h41m38.869321776s) [upload=334559]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Mar 18 21:41:41.272555523 (298h41m38.86932655s) [upload=334559]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 21:42:30.253452076 (298h42m27.850223099s) [upload=334559]: Done uploading segments... Mar 18 21:42:30.253475139 (298h42m27.850246163s) [upload=334559]: Upload finished. err: <nil> Duration:* 193.952505ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 11 out of 60 is failed (threshold=3) [.XXXXXX.X...XX.X.X..........................................] duration: 271.725436ms trace: https://jaeger.dev.storj.io/trace/47004c0815b844e7 FAIL (2026-03-18 22:26:26 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 18 21:53:36.090826124 (298h53m33.687597155s) [upload=334767]: Starting upload Mar 18 21:53:36.090977797 (298h53m33.687748822s) [upload=334767]: Done starting upload Mar 18 21:53:36.091078093 (298h53m33.687849119s) [upload=334767]: Uploading segments... Mar 18 21:53:36.091268776 (298h53m33.688040083s) [upload=334767]: Got next segment. Inline: true Mar 18 21:53:36.091279074 (298h53m33.688050103s) [upload=334767]: Waiting for error group managing segments... Mar 18 21:53:36.091372978 (298h53m33.688144001s) [upload=334767]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 21:53:36.091379985 (298h53m33.688151008s) [upload=334767]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Mar 18 21:53:36.091384265 (298h53m33.688155287s) [upload=334767]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 21:54:04.718146694 (298h54m2.314917717s) [upload=334767]: Done uploading segments... Mar 18 21:54:04.718169125 (298h54m2.314940155s) [upload=334767]: Upload finished. err: <nil> Duration:* 271.757015ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 19 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXX.........................................] duration: 510.856872ms trace: https://jaeger.dev.storj.io/trace/6cabdd7d47c9cb44 FAIL (2026-03-18 22:26:23 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 510.887299ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [XXXX.....XXXX.XXX...........................................] duration: 474.619177ms trace: https://jaeger.dev.storj.io/trace/6dceff78c86f0774 FAIL (2026-03-18 22:26:17 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 18 21:54:17.190275520 (298h54m14.787046534s) [upload=334781]: Starting upload Mar 18 21:54:17.190367052 (298h54m14.787138068s) [upload=334781]: Done starting upload Mar 18 21:54:17.190448332 (298h54m14.787219348s) [upload=334781]: Uploading segments... Mar 18 21:54:17.190705184 (298h54m14.787476207s) [upload=334781]: Got next segment. Inline: false Mar 18 21:54:17.190783980 (298h54m14.787555002s) [upload=334781]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 21:54:17.190804859 (298h54m14.787575882s) [upload=334781]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 18 21:54:25.452304457 (298h54m23.049075487s) [seg_pos={0 0}, upload=334781]: Begin upload segment... Mar 18 21:54:25.452374690 (298h54m23.049145713s) [seg_pos={0 0}, upload=334781]: Done begin upload segment. Mar 18 21:54:25.452463335 (298h54m23.049234366s) [noise=true , address=145.79.227.46:4604 , piece_id=DAGTJ24IRMED5FNGQHRYTXEIXYTDVSRW2IT23MC2UC45SES3PE7Q, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.452479705 (298h54m23.049250739s) [noise=true , address=45.91.116.9:4264 , piece_id=EE3TXU5KSPHKMNNCI3BHIGFKQNYZQRBDO4YT3TTR5U66YJMFT6XQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.453224160 (298h54m23.049995188s) [noise=true , address=79.127.219.46:4892 , piece_id=533AWUNWA66VX4AZF7Y3AANCF65GMESWZD3IJWBC5IOKB4QMVTZA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.454147610 (298h54m23.050918643s) [noise=true , address=145.79.227.48:4307 , piece_id=D6WSRVCL6UO4KQ542EKZWA6IP7FLA3ZUGSBXCSZUMLPFCJ5G5CQA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.454790080 (298h54m23.051561121s) [noise=true , address=145.79.227.2:4116 , piece_id=KEKK3M6M37G7Y7HXPULI2RSX5CFGUNCM3LILYNQOZ2PASM742ZGQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.455678920 (298h54m23.052449952s) [noise=true , address=145.79.227.19:4080 , piece_id=ANOMNO3HLII5GPWIT2RDX6P7WOBTVJXMOJM5PAPDMK7ST2NYJWMA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.456387102 (298h54m23.053158136s) [noise=true , address=145.79.227.37:4569 , piece_id=PTE4YT3YJ5M3NDKXKQOUFTGQ3T5FDAUZB74QHJBVH6HQL53QXLUA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.456987483 (298h54m23.053758515s) [noise=true , address=145.79.227.4:4190 , piece_id=LDID6UGFJ4GWYK635S5OE6334DEULOPVSOLOCLYQCBOFTXRZEKRQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.457980795 (298h54m23.054751829s) [noise=true , address=79.127.219.45:4855 , piece_id=644Z2MAXJ423ZF4JGZ2YLPM6XJRZT7IUJCZRRE7ZJMHB3EW7KB4Q, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.459806473 (298h54m23.056577503s) [noise=true , address=145.79.227.50:4376 , piece_id=BHABIBKA3ZENQUNU2OVBVUJZVWEQRZD435EHA3MEB25KMNBVFQIA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.460305248 (298h54m23.057076281s) [noise=true , address=77.90.31.16:4737 , piece_id=SMQOC6BKSIEIE5FXEYZGOIWRZ6GORC7GCOVEOFZXPQX4FGILVIPA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.461552822 (298h54m23.058323848s) [noise=true , address=79.127.219.41:4926 , piece_id=AVSCKJ5WDK4OBWWUSLUQNOPZI7BGTTDGT4XJYSLQAJPUHRYY4KVQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.463144575 (298h54m23.059915608s) [noise=true , address=145.79.227.36:4518 , piece_id=R5PDKAATCPFI3LOOUX6HJ4OP64NQ72N3RAOZS3GYFH5VSWPFU6OQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.464837828 (298h54m23.06160886s) [noise=true , address=145.79.227.26:4656 , piece_id=DDNFTYMUJMFHTZXFHJTFUURSVOJBQX2KVT6UWLBCZ5TIOVBDVOVQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.466411504 (298h54m23.063182537s) [noise=true , address=145.79.227.6:4261 , piece_id=CYOWHNR7X3Y4YE5SFHPNDBOU5QQ6M6DZIRAO3VX56KQBL3OP2FLQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.469091278 (298h54m23.065862311s) [noise=true , address=45.91.116.11:4118 , piece_id=CVKN5WPP2OTRAHYWJKDLCJV3OZ3I6G5CLYXTSEKJDQD7LRTJ55EA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.586976347 (298h54m23.183747361s) [upload=334781]: Next returned nil segment Mar 18 21:54:25.587000505 (298h54m23.183771534s) [upload=334781]: Waiting for error group managing segments... Mar 18 21:54:25.596319798 (298h54m23.193090825s) [noise=true , address=79.127.219.41:4926 , piece_id=AVSCKJ5WDK4OBWWUSLUQNOPZI7BGTTDGT4XJYSLQAJPUHRYY4KVQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.596985876 (298h54m23.1937569s) [noise=true , address=45.91.116.9:4264 , piece_id=EE3TXU5KSPHKMNNCI3BHIGFKQNYZQRBDO4YT3TTR5U66YJMFT6XQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.598026901 (298h54m23.194797934s) [noise=true , address=145.79.227.26:4656 , piece_id=DDNFTYMUJMFHTZXFHJTFUURSVOJBQX2KVT6UWLBCZ5TIOVBDVOVQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.598566162 (298h54m23.195337186s) [noise=true , address=145.79.227.50:4376 , piece_id=BHABIBKA3ZENQUNU2OVBVUJZVWEQRZD435EHA3MEB25KMNBVFQIA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.598781105 (298h54m23.19555213s) [noise=true , address=77.90.31.16:4737 , piece_id=SMQOC6BKSIEIE5FXEYZGOIWRZ6GORC7GCOVEOFZXPQX4FGILVIPA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.600260475 (298h54m23.197031509s) [noise=true , address=145.79.227.6:4261 , piece_id=CYOWHNR7X3Y4YE5SFHPNDBOU5QQ6M6DZIRAO3VX56KQBL3OP2FLQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.600344398 (298h54m23.197115411s) [noise=true , address=145.79.227.4:4190 , piece_id=LDID6UGFJ4GWYK635S5OE6334DEULOPVSOLOCLYQCBOFTXRZEKRQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.602467676 (298h54m23.19923871s) [noise=true , address=145.79.227.2:4116 , piece_id=KEKK3M6M37G7Y7HXPULI2RSX5CFGUNCM3LILYNQOZ2PASM742ZGQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.602977247 (298h54m23.199748272s) [noise=true , address=145.79.227.48:4307 , piece_id=D6WSRVCL6UO4KQ542EKZWA6IP7FLA3ZUGSBXCSZUMLPFCJ5G5CQA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.603520888 (298h54m23.200291919s) [noise=true , address=79.127.219.45:4855 , piece_id=644Z2MAXJ423ZF4JGZ2YLPM6XJRZT7IUJCZRRE7ZJMHB3EW7KB4Q, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.603907549 (298h54m23.200678573s) [noise=true , address=145.79.227.36:4518 , piece_id=R5PDKAATCPFI3LOOUX6HJ4OP64NQ72N3RAOZS3GYFH5VSWPFU6OQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.604438080 (298h54m23.201209113s) [noise=true , address=45.91.116.11:4118 , piece_id=CVKN5WPP2OTRAHYWJKDLCJV3OZ3I6G5CLYXTSEKJDQD7LRTJ55EA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.618924566 (298h54m23.21569559s) [noise=true , address=79.127.219.46:4892 , piece_id=533AWUNWA66VX4AZF7Y3AANCF65GMESWZD3IJWBC5IOKB4QMVTZA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.619077190 (298h54m23.215848214s) [seg_pos={0 0}, upload=334781]: Satisfied cohort requirements after 13 pieces Mar 18 21:54:25.619546929 (298h54m23.216317961s) [noise=true , address=145.79.227.46:4604 , piece_id=DAGTJ24IRMED5FNGQHRYTXEIXYTDVSRW2IT23MC2UC45SES3PE7Q, seg_pos={0 0}, upload=334781]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy): protocol: expected piece hash; context canceled; EOF Mar 18 21:54:25.619578476 (298h54m23.21634951s) [noise=true , address=145.79.227.46:4604 , piece_id=DAGTJ24IRMED5FNGQHRYTXEIXYTDVSRW2IT23MC2UC45SES3PE7Q, seg_pos={0 0}, upload=334781]: Long tail cancellation due to optimal threshold Mar 18 21:54:25.620166659 (298h54m23.216937744s) [noise=true , address=145.79.227.19:4080 , piece_id=ANOMNO3HLII5GPWIT2RDX6P7WOBTVJXMOJM5PAPDMK7ST2NYJWMA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KK51wudKiZb39Ku7wZhgBYEYmgsZTHfjiNNLmpBU9uuNFapt3): protocol: expected piece hash; context canceled; EOF Mar 18 21:54:25.620178720 (298h54m23.216949746s) [noise=true , address=145.79.227.19:4080 , piece_id=ANOMNO3HLII5GPWIT2RDX6P7WOBTVJXMOJM5PAPDMK7ST2NYJWMA, seg_pos={0 0}, upload=334781]: Long tail cancellation due to optimal threshold Mar 18 21:54:25.620702987 (298h54m23.217474021s) [noise=true , address=145.79.227.37:4569 , piece_id=PTE4YT3YJ5M3NDKXKQOUFTGQ3T5FDAUZB74QHJBVH6HQL53QXLUA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:18TqoejkuHLqZWhEPe3QPH1weVXnWJhaeoi8BGkvjCTLW6t4qT): protocol: expected piece hash; context canceled; EOF Mar 18 21:54:25.620718441 (298h54m23.217489474s) [noise=true , address=145.79.227.37:4569 , piece_id=PTE4YT3YJ5M3NDKXKQOUFTGQ3T5FDAUZB74QHJBVH6HQL53QXLUA, seg_pos={0 0}, upload=334781]: Long tail cancellation due to optimal threshold Mar 18 21:54:25.620748329 (298h54m23.21751937s) [seg_pos={0 0}, upload=334781]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Mar 18 21:54:25.620912612 (298h54m23.21768364s) [upload=334781]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 18 21:54:25.620920854 (298h54m23.217691883s) [upload=334781]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 21:54:38.136688455 (298h54m35.733459485s) [upload=334781]: Done uploading segments... Mar 18 21:54:38.136713048 (298h54m35.73348408s) [upload=334781]: Upload finished. err: <nil> Duration: 474.718058ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 20 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXX........................................] duration: 559.829668ms trace: https://jaeger.dev.storj.io/trace/6cabdd7d47c9cb44 FAIL (2026-03-18 22:26:03 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 559.920829ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 10 out of 60 is failed (threshold=3) [.XXX..XX.X..X..X.XX.........................................] duration: 246.867329ms trace: https://jaeger.dev.storj.io/trace/7c4371cfa8f7834c FAIL (2026-03-18 22:25:58 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 246.894158ms core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 11 out of 60 is failed (threshold=3) [.X.XXXXXX.X..XXX............................................] duration: 1.781932375s trace: https://jaeger.dev.storj.io/trace/27ed2876d5f3e85d FAIL (2026-03-18 22:25:56 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 1.781961131s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [XXXXX.....XXXX.XXX..........................................] duration: 464.276721ms trace: https://jaeger.dev.storj.io/trace/6dceff78c86f0774 FAIL (2026-03-18 22:25:50 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 18 21:54:17.190275520 (298h54m14.787046534s) [upload=334781]: Starting upload Mar 18 21:54:17.190367052 (298h54m14.787138068s) [upload=334781]: Done starting upload Mar 18 21:54:17.190448332 (298h54m14.787219348s) [upload=334781]: Uploading segments... Mar 18 21:54:17.190705184 (298h54m14.787476207s) [upload=334781]: Got next segment. Inline: false Mar 18 21:54:17.190783980 (298h54m14.787555002s) [upload=334781]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 21:54:17.190804859 (298h54m14.787575882s) [upload=334781]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 18 21:54:25.452304457 (298h54m23.049075487s) [seg_pos={0 0}, upload=334781]: Begin upload segment... Mar 18 21:54:25.452374690 (298h54m23.049145713s) [seg_pos={0 0}, upload=334781]: Done begin upload segment. Mar 18 21:54:25.452463335 (298h54m23.049234366s) [noise=true , address=145.79.227.46:4604 , piece_id=DAGTJ24IRMED5FNGQHRYTXEIXYTDVSRW2IT23MC2UC45SES3PE7Q, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.452479705 (298h54m23.049250739s) [noise=true , address=45.91.116.9:4264 , piece_id=EE3TXU5KSPHKMNNCI3BHIGFKQNYZQRBDO4YT3TTR5U66YJMFT6XQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.453224160 (298h54m23.049995188s) [noise=true , address=79.127.219.46:4892 , piece_id=533AWUNWA66VX4AZF7Y3AANCF65GMESWZD3IJWBC5IOKB4QMVTZA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.454147610 (298h54m23.050918643s) [noise=true , address=145.79.227.48:4307 , piece_id=D6WSRVCL6UO4KQ542EKZWA6IP7FLA3ZUGSBXCSZUMLPFCJ5G5CQA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.454790080 (298h54m23.051561121s) [noise=true , address=145.79.227.2:4116 , piece_id=KEKK3M6M37G7Y7HXPULI2RSX5CFGUNCM3LILYNQOZ2PASM742ZGQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.455678920 (298h54m23.052449952s) [noise=true , address=145.79.227.19:4080 , piece_id=ANOMNO3HLII5GPWIT2RDX6P7WOBTVJXMOJM5PAPDMK7ST2NYJWMA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.456387102 (298h54m23.053158136s) [noise=true , address=145.79.227.37:4569 , piece_id=PTE4YT3YJ5M3NDKXKQOUFTGQ3T5FDAUZB74QHJBVH6HQL53QXLUA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.456987483 (298h54m23.053758515s) [noise=true , address=145.79.227.4:4190 , piece_id=LDID6UGFJ4GWYK635S5OE6334DEULOPVSOLOCLYQCBOFTXRZEKRQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.457980795 (298h54m23.054751829s) [noise=true , address=79.127.219.45:4855 , piece_id=644Z2MAXJ423ZF4JGZ2YLPM6XJRZT7IUJCZRRE7ZJMHB3EW7KB4Q, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.459806473 (298h54m23.056577503s) [noise=true , address=145.79.227.50:4376 , piece_id=BHABIBKA3ZENQUNU2OVBVUJZVWEQRZD435EHA3MEB25KMNBVFQIA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.460305248 (298h54m23.057076281s) [noise=true , address=77.90.31.16:4737 , piece_id=SMQOC6BKSIEIE5FXEYZGOIWRZ6GORC7GCOVEOFZXPQX4FGILVIPA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.461552822 (298h54m23.058323848s) [noise=true , address=79.127.219.41:4926 , piece_id=AVSCKJ5WDK4OBWWUSLUQNOPZI7BGTTDGT4XJYSLQAJPUHRYY4KVQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.463144575 (298h54m23.059915608s) [noise=true , address=145.79.227.36:4518 , piece_id=R5PDKAATCPFI3LOOUX6HJ4OP64NQ72N3RAOZS3GYFH5VSWPFU6OQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.464837828 (298h54m23.06160886s) [noise=true , address=145.79.227.26:4656 , piece_id=DDNFTYMUJMFHTZXFHJTFUURSVOJBQX2KVT6UWLBCZ5TIOVBDVOVQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.466411504 (298h54m23.063182537s) [noise=true , address=145.79.227.6:4261 , piece_id=CYOWHNR7X3Y4YE5SFHPNDBOU5QQ6M6DZIRAO3VX56KQBL3OP2FLQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.469091278 (298h54m23.065862311s) [noise=true , address=45.91.116.11:4118 , piece_id=CVKN5WPP2OTRAHYWJKDLCJV3OZ3I6G5CLYXTSEKJDQD7LRTJ55EA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.586976347 (298h54m23.183747361s) [upload=334781]: Next returned nil segment Mar 18 21:54:25.587000505 (298h54m23.183771534s) [upload=334781]: Waiting for error group managing segments... Mar 18 21:54:25.596319798 (298h54m23.193090825s) [noise=true , address=79.127.219.41:4926 , piece_id=AVSCKJ5WDK4OBWWUSLUQNOPZI7BGTTDGT4XJYSLQAJPUHRYY4KVQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.596985876 (298h54m23.1937569s) [noise=true , address=45.91.116.9:4264 , piece_id=EE3TXU5KSPHKMNNCI3BHIGFKQNYZQRBDO4YT3TTR5U66YJMFT6XQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.598026901 (298h54m23.194797934s) [noise=true , address=145.79.227.26:4656 , piece_id=DDNFTYMUJMFHTZXFHJTFUURSVOJBQX2KVT6UWLBCZ5TIOVBDVOVQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.598566162 (298h54m23.195337186s) [noise=true , address=145.79.227.50:4376 , piece_id=BHABIBKA3ZENQUNU2OVBVUJZVWEQRZD435EHA3MEB25KMNBVFQIA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.598781105 (298h54m23.19555213s) [noise=true , address=77.90.31.16:4737 , piece_id=SMQOC6BKSIEIE5FXEYZGOIWRZ6GORC7GCOVEOFZXPQX4FGILVIPA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.600260475 (298h54m23.197031509s) [noise=true , address=145.79.227.6:4261 , piece_id=CYOWHNR7X3Y4YE5SFHPNDBOU5QQ6M6DZIRAO3VX56KQBL3OP2FLQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.600344398 (298h54m23.197115411s) [noise=true , address=145.79.227.4:4190 , piece_id=LDID6UGFJ4GWYK635S5OE6334DEULOPVSOLOCLYQCBOFTXRZEKRQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.602467676 (298h54m23.19923871s) [noise=true , address=145.79.227.2:4116 , piece_id=KEKK3M6M37G7Y7HXPULI2RSX5CFGUNCM3LILYNQOZ2PASM742ZGQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.602977247 (298h54m23.199748272s) [noise=true , address=145.79.227.48:4307 , piece_id=D6WSRVCL6UO4KQ542EKZWA6IP7FLA3ZUGSBXCSZUMLPFCJ5G5CQA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.603520888 (298h54m23.200291919s) [noise=true , address=79.127.219.45:4855 , piece_id=644Z2MAXJ423ZF4JGZ2YLPM6XJRZT7IUJCZRRE7ZJMHB3EW7KB4Q, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.603907549 (298h54m23.200678573s) [noise=true , address=145.79.227.36:4518 , piece_id=R5PDKAATCPFI3LOOUX6HJ4OP64NQ72N3RAOZS3GYFH5VSWPFU6OQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.604438080 (298h54m23.201209113s) [noise=true , address=45.91.116.11:4118 , piece_id=CVKN5WPP2OTRAHYWJKDLCJV3OZ3I6G5CLYXTSEKJDQD7LRTJ55EA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.618924566 (298h54m23.21569559s) [noise=true , address=79.127.219.46:4892 , piece_id=533AWUNWA66VX4AZF7Y3AANCF65GMESWZD3IJWBC5IOKB4QMVTZA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.619077190 (298h54m23.215848214s) [seg_pos={0 0}, upload=334781]: Satisfied cohort requirements after 13 pieces Mar 18 21:54:25.619546929 (298h54m23.216317961s) [noise=true , address=145.79.227.46:4604 , piece_id=DAGTJ24IRMED5FNGQHRYTXEIXYTDVSRW2IT23MC2UC45SES3PE7Q, seg_pos={0 0}, upload=334781]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy): protocol: expected piece hash; context canceled; EOF Mar 18 21:54:25.619578476 (298h54m23.21634951s) [noise=true , address=145.79.227.46:4604 , piece_id=DAGTJ24IRMED5FNGQHRYTXEIXYTDVSRW2IT23MC2UC45SES3PE7Q, seg_pos={0 0}, upload=334781]: Long tail cancellation due to optimal threshold Mar 18 21:54:25.620166659 (298h54m23.216937744s) [noise=true , address=145.79.227.19:4080 , piece_id=ANOMNO3HLII5GPWIT2RDX6P7WOBTVJXMOJM5PAPDMK7ST2NYJWMA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KK51wudKiZb39Ku7wZhgBYEYmgsZTHfjiNNLmpBU9uuNFapt3): protocol: expected piece hash; context canceled; EOF Mar 18 21:54:25.620178720 (298h54m23.216949746s) [noise=true , address=145.79.227.19:4080 , piece_id=ANOMNO3HLII5GPWIT2RDX6P7WOBTVJXMOJM5PAPDMK7ST2NYJWMA, seg_pos={0 0}, upload=334781]: Long tail cancellation due to optimal threshold Mar 18 21:54:25.620702987 (298h54m23.217474021s) [noise=true , address=145.79.227.37:4569 , piece_id=PTE4YT3YJ5M3NDKXKQOUFTGQ3T5FDAUZB74QHJBVH6HQL53QXLUA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:18TqoejkuHLqZWhEPe3QPH1weVXnWJhaeoi8BGkvjCTLW6t4qT): protocol: expected piece hash; context canceled; EOF Mar 18 21:54:25.620718441 (298h54m23.217489474s) [noise=true , address=145.79.227.37:4569 , piece_id=PTE4YT3YJ5M3NDKXKQOUFTGQ3T5FDAUZB74QHJBVH6HQL53QXLUA, seg_pos={0 0}, upload=334781]: Long tail cancellation due to optimal threshold Mar 18 21:54:25.620748329 (298h54m23.21751937s) [seg_pos={0 0}, upload=334781]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Mar 18 21:54:25.620912612 (298h54m23.21768364s) [upload=334781]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 18 21:54:25.620920854 (298h54m23.217691883s) [upload=334781]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 21:54:38.136688455 (298h54m35.733459485s) [upload=334781]: Done uploading segments... Mar 18 21:54:38.136713048 (298h54m35.73348408s) [upload=334781]: Upload finished. err: <nil> Duration: 464.308242ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 11 out of 60 is failed (threshold=3) [..XXXXXX.X...XX.X.X.........................................] duration: 268.860569ms trace: https://jaeger.dev.storj.io/trace/47004c0815b844e7 FAIL (2026-03-18 22:25:49 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 18 21:53:36.090826124 (298h53m33.687597155s) [upload=334767]: Starting upload Mar 18 21:53:36.090977797 (298h53m33.687748822s) [upload=334767]: Done starting upload Mar 18 21:53:36.091078093 (298h53m33.687849119s) [upload=334767]: Uploading segments... Mar 18 21:53:36.091268776 (298h53m33.688040083s) [upload=334767]: Got next segment. Inline: true Mar 18 21:53:36.091279074 (298h53m33.688050103s) [upload=334767]: Waiting for error group managing segments... Mar 18 21:53:36.091372978 (298h53m33.688144001s) [upload=334767]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 21:53:36.091379985 (298h53m33.688151008s) [upload=334767]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Mar 18 21:53:36.091384265 (298h53m33.688155287s) [upload=334767]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 21:54:04.718146694 (298h54m2.314917717s) [upload=334767]: Done uploading segments... Mar 18 21:54:04.718169125 (298h54m2.314940155s) [upload=334767]: Upload finished. err: <nil> Duration:* 268.904948ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 193.334922ms trace: https://jaeger.dev.storj.io/trace/2ab94940f3da3187 FAIL (2026-03-18 22:25:25 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 18 21:41:41.272084084 (298h41m38.868855112s) [upload=334559]: Starting upload Mar 18 21:41:41.272169036 (298h41m38.868940066s) [upload=334559]: Done starting upload Mar 18 21:41:41.272251155 (298h41m38.869022168s) [upload=334559]: Uploading segments... Mar 18 21:41:41.272358183 (298h41m38.869129211s) [upload=334559]: Got next segment. Inline: true Mar 18 21:41:41.272367512 (298h41m38.869138542s) [upload=334559]: Waiting for error group managing segments... Mar 18 21:41:41.272525860 (298h41m38.869296906s) [upload=334559]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 21:41:41.272550748 (298h41m38.869321776s) [upload=334559]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Mar 18 21:41:41.272555523 (298h41m38.86932655s) [upload=334559]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 21:42:30.253452076 (298h42m27.850223099s) [upload=334559]: Done uploading segments... Mar 18 21:42:30.253475139 (298h42m27.850246163s) [upload=334559]: Upload finished. err: <nil> Duration:* 193.40782ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 10 out of 60 is failed (threshold=3) [..XXX..XX.X..X..X.XX........................................] duration: 267.709044ms trace: https://jaeger.dev.storj.io/trace/7c4371cfa8f7834c FAIL (2026-03-18 22:25:19 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 267.741316ms core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 11 out of 60 is failed (threshold=3) [...XXXXXX.X...XX.X.X........................................] duration: 314.104521ms trace: https://jaeger.dev.storj.io/trace/47004c0815b844e7 FAIL (2026-03-18 22:25:00 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 18 21:53:36.090826124 (298h53m33.687597155s) [upload=334767]: Starting upload Mar 18 21:53:36.090977797 (298h53m33.687748822s) [upload=334767]: Done starting upload Mar 18 21:53:36.091078093 (298h53m33.687849119s) [upload=334767]: Uploading segments... Mar 18 21:53:36.091268776 (298h53m33.688040083s) [upload=334767]: Got next segment. Inline: true Mar 18 21:53:36.091279074 (298h53m33.688050103s) [upload=334767]: Waiting for error group managing segments... Mar 18 21:53:36.091372978 (298h53m33.688144001s) [upload=334767]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 21:53:36.091379985 (298h53m33.688151008s) [upload=334767]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Mar 18 21:53:36.091384265 (298h53m33.688155287s) [upload=334767]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 21:54:04.718146694 (298h54m2.314917717s) [upload=334767]: Done uploading segments... Mar 18 21:54:04.718169125 (298h54m2.314940155s) [upload=334767]: Upload finished. err: <nil> Duration:* 314.134216ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 192.199705ms trace: https://jaeger.dev.storj.io/trace/2ab94940f3da3187 FAIL (2026-03-18 22:24:54 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 18 21:41:41.272084084 (298h41m38.868855112s) [upload=334559]: Starting upload Mar 18 21:41:41.272169036 (298h41m38.868940066s) [upload=334559]: Done starting upload Mar 18 21:41:41.272251155 (298h41m38.869022168s) [upload=334559]: Uploading segments... Mar 18 21:41:41.272358183 (298h41m38.869129211s) [upload=334559]: Got next segment. Inline: true Mar 18 21:41:41.272367512 (298h41m38.869138542s) [upload=334559]: Waiting for error group managing segments... Mar 18 21:41:41.272525860 (298h41m38.869296906s) [upload=334559]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 21:41:41.272550748 (298h41m38.869321776s) [upload=334559]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Mar 18 21:41:41.272555523 (298h41m38.86932655s) [upload=334559]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 21:42:30.253452076 (298h42m27.850223099s) [upload=334559]: Done uploading segments... Mar 18 21:42:30.253475139 (298h42m27.850246163s) [upload=334559]: Upload finished. err: <nil> Duration:* 192.225028ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 11 out of 60 is failed (threshold=3) [..X.XXXXXX.X..XXX...........................................] duration: 1.750652814s trace: https://jaeger.dev.storj.io/trace/27ed2876d5f3e85d FAIL (2026-03-18 22:24:52 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 1.750688251s core uplink us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 20 out of 60 is failed (threshold=6) [.XXXXXXXXXXXXXXXXXXXX.......................................] duration: 677.050573ms trace: https://jaeger.dev.storj.io/trace/6cabdd7d47c9cb44 FAIL (2026-03-18 22:24:49 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 677.075551ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [XXXXXX.....XXXX.XXX.........................................] duration: 537.830263ms trace: https://jaeger.dev.storj.io/trace/6dceff78c86f0774 FAIL (2026-03-18 22:24:47 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 18 21:54:17.190275520 (298h54m14.787046534s) [upload=334781]: Starting upload Mar 18 21:54:17.190367052 (298h54m14.787138068s) [upload=334781]: Done starting upload Mar 18 21:54:17.190448332 (298h54m14.787219348s) [upload=334781]: Uploading segments... Mar 18 21:54:17.190705184 (298h54m14.787476207s) [upload=334781]: Got next segment. Inline: false Mar 18 21:54:17.190783980 (298h54m14.787555002s) [upload=334781]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 21:54:17.190804859 (298h54m14.787575882s) [upload=334781]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 18 21:54:25.452304457 (298h54m23.049075487s) [seg_pos={0 0}, upload=334781]: Begin upload segment... Mar 18 21:54:25.452374690 (298h54m23.049145713s) [seg_pos={0 0}, upload=334781]: Done begin upload segment. Mar 18 21:54:25.452463335 (298h54m23.049234366s) [noise=true , address=145.79.227.46:4604 , piece_id=DAGTJ24IRMED5FNGQHRYTXEIXYTDVSRW2IT23MC2UC45SES3PE7Q, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.452479705 (298h54m23.049250739s) [noise=true , address=45.91.116.9:4264 , piece_id=EE3TXU5KSPHKMNNCI3BHIGFKQNYZQRBDO4YT3TTR5U66YJMFT6XQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.453224160 (298h54m23.049995188s) [noise=true , address=79.127.219.46:4892 , piece_id=533AWUNWA66VX4AZF7Y3AANCF65GMESWZD3IJWBC5IOKB4QMVTZA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.454147610 (298h54m23.050918643s) [noise=true , address=145.79.227.48:4307 , piece_id=D6WSRVCL6UO4KQ542EKZWA6IP7FLA3ZUGSBXCSZUMLPFCJ5G5CQA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.454790080 (298h54m23.051561121s) [noise=true , address=145.79.227.2:4116 , piece_id=KEKK3M6M37G7Y7HXPULI2RSX5CFGUNCM3LILYNQOZ2PASM742ZGQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.455678920 (298h54m23.052449952s) [noise=true , address=145.79.227.19:4080 , piece_id=ANOMNO3HLII5GPWIT2RDX6P7WOBTVJXMOJM5PAPDMK7ST2NYJWMA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.456387102 (298h54m23.053158136s) [noise=true , address=145.79.227.37:4569 , piece_id=PTE4YT3YJ5M3NDKXKQOUFTGQ3T5FDAUZB74QHJBVH6HQL53QXLUA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.456987483 (298h54m23.053758515s) [noise=true , address=145.79.227.4:4190 , piece_id=LDID6UGFJ4GWYK635S5OE6334DEULOPVSOLOCLYQCBOFTXRZEKRQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.457980795 (298h54m23.054751829s) [noise=true , address=79.127.219.45:4855 , piece_id=644Z2MAXJ423ZF4JGZ2YLPM6XJRZT7IUJCZRRE7ZJMHB3EW7KB4Q, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.459806473 (298h54m23.056577503s) [noise=true , address=145.79.227.50:4376 , piece_id=BHABIBKA3ZENQUNU2OVBVUJZVWEQRZD435EHA3MEB25KMNBVFQIA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.460305248 (298h54m23.057076281s) [noise=true , address=77.90.31.16:4737 , piece_id=SMQOC6BKSIEIE5FXEYZGOIWRZ6GORC7GCOVEOFZXPQX4FGILVIPA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.461552822 (298h54m23.058323848s) [noise=true , address=79.127.219.41:4926 , piece_id=AVSCKJ5WDK4OBWWUSLUQNOPZI7BGTTDGT4XJYSLQAJPUHRYY4KVQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.463144575 (298h54m23.059915608s) [noise=true , address=145.79.227.36:4518 , piece_id=R5PDKAATCPFI3LOOUX6HJ4OP64NQ72N3RAOZS3GYFH5VSWPFU6OQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.464837828 (298h54m23.06160886s) [noise=true , address=145.79.227.26:4656 , piece_id=DDNFTYMUJMFHTZXFHJTFUURSVOJBQX2KVT6UWLBCZ5TIOVBDVOVQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.466411504 (298h54m23.063182537s) [noise=true , address=145.79.227.6:4261 , piece_id=CYOWHNR7X3Y4YE5SFHPNDBOU5QQ6M6DZIRAO3VX56KQBL3OP2FLQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.469091278 (298h54m23.065862311s) [noise=true , address=45.91.116.11:4118 , piece_id=CVKN5WPP2OTRAHYWJKDLCJV3OZ3I6G5CLYXTSEKJDQD7LRTJ55EA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.586976347 (298h54m23.183747361s) [upload=334781]: Next returned nil segment Mar 18 21:54:25.587000505 (298h54m23.183771534s) [upload=334781]: Waiting for error group managing segments... Mar 18 21:54:25.596319798 (298h54m23.193090825s) [noise=true , address=79.127.219.41:4926 , piece_id=AVSCKJ5WDK4OBWWUSLUQNOPZI7BGTTDGT4XJYSLQAJPUHRYY4KVQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.596985876 (298h54m23.1937569s) [noise=true , address=45.91.116.9:4264 , piece_id=EE3TXU5KSPHKMNNCI3BHIGFKQNYZQRBDO4YT3TTR5U66YJMFT6XQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.598026901 (298h54m23.194797934s) [noise=true , address=145.79.227.26:4656 , piece_id=DDNFTYMUJMFHTZXFHJTFUURSVOJBQX2KVT6UWLBCZ5TIOVBDVOVQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.598566162 (298h54m23.195337186s) [noise=true , address=145.79.227.50:4376 , piece_id=BHABIBKA3ZENQUNU2OVBVUJZVWEQRZD435EHA3MEB25KMNBVFQIA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.598781105 (298h54m23.19555213s) [noise=true , address=77.90.31.16:4737 , piece_id=SMQOC6BKSIEIE5FXEYZGOIWRZ6GORC7GCOVEOFZXPQX4FGILVIPA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.600260475 (298h54m23.197031509s) [noise=true , address=145.79.227.6:4261 , piece_id=CYOWHNR7X3Y4YE5SFHPNDBOU5QQ6M6DZIRAO3VX56KQBL3OP2FLQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.600344398 (298h54m23.197115411s) [noise=true , address=145.79.227.4:4190 , piece_id=LDID6UGFJ4GWYK635S5OE6334DEULOPVSOLOCLYQCBOFTXRZEKRQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.602467676 (298h54m23.19923871s) [noise=true , address=145.79.227.2:4116 , piece_id=KEKK3M6M37G7Y7HXPULI2RSX5CFGUNCM3LILYNQOZ2PASM742ZGQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.602977247 (298h54m23.199748272s) [noise=true , address=145.79.227.48:4307 , piece_id=D6WSRVCL6UO4KQ542EKZWA6IP7FLA3ZUGSBXCSZUMLPFCJ5G5CQA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.603520888 (298h54m23.200291919s) [noise=true , address=79.127.219.45:4855 , piece_id=644Z2MAXJ423ZF4JGZ2YLPM6XJRZT7IUJCZRRE7ZJMHB3EW7KB4Q, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.603907549 (298h54m23.200678573s) [noise=true , address=145.79.227.36:4518 , piece_id=R5PDKAATCPFI3LOOUX6HJ4OP64NQ72N3RAOZS3GYFH5VSWPFU6OQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.604438080 (298h54m23.201209113s) [noise=true , address=45.91.116.11:4118 , piece_id=CVKN5WPP2OTRAHYWJKDLCJV3OZ3I6G5CLYXTSEKJDQD7LRTJ55EA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.618924566 (298h54m23.21569559s) [noise=true , address=79.127.219.46:4892 , piece_id=533AWUNWA66VX4AZF7Y3AANCF65GMESWZD3IJWBC5IOKB4QMVTZA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.619077190 (298h54m23.215848214s) [seg_pos={0 0}, upload=334781]: Satisfied cohort requirements after 13 pieces Mar 18 21:54:25.619546929 (298h54m23.216317961s) [noise=true , address=145.79.227.46:4604 , piece_id=DAGTJ24IRMED5FNGQHRYTXEIXYTDVSRW2IT23MC2UC45SES3PE7Q, seg_pos={0 0}, upload=334781]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy): protocol: expected piece hash; context canceled; EOF Mar 18 21:54:25.619578476 (298h54m23.21634951s) [noise=true , address=145.79.227.46:4604 , piece_id=DAGTJ24IRMED5FNGQHRYTXEIXYTDVSRW2IT23MC2UC45SES3PE7Q, seg_pos={0 0}, upload=334781]: Long tail cancellation due to optimal threshold Mar 18 21:54:25.620166659 (298h54m23.216937744s) [noise=true , address=145.79.227.19:4080 , piece_id=ANOMNO3HLII5GPWIT2RDX6P7WOBTVJXMOJM5PAPDMK7ST2NYJWMA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KK51wudKiZb39Ku7wZhgBYEYmgsZTHfjiNNLmpBU9uuNFapt3): protocol: expected piece hash; context canceled; EOF Mar 18 21:54:25.620178720 (298h54m23.216949746s) [noise=true , address=145.79.227.19:4080 , piece_id=ANOMNO3HLII5GPWIT2RDX6P7WOBTVJXMOJM5PAPDMK7ST2NYJWMA, seg_pos={0 0}, upload=334781]: Long tail cancellation due to optimal threshold Mar 18 21:54:25.620702987 (298h54m23.217474021s) [noise=true , address=145.79.227.37:4569 , piece_id=PTE4YT3YJ5M3NDKXKQOUFTGQ3T5FDAUZB74QHJBVH6HQL53QXLUA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:18TqoejkuHLqZWhEPe3QPH1weVXnWJhaeoi8BGkvjCTLW6t4qT): protocol: expected piece hash; context canceled; EOF Mar 18 21:54:25.620718441 (298h54m23.217489474s) [noise=true , address=145.79.227.37:4569 , piece_id=PTE4YT3YJ5M3NDKXKQOUFTGQ3T5FDAUZB74QHJBVH6HQL53QXLUA, seg_pos={0 0}, upload=334781]: Long tail cancellation due to optimal threshold Mar 18 21:54:25.620748329 (298h54m23.21751937s) [seg_pos={0 0}, upload=334781]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Mar 18 21:54:25.620912612 (298h54m23.21768364s) [upload=334781]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 18 21:54:25.620920854 (298h54m23.217691883s) [upload=334781]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 21:54:38.136688455 (298h54m35.733459485s) [upload=334781]: Done uploading segments... Mar 18 21:54:38.136713048 (298h54m35.73348408s) [upload=334781]: Upload finished. err: <nil> Duration: 537.860148ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 10 out of 60 is failed (threshold=3) [...XXX..XX.X..X..X.XX.......................................] duration: 262.563981ms trace: https://jaeger.dev.storj.io/trace/7c4371cfa8f7834c FAIL (2026-03-18 22:24:47 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 262.588769ms core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 227.397336ms trace: https://jaeger.dev.storj.io/trace/2ab94940f3da3187 FAIL (2026-03-18 22:24:29 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 18 21:41:41.272084084 (298h41m38.868855112s) [upload=334559]: Starting upload Mar 18 21:41:41.272169036 (298h41m38.868940066s) [upload=334559]: Done starting upload Mar 18 21:41:41.272251155 (298h41m38.869022168s) [upload=334559]: Uploading segments... Mar 18 21:41:41.272358183 (298h41m38.869129211s) [upload=334559]: Got next segment. Inline: true Mar 18 21:41:41.272367512 (298h41m38.869138542s) [upload=334559]: Waiting for error group managing segments... Mar 18 21:41:41.272525860 (298h41m38.869296906s) [upload=334559]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 21:41:41.272550748 (298h41m38.869321776s) [upload=334559]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Mar 18 21:41:41.272555523 (298h41m38.86932655s) [upload=334559]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 21:42:30.253452076 (298h42m27.850223099s) [upload=334559]: Done uploading segments... Mar 18 21:42:30.253475139 (298h42m27.850246163s) [upload=334559]: Upload finished. err: <nil> Duration:* 227.440575ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 11 out of 60 is failed (threshold=3) [...X.XXXXXX.X..XXX..........................................] duration: 1.780347141s trace: https://jaeger.dev.storj.io/trace/27ed2876d5f3e85d FAIL (2026-03-18 22:24:10 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 1.780392568s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 11 out of 60 is failed (threshold=3) [X...XXX..XX.X..X..X.XX......................................] duration: 278.506972ms trace: https://jaeger.dev.storj.io/trace/7c4371cfa8f7834c FAIL (2026-03-18 22:23:56 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 278.539452ms core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [.XXXXXX.....XXXX.XXX........................................] duration: 533.0319ms trace: https://jaeger.dev.storj.io/trace/6dceff78c86f0774 FAIL (2026-03-18 22:23:53 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 18 21:54:17.190275520 (298h54m14.787046534s) [upload=334781]: Starting upload Mar 18 21:54:17.190367052 (298h54m14.787138068s) [upload=334781]: Done starting upload Mar 18 21:54:17.190448332 (298h54m14.787219348s) [upload=334781]: Uploading segments... Mar 18 21:54:17.190705184 (298h54m14.787476207s) [upload=334781]: Got next segment. Inline: false Mar 18 21:54:17.190783980 (298h54m14.787555002s) [upload=334781]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 21:54:17.190804859 (298h54m14.787575882s) [upload=334781]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 18 21:54:25.452304457 (298h54m23.049075487s) [seg_pos={0 0}, upload=334781]: Begin upload segment... Mar 18 21:54:25.452374690 (298h54m23.049145713s) [seg_pos={0 0}, upload=334781]: Done begin upload segment. Mar 18 21:54:25.452463335 (298h54m23.049234366s) [noise=true , address=145.79.227.46:4604 , piece_id=DAGTJ24IRMED5FNGQHRYTXEIXYTDVSRW2IT23MC2UC45SES3PE7Q, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.452479705 (298h54m23.049250739s) [noise=true , address=45.91.116.9:4264 , piece_id=EE3TXU5KSPHKMNNCI3BHIGFKQNYZQRBDO4YT3TTR5U66YJMFT6XQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.453224160 (298h54m23.049995188s) [noise=true , address=79.127.219.46:4892 , piece_id=533AWUNWA66VX4AZF7Y3AANCF65GMESWZD3IJWBC5IOKB4QMVTZA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.454147610 (298h54m23.050918643s) [noise=true , address=145.79.227.48:4307 , piece_id=D6WSRVCL6UO4KQ542EKZWA6IP7FLA3ZUGSBXCSZUMLPFCJ5G5CQA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.454790080 (298h54m23.051561121s) [noise=true , address=145.79.227.2:4116 , piece_id=KEKK3M6M37G7Y7HXPULI2RSX5CFGUNCM3LILYNQOZ2PASM742ZGQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.455678920 (298h54m23.052449952s) [noise=true , address=145.79.227.19:4080 , piece_id=ANOMNO3HLII5GPWIT2RDX6P7WOBTVJXMOJM5PAPDMK7ST2NYJWMA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.456387102 (298h54m23.053158136s) [noise=true , address=145.79.227.37:4569 , piece_id=PTE4YT3YJ5M3NDKXKQOUFTGQ3T5FDAUZB74QHJBVH6HQL53QXLUA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.456987483 (298h54m23.053758515s) [noise=true , address=145.79.227.4:4190 , piece_id=LDID6UGFJ4GWYK635S5OE6334DEULOPVSOLOCLYQCBOFTXRZEKRQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.457980795 (298h54m23.054751829s) [noise=true , address=79.127.219.45:4855 , piece_id=644Z2MAXJ423ZF4JGZ2YLPM6XJRZT7IUJCZRRE7ZJMHB3EW7KB4Q, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.459806473 (298h54m23.056577503s) [noise=true , address=145.79.227.50:4376 , piece_id=BHABIBKA3ZENQUNU2OVBVUJZVWEQRZD435EHA3MEB25KMNBVFQIA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.460305248 (298h54m23.057076281s) [noise=true , address=77.90.31.16:4737 , piece_id=SMQOC6BKSIEIE5FXEYZGOIWRZ6GORC7GCOVEOFZXPQX4FGILVIPA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.461552822 (298h54m23.058323848s) [noise=true , address=79.127.219.41:4926 , piece_id=AVSCKJ5WDK4OBWWUSLUQNOPZI7BGTTDGT4XJYSLQAJPUHRYY4KVQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.463144575 (298h54m23.059915608s) [noise=true , address=145.79.227.36:4518 , piece_id=R5PDKAATCPFI3LOOUX6HJ4OP64NQ72N3RAOZS3GYFH5VSWPFU6OQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.464837828 (298h54m23.06160886s) [noise=true , address=145.79.227.26:4656 , piece_id=DDNFTYMUJMFHTZXFHJTFUURSVOJBQX2KVT6UWLBCZ5TIOVBDVOVQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.466411504 (298h54m23.063182537s) [noise=true , address=145.79.227.6:4261 , piece_id=CYOWHNR7X3Y4YE5SFHPNDBOU5QQ6M6DZIRAO3VX56KQBL3OP2FLQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.469091278 (298h54m23.065862311s) [noise=true , address=45.91.116.11:4118 , piece_id=CVKN5WPP2OTRAHYWJKDLCJV3OZ3I6G5CLYXTSEKJDQD7LRTJ55EA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.586976347 (298h54m23.183747361s) [upload=334781]: Next returned nil segment Mar 18 21:54:25.587000505 (298h54m23.183771534s) [upload=334781]: Waiting for error group managing segments... Mar 18 21:54:25.596319798 (298h54m23.193090825s) [noise=true , address=79.127.219.41:4926 , piece_id=AVSCKJ5WDK4OBWWUSLUQNOPZI7BGTTDGT4XJYSLQAJPUHRYY4KVQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.596985876 (298h54m23.1937569s) [noise=true , address=45.91.116.9:4264 , piece_id=EE3TXU5KSPHKMNNCI3BHIGFKQNYZQRBDO4YT3TTR5U66YJMFT6XQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.598026901 (298h54m23.194797934s) [noise=true , address=145.79.227.26:4656 , piece_id=DDNFTYMUJMFHTZXFHJTFUURSVOJBQX2KVT6UWLBCZ5TIOVBDVOVQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.598566162 (298h54m23.195337186s) [noise=true , address=145.79.227.50:4376 , piece_id=BHABIBKA3ZENQUNU2OVBVUJZVWEQRZD435EHA3MEB25KMNBVFQIA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.598781105 (298h54m23.19555213s) [noise=true , address=77.90.31.16:4737 , piece_id=SMQOC6BKSIEIE5FXEYZGOIWRZ6GORC7GCOVEOFZXPQX4FGILVIPA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.600260475 (298h54m23.197031509s) [noise=true , address=145.79.227.6:4261 , piece_id=CYOWHNR7X3Y4YE5SFHPNDBOU5QQ6M6DZIRAO3VX56KQBL3OP2FLQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.600344398 (298h54m23.197115411s) [noise=true , address=145.79.227.4:4190 , piece_id=LDID6UGFJ4GWYK635S5OE6334DEULOPVSOLOCLYQCBOFTXRZEKRQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.602467676 (298h54m23.19923871s) [noise=true , address=145.79.227.2:4116 , piece_id=KEKK3M6M37G7Y7HXPULI2RSX5CFGUNCM3LILYNQOZ2PASM742ZGQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.602977247 (298h54m23.199748272s) [noise=true , address=145.79.227.48:4307 , piece_id=D6WSRVCL6UO4KQ542EKZWA6IP7FLA3ZUGSBXCSZUMLPFCJ5G5CQA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.603520888 (298h54m23.200291919s) [noise=true , address=79.127.219.45:4855 , piece_id=644Z2MAXJ423ZF4JGZ2YLPM6XJRZT7IUJCZRRE7ZJMHB3EW7KB4Q, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.603907549 (298h54m23.200678573s) [noise=true , address=145.79.227.36:4518 , piece_id=R5PDKAATCPFI3LOOUX6HJ4OP64NQ72N3RAOZS3GYFH5VSWPFU6OQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.604438080 (298h54m23.201209113s) [noise=true , address=45.91.116.11:4118 , piece_id=CVKN5WPP2OTRAHYWJKDLCJV3OZ3I6G5CLYXTSEKJDQD7LRTJ55EA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.618924566 (298h54m23.21569559s) [noise=true , address=79.127.219.46:4892 , piece_id=533AWUNWA66VX4AZF7Y3AANCF65GMESWZD3IJWBC5IOKB4QMVTZA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.619077190 (298h54m23.215848214s) [seg_pos={0 0}, upload=334781]: Satisfied cohort requirements after 13 pieces Mar 18 21:54:25.619546929 (298h54m23.216317961s) [noise=true , address=145.79.227.46:4604 , piece_id=DAGTJ24IRMED5FNGQHRYTXEIXYTDVSRW2IT23MC2UC45SES3PE7Q, seg_pos={0 0}, upload=334781]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy): protocol: expected piece hash; context canceled; EOF Mar 18 21:54:25.619578476 (298h54m23.21634951s) [noise=true , address=145.79.227.46:4604 , piece_id=DAGTJ24IRMED5FNGQHRYTXEIXYTDVSRW2IT23MC2UC45SES3PE7Q, seg_pos={0 0}, upload=334781]: Long tail cancellation due to optimal threshold Mar 18 21:54:25.620166659 (298h54m23.216937744s) [noise=true , address=145.79.227.19:4080 , piece_id=ANOMNO3HLII5GPWIT2RDX6P7WOBTVJXMOJM5PAPDMK7ST2NYJWMA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KK51wudKiZb39Ku7wZhgBYEYmgsZTHfjiNNLmpBU9uuNFapt3): protocol: expected piece hash; context canceled; EOF Mar 18 21:54:25.620178720 (298h54m23.216949746s) [noise=true , address=145.79.227.19:4080 , piece_id=ANOMNO3HLII5GPWIT2RDX6P7WOBTVJXMOJM5PAPDMK7ST2NYJWMA, seg_pos={0 0}, upload=334781]: Long tail cancellation due to optimal threshold Mar 18 21:54:25.620702987 (298h54m23.217474021s) [noise=true , address=145.79.227.37:4569 , piece_id=PTE4YT3YJ5M3NDKXKQOUFTGQ3T5FDAUZB74QHJBVH6HQL53QXLUA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:18TqoejkuHLqZWhEPe3QPH1weVXnWJhaeoi8BGkvjCTLW6t4qT): protocol: expected piece hash; context canceled; EOF Mar 18 21:54:25.620718441 (298h54m23.217489474s) [noise=true , address=145.79.227.37:4569 , piece_id=PTE4YT3YJ5M3NDKXKQOUFTGQ3T5FDAUZB74QHJBVH6HQL53QXLUA, seg_pos={0 0}, upload=334781]: Long tail cancellation due to optimal threshold Mar 18 21:54:25.620748329 (298h54m23.21751937s) [seg_pos={0 0}, upload=334781]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Mar 18 21:54:25.620912612 (298h54m23.21768364s) [upload=334781]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 18 21:54:25.620920854 (298h54m23.217691883s) [upload=334781]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 21:54:38.136688455 (298h54m35.733459485s) [upload=334781]: Done uploading segments... Mar 18 21:54:38.136713048 (298h54m35.73348408s) [upload=334781]: Upload finished. err: <nil> Duration: 533.051335ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 20 out of 60 is failed (threshold=6) [..XXXXXXXXXXXXXXXXXXXX......................................] duration: 602.239514ms trace: https://jaeger.dev.storj.io/trace/6cabdd7d47c9cb44 FAIL (2026-03-18 22:23:50 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 602.27021ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 11 out of 60 is failed (threshold=3) [....XXXXXX.X...XX.X.X.......................................] duration: 260.483511ms trace: https://jaeger.dev.storj.io/trace/47004c0815b844e7 FAIL (2026-03-18 22:23:48 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 18 21:53:36.090826124 (298h53m33.687597155s) [upload=334767]: Starting upload Mar 18 21:53:36.090977797 (298h53m33.687748822s) [upload=334767]: Done starting upload Mar 18 21:53:36.091078093 (298h53m33.687849119s) [upload=334767]: Uploading segments... Mar 18 21:53:36.091268776 (298h53m33.688040083s) [upload=334767]: Got next segment. Inline: true Mar 18 21:53:36.091279074 (298h53m33.688050103s) [upload=334767]: Waiting for error group managing segments... Mar 18 21:53:36.091372978 (298h53m33.688144001s) [upload=334767]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 21:53:36.091379985 (298h53m33.688151008s) [upload=334767]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Mar 18 21:53:36.091384265 (298h53m33.688155287s) [upload=334767]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 21:54:04.718146694 (298h54m2.314917717s) [upload=334767]: Done uploading segments... Mar 18 21:54:04.718169125 (298h54m2.314940155s) [upload=334767]: Upload finished. err: <nil> Duration:* 260.51184ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 11 out of 60 is failed (threshold=3) [.....XXXXXX.X...XX.X.X......................................] duration: 276.876196ms trace: https://jaeger.dev.storj.io/trace/47004c0815b844e7 FAIL (2026-03-18 22:23:24 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 18 21:53:36.090826124 (298h53m33.687597155s) [upload=334767]: Starting upload Mar 18 21:53:36.090977797 (298h53m33.687748822s) [upload=334767]: Done starting upload Mar 18 21:53:36.091078093 (298h53m33.687849119s) [upload=334767]: Uploading segments... Mar 18 21:53:36.091268776 (298h53m33.688040083s) [upload=334767]: Got next segment. Inline: true Mar 18 21:53:36.091279074 (298h53m33.688050103s) [upload=334767]: Waiting for error group managing segments... Mar 18 21:53:36.091372978 (298h53m33.688144001s) [upload=334767]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 21:53:36.091379985 (298h53m33.688151008s) [upload=334767]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Mar 18 21:53:36.091384265 (298h53m33.688155287s) [upload=334767]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 21:54:04.718146694 (298h54m2.314917717s) [upload=334767]: Done uploading segments... Mar 18 21:54:04.718169125 (298h54m2.314940155s) [upload=334767]: Upload finished. err: <nil> Duration:* 276.901884ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 197.318735ms trace: https://jaeger.dev.storj.io/trace/2ab94940f3da3187 FAIL (2026-03-18 22:23:24 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 18 21:41:41.272084084 (298h41m38.868855112s) [upload=334559]: Starting upload Mar 18 21:41:41.272169036 (298h41m38.868940066s) [upload=334559]: Done starting upload Mar 18 21:41:41.272251155 (298h41m38.869022168s) [upload=334559]: Uploading segments... Mar 18 21:41:41.272358183 (298h41m38.869129211s) [upload=334559]: Got next segment. Inline: true Mar 18 21:41:41.272367512 (298h41m38.869138542s) [upload=334559]: Waiting for error group managing segments... Mar 18 21:41:41.272525860 (298h41m38.869296906s) [upload=334559]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 21:41:41.272550748 (298h41m38.869321776s) [upload=334559]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Mar 18 21:41:41.272555523 (298h41m38.86932655s) [upload=334559]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 21:42:30.253452076 (298h42m27.850223099s) [upload=334559]: Done uploading segments... Mar 18 21:42:30.253475139 (298h42m27.850246163s) [upload=334559]: Upload finished. err: <nil> Duration:* 197.352579ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 11 out of 60 is failed (threshold=3) [....X.XXXXXX.X..XXX.........................................] duration: 1.802589929s trace: https://jaeger.dev.storj.io/trace/27ed2876d5f3e85d FAIL (2026-03-18 22:23:11 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 1.802623249s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [..XXXXXX.....XXXX.XXX.......................................] duration: 655.989999ms trace: https://jaeger.dev.storj.io/trace/6dceff78c86f0774 FAIL (2026-03-18 22:23:01 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 18 21:54:17.190275520 (298h54m14.787046534s) [upload=334781]: Starting upload Mar 18 21:54:17.190367052 (298h54m14.787138068s) [upload=334781]: Done starting upload Mar 18 21:54:17.190448332 (298h54m14.787219348s) [upload=334781]: Uploading segments... Mar 18 21:54:17.190705184 (298h54m14.787476207s) [upload=334781]: Got next segment. Inline: false Mar 18 21:54:17.190783980 (298h54m14.787555002s) [upload=334781]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 21:54:17.190804859 (298h54m14.787575882s) [upload=334781]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 18 21:54:25.452304457 (298h54m23.049075487s) [seg_pos={0 0}, upload=334781]: Begin upload segment... Mar 18 21:54:25.452374690 (298h54m23.049145713s) [seg_pos={0 0}, upload=334781]: Done begin upload segment. Mar 18 21:54:25.452463335 (298h54m23.049234366s) [noise=true , address=145.79.227.46:4604 , piece_id=DAGTJ24IRMED5FNGQHRYTXEIXYTDVSRW2IT23MC2UC45SES3PE7Q, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.452479705 (298h54m23.049250739s) [noise=true , address=45.91.116.9:4264 , piece_id=EE3TXU5KSPHKMNNCI3BHIGFKQNYZQRBDO4YT3TTR5U66YJMFT6XQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.453224160 (298h54m23.049995188s) [noise=true , address=79.127.219.46:4892 , piece_id=533AWUNWA66VX4AZF7Y3AANCF65GMESWZD3IJWBC5IOKB4QMVTZA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.454147610 (298h54m23.050918643s) [noise=true , address=145.79.227.48:4307 , piece_id=D6WSRVCL6UO4KQ542EKZWA6IP7FLA3ZUGSBXCSZUMLPFCJ5G5CQA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.454790080 (298h54m23.051561121s) [noise=true , address=145.79.227.2:4116 , piece_id=KEKK3M6M37G7Y7HXPULI2RSX5CFGUNCM3LILYNQOZ2PASM742ZGQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.455678920 (298h54m23.052449952s) [noise=true , address=145.79.227.19:4080 , piece_id=ANOMNO3HLII5GPWIT2RDX6P7WOBTVJXMOJM5PAPDMK7ST2NYJWMA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.456387102 (298h54m23.053158136s) [noise=true , address=145.79.227.37:4569 , piece_id=PTE4YT3YJ5M3NDKXKQOUFTGQ3T5FDAUZB74QHJBVH6HQL53QXLUA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.456987483 (298h54m23.053758515s) [noise=true , address=145.79.227.4:4190 , piece_id=LDID6UGFJ4GWYK635S5OE6334DEULOPVSOLOCLYQCBOFTXRZEKRQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.457980795 (298h54m23.054751829s) [noise=true , address=79.127.219.45:4855 , piece_id=644Z2MAXJ423ZF4JGZ2YLPM6XJRZT7IUJCZRRE7ZJMHB3EW7KB4Q, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.459806473 (298h54m23.056577503s) [noise=true , address=145.79.227.50:4376 , piece_id=BHABIBKA3ZENQUNU2OVBVUJZVWEQRZD435EHA3MEB25KMNBVFQIA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.460305248 (298h54m23.057076281s) [noise=true , address=77.90.31.16:4737 , piece_id=SMQOC6BKSIEIE5FXEYZGOIWRZ6GORC7GCOVEOFZXPQX4FGILVIPA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.461552822 (298h54m23.058323848s) [noise=true , address=79.127.219.41:4926 , piece_id=AVSCKJ5WDK4OBWWUSLUQNOPZI7BGTTDGT4XJYSLQAJPUHRYY4KVQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.463144575 (298h54m23.059915608s) [noise=true , address=145.79.227.36:4518 , piece_id=R5PDKAATCPFI3LOOUX6HJ4OP64NQ72N3RAOZS3GYFH5VSWPFU6OQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.464837828 (298h54m23.06160886s) [noise=true , address=145.79.227.26:4656 , piece_id=DDNFTYMUJMFHTZXFHJTFUURSVOJBQX2KVT6UWLBCZ5TIOVBDVOVQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.466411504 (298h54m23.063182537s) [noise=true , address=145.79.227.6:4261 , piece_id=CYOWHNR7X3Y4YE5SFHPNDBOU5QQ6M6DZIRAO3VX56KQBL3OP2FLQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.469091278 (298h54m23.065862311s) [noise=true , address=45.91.116.11:4118 , piece_id=CVKN5WPP2OTRAHYWJKDLCJV3OZ3I6G5CLYXTSEKJDQD7LRTJ55EA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.586976347 (298h54m23.183747361s) [upload=334781]: Next returned nil segment Mar 18 21:54:25.587000505 (298h54m23.183771534s) [upload=334781]: Waiting for error group managing segments... Mar 18 21:54:25.596319798 (298h54m23.193090825s) [noise=true , address=79.127.219.41:4926 , piece_id=AVSCKJ5WDK4OBWWUSLUQNOPZI7BGTTDGT4XJYSLQAJPUHRYY4KVQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.596985876 (298h54m23.1937569s) [noise=true , address=45.91.116.9:4264 , piece_id=EE3TXU5KSPHKMNNCI3BHIGFKQNYZQRBDO4YT3TTR5U66YJMFT6XQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.598026901 (298h54m23.194797934s) [noise=true , address=145.79.227.26:4656 , piece_id=DDNFTYMUJMFHTZXFHJTFUURSVOJBQX2KVT6UWLBCZ5TIOVBDVOVQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.598566162 (298h54m23.195337186s) [noise=true , address=145.79.227.50:4376 , piece_id=BHABIBKA3ZENQUNU2OVBVUJZVWEQRZD435EHA3MEB25KMNBVFQIA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.598781105 (298h54m23.19555213s) [noise=true , address=77.90.31.16:4737 , piece_id=SMQOC6BKSIEIE5FXEYZGOIWRZ6GORC7GCOVEOFZXPQX4FGILVIPA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.600260475 (298h54m23.197031509s) [noise=true , address=145.79.227.6:4261 , piece_id=CYOWHNR7X3Y4YE5SFHPNDBOU5QQ6M6DZIRAO3VX56KQBL3OP2FLQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.600344398 (298h54m23.197115411s) [noise=true , address=145.79.227.4:4190 , piece_id=LDID6UGFJ4GWYK635S5OE6334DEULOPVSOLOCLYQCBOFTXRZEKRQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.602467676 (298h54m23.19923871s) [noise=true , address=145.79.227.2:4116 , piece_id=KEKK3M6M37G7Y7HXPULI2RSX5CFGUNCM3LILYNQOZ2PASM742ZGQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.602977247 (298h54m23.199748272s) [noise=true , address=145.79.227.48:4307 , piece_id=D6WSRVCL6UO4KQ542EKZWA6IP7FLA3ZUGSBXCSZUMLPFCJ5G5CQA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.603520888 (298h54m23.200291919s) [noise=true , address=79.127.219.45:4855 , piece_id=644Z2MAXJ423ZF4JGZ2YLPM6XJRZT7IUJCZRRE7ZJMHB3EW7KB4Q, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.603907549 (298h54m23.200678573s) [noise=true , address=145.79.227.36:4518 , piece_id=R5PDKAATCPFI3LOOUX6HJ4OP64NQ72N3RAOZS3GYFH5VSWPFU6OQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.604438080 (298h54m23.201209113s) [noise=true , address=45.91.116.11:4118 , piece_id=CVKN5WPP2OTRAHYWJKDLCJV3OZ3I6G5CLYXTSEKJDQD7LRTJ55EA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.618924566 (298h54m23.21569559s) [noise=true , address=79.127.219.46:4892 , piece_id=533AWUNWA66VX4AZF7Y3AANCF65GMESWZD3IJWBC5IOKB4QMVTZA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.619077190 (298h54m23.215848214s) [seg_pos={0 0}, upload=334781]: Satisfied cohort requirements after 13 pieces Mar 18 21:54:25.619546929 (298h54m23.216317961s) [noise=true , address=145.79.227.46:4604 , piece_id=DAGTJ24IRMED5FNGQHRYTXEIXYTDVSRW2IT23MC2UC45SES3PE7Q, seg_pos={0 0}, upload=334781]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy): protocol: expected piece hash; context canceled; EOF Mar 18 21:54:25.619578476 (298h54m23.21634951s) [noise=true , address=145.79.227.46:4604 , piece_id=DAGTJ24IRMED5FNGQHRYTXEIXYTDVSRW2IT23MC2UC45SES3PE7Q, seg_pos={0 0}, upload=334781]: Long tail cancellation due to optimal threshold Mar 18 21:54:25.620166659 (298h54m23.216937744s) [noise=true , address=145.79.227.19:4080 , piece_id=ANOMNO3HLII5GPWIT2RDX6P7WOBTVJXMOJM5PAPDMK7ST2NYJWMA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KK51wudKiZb39Ku7wZhgBYEYmgsZTHfjiNNLmpBU9uuNFapt3): protocol: expected piece hash; context canceled; EOF Mar 18 21:54:25.620178720 (298h54m23.216949746s) [noise=true , address=145.79.227.19:4080 , piece_id=ANOMNO3HLII5GPWIT2RDX6P7WOBTVJXMOJM5PAPDMK7ST2NYJWMA, seg_pos={0 0}, upload=334781]: Long tail cancellation due to optimal threshold Mar 18 21:54:25.620702987 (298h54m23.217474021s) [noise=true , address=145.79.227.37:4569 , piece_id=PTE4YT3YJ5M3NDKXKQOUFTGQ3T5FDAUZB74QHJBVH6HQL53QXLUA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:18TqoejkuHLqZWhEPe3QPH1weVXnWJhaeoi8BGkvjCTLW6t4qT): protocol: expected piece hash; context canceled; EOF Mar 18 21:54:25.620718441 (298h54m23.217489474s) [noise=true , address=145.79.227.37:4569 , piece_id=PTE4YT3YJ5M3NDKXKQOUFTGQ3T5FDAUZB74QHJBVH6HQL53QXLUA, seg_pos={0 0}, upload=334781]: Long tail cancellation due to optimal threshold Mar 18 21:54:25.620748329 (298h54m23.21751937s) [seg_pos={0 0}, upload=334781]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Mar 18 21:54:25.620912612 (298h54m23.21768364s) [upload=334781]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 18 21:54:25.620920854 (298h54m23.217691883s) [upload=334781]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 21:54:38.136688455 (298h54m35.733459485s) [upload=334781]: Done uploading segments... Mar 18 21:54:38.136713048 (298h54m35.73348408s) [upload=334781]: Upload finished. err: <nil> Duration: 656.034296ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 11 out of 60 is failed (threshold=3) [.X...XXX..XX.X..X..X.XX.....................................] duration: 242.740298ms trace: https://jaeger.dev.storj.io/trace/7c4371cfa8f7834c FAIL (2026-03-18 22:22:54 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 242.77349ms core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 11 out of 60 is failed (threshold=3) [.....X.XXXXXX.X..XXX........................................] duration: 1.781767382s trace: https://jaeger.dev.storj.io/trace/27ed2876d5f3e85d FAIL (2026-03-18 22:22:49 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 1.781840188s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 262.3232ms trace: https://jaeger.dev.storj.io/trace/2ab94940f3da3187 FAIL (2026-03-18 22:22:49 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 18 21:41:41.272084084 (298h41m38.868855112s) [upload=334559]: Starting upload Mar 18 21:41:41.272169036 (298h41m38.868940066s) [upload=334559]: Done starting upload Mar 18 21:41:41.272251155 (298h41m38.869022168s) [upload=334559]: Uploading segments... Mar 18 21:41:41.272358183 (298h41m38.869129211s) [upload=334559]: Got next segment. Inline: true Mar 18 21:41:41.272367512 (298h41m38.869138542s) [upload=334559]: Waiting for error group managing segments... Mar 18 21:41:41.272525860 (298h41m38.869296906s) [upload=334559]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 21:41:41.272550748 (298h41m38.869321776s) [upload=334559]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Mar 18 21:41:41.272555523 (298h41m38.86932655s) [upload=334559]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 21:42:30.253452076 (298h42m27.850223099s) [upload=334559]: Done uploading segments... Mar 18 21:42:30.253475139 (298h42m27.850246163s) [upload=334559]: Upload finished. err: <nil> Duration:* 262.355962ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 21 out of 60 is failed (threshold=6) [X..XXXXXXXXXXXXXXXXXXXX.....................................] duration: 731.309317ms trace: https://jaeger.dev.storj.io/trace/6cabdd7d47c9cb44 FAIL (2026-03-18 22:22:47 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 731.348482ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 11 out of 60 is failed (threshold=3) [..X...XXX..XX.X..X..X.XX....................................] duration: 288.101528ms trace: https://jaeger.dev.storj.io/trace/7c4371cfa8f7834c FAIL (2026-03-18 22:22:31 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 288.12847ms core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 11 out of 60 is failed (threshold=3) [......XXXXXX.X...XX.X.X.....................................] duration: 244.654958ms trace: https://jaeger.dev.storj.io/trace/47004c0815b844e7 FAIL (2026-03-18 22:22:22 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 18 21:53:36.090826124 (298h53m33.687597155s) [upload=334767]: Starting upload Mar 18 21:53:36.090977797 (298h53m33.687748822s) [upload=334767]: Done starting upload Mar 18 21:53:36.091078093 (298h53m33.687849119s) [upload=334767]: Uploading segments... Mar 18 21:53:36.091268776 (298h53m33.688040083s) [upload=334767]: Got next segment. Inline: true Mar 18 21:53:36.091279074 (298h53m33.688050103s) [upload=334767]: Waiting for error group managing segments... Mar 18 21:53:36.091372978 (298h53m33.688144001s) [upload=334767]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 21:53:36.091379985 (298h53m33.688151008s) [upload=334767]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Mar 18 21:53:36.091384265 (298h53m33.688155287s) [upload=334767]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 21:54:04.718146694 (298h54m2.314917717s) [upload=334767]: Done uploading segments... Mar 18 21:54:04.718169125 (298h54m2.314940155s) [upload=334767]: Upload finished. err: <nil> Duration:* 244.688358ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [...XXXXXX.....XXXX.XXX......................................] duration: 449.024761ms trace: https://jaeger.dev.storj.io/trace/6dceff78c86f0774 FAIL (2026-03-18 22:22:19 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 18 21:54:17.190275520 (298h54m14.787046534s) [upload=334781]: Starting upload Mar 18 21:54:17.190367052 (298h54m14.787138068s) [upload=334781]: Done starting upload Mar 18 21:54:17.190448332 (298h54m14.787219348s) [upload=334781]: Uploading segments... Mar 18 21:54:17.190705184 (298h54m14.787476207s) [upload=334781]: Got next segment. Inline: false Mar 18 21:54:17.190783980 (298h54m14.787555002s) [upload=334781]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 21:54:17.190804859 (298h54m14.787575882s) [upload=334781]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 18 21:54:25.452304457 (298h54m23.049075487s) [seg_pos={0 0}, upload=334781]: Begin upload segment... Mar 18 21:54:25.452374690 (298h54m23.049145713s) [seg_pos={0 0}, upload=334781]: Done begin upload segment. Mar 18 21:54:25.452463335 (298h54m23.049234366s) [noise=true , address=145.79.227.46:4604 , piece_id=DAGTJ24IRMED5FNGQHRYTXEIXYTDVSRW2IT23MC2UC45SES3PE7Q, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.452479705 (298h54m23.049250739s) [noise=true , address=45.91.116.9:4264 , piece_id=EE3TXU5KSPHKMNNCI3BHIGFKQNYZQRBDO4YT3TTR5U66YJMFT6XQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.453224160 (298h54m23.049995188s) [noise=true , address=79.127.219.46:4892 , piece_id=533AWUNWA66VX4AZF7Y3AANCF65GMESWZD3IJWBC5IOKB4QMVTZA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.454147610 (298h54m23.050918643s) [noise=true , address=145.79.227.48:4307 , piece_id=D6WSRVCL6UO4KQ542EKZWA6IP7FLA3ZUGSBXCSZUMLPFCJ5G5CQA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.454790080 (298h54m23.051561121s) [noise=true , address=145.79.227.2:4116 , piece_id=KEKK3M6M37G7Y7HXPULI2RSX5CFGUNCM3LILYNQOZ2PASM742ZGQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.455678920 (298h54m23.052449952s) [noise=true , address=145.79.227.19:4080 , piece_id=ANOMNO3HLII5GPWIT2RDX6P7WOBTVJXMOJM5PAPDMK7ST2NYJWMA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.456387102 (298h54m23.053158136s) [noise=true , address=145.79.227.37:4569 , piece_id=PTE4YT3YJ5M3NDKXKQOUFTGQ3T5FDAUZB74QHJBVH6HQL53QXLUA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.456987483 (298h54m23.053758515s) [noise=true , address=145.79.227.4:4190 , piece_id=LDID6UGFJ4GWYK635S5OE6334DEULOPVSOLOCLYQCBOFTXRZEKRQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.457980795 (298h54m23.054751829s) [noise=true , address=79.127.219.45:4855 , piece_id=644Z2MAXJ423ZF4JGZ2YLPM6XJRZT7IUJCZRRE7ZJMHB3EW7KB4Q, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.459806473 (298h54m23.056577503s) [noise=true , address=145.79.227.50:4376 , piece_id=BHABIBKA3ZENQUNU2OVBVUJZVWEQRZD435EHA3MEB25KMNBVFQIA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.460305248 (298h54m23.057076281s) [noise=true , address=77.90.31.16:4737 , piece_id=SMQOC6BKSIEIE5FXEYZGOIWRZ6GORC7GCOVEOFZXPQX4FGILVIPA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.461552822 (298h54m23.058323848s) [noise=true , address=79.127.219.41:4926 , piece_id=AVSCKJ5WDK4OBWWUSLUQNOPZI7BGTTDGT4XJYSLQAJPUHRYY4KVQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.463144575 (298h54m23.059915608s) [noise=true , address=145.79.227.36:4518 , piece_id=R5PDKAATCPFI3LOOUX6HJ4OP64NQ72N3RAOZS3GYFH5VSWPFU6OQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.464837828 (298h54m23.06160886s) [noise=true , address=145.79.227.26:4656 , piece_id=DDNFTYMUJMFHTZXFHJTFUURSVOJBQX2KVT6UWLBCZ5TIOVBDVOVQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.466411504 (298h54m23.063182537s) [noise=true , address=145.79.227.6:4261 , piece_id=CYOWHNR7X3Y4YE5SFHPNDBOU5QQ6M6DZIRAO3VX56KQBL3OP2FLQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.469091278 (298h54m23.065862311s) [noise=true , address=45.91.116.11:4118 , piece_id=CVKN5WPP2OTRAHYWJKDLCJV3OZ3I6G5CLYXTSEKJDQD7LRTJ55EA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.586976347 (298h54m23.183747361s) [upload=334781]: Next returned nil segment Mar 18 21:54:25.587000505 (298h54m23.183771534s) [upload=334781]: Waiting for error group managing segments... Mar 18 21:54:25.596319798 (298h54m23.193090825s) [noise=true , address=79.127.219.41:4926 , piece_id=AVSCKJ5WDK4OBWWUSLUQNOPZI7BGTTDGT4XJYSLQAJPUHRYY4KVQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.596985876 (298h54m23.1937569s) [noise=true , address=45.91.116.9:4264 , piece_id=EE3TXU5KSPHKMNNCI3BHIGFKQNYZQRBDO4YT3TTR5U66YJMFT6XQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.598026901 (298h54m23.194797934s) [noise=true , address=145.79.227.26:4656 , piece_id=DDNFTYMUJMFHTZXFHJTFUURSVOJBQX2KVT6UWLBCZ5TIOVBDVOVQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.598566162 (298h54m23.195337186s) [noise=true , address=145.79.227.50:4376 , piece_id=BHABIBKA3ZENQUNU2OVBVUJZVWEQRZD435EHA3MEB25KMNBVFQIA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.598781105 (298h54m23.19555213s) [noise=true , address=77.90.31.16:4737 , piece_id=SMQOC6BKSIEIE5FXEYZGOIWRZ6GORC7GCOVEOFZXPQX4FGILVIPA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.600260475 (298h54m23.197031509s) [noise=true , address=145.79.227.6:4261 , piece_id=CYOWHNR7X3Y4YE5SFHPNDBOU5QQ6M6DZIRAO3VX56KQBL3OP2FLQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.600344398 (298h54m23.197115411s) [noise=true , address=145.79.227.4:4190 , piece_id=LDID6UGFJ4GWYK635S5OE6334DEULOPVSOLOCLYQCBOFTXRZEKRQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.602467676 (298h54m23.19923871s) [noise=true , address=145.79.227.2:4116 , piece_id=KEKK3M6M37G7Y7HXPULI2RSX5CFGUNCM3LILYNQOZ2PASM742ZGQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.602977247 (298h54m23.199748272s) [noise=true , address=145.79.227.48:4307 , piece_id=D6WSRVCL6UO4KQ542EKZWA6IP7FLA3ZUGSBXCSZUMLPFCJ5G5CQA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.603520888 (298h54m23.200291919s) [noise=true , address=79.127.219.45:4855 , piece_id=644Z2MAXJ423ZF4JGZ2YLPM6XJRZT7IUJCZRRE7ZJMHB3EW7KB4Q, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.603907549 (298h54m23.200678573s) [noise=true , address=145.79.227.36:4518 , piece_id=R5PDKAATCPFI3LOOUX6HJ4OP64NQ72N3RAOZS3GYFH5VSWPFU6OQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.604438080 (298h54m23.201209113s) [noise=true , address=45.91.116.11:4118 , piece_id=CVKN5WPP2OTRAHYWJKDLCJV3OZ3I6G5CLYXTSEKJDQD7LRTJ55EA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.618924566 (298h54m23.21569559s) [noise=true , address=79.127.219.46:4892 , piece_id=533AWUNWA66VX4AZF7Y3AANCF65GMESWZD3IJWBC5IOKB4QMVTZA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.619077190 (298h54m23.215848214s) [seg_pos={0 0}, upload=334781]: Satisfied cohort requirements after 13 pieces Mar 18 21:54:25.619546929 (298h54m23.216317961s) [noise=true , address=145.79.227.46:4604 , piece_id=DAGTJ24IRMED5FNGQHRYTXEIXYTDVSRW2IT23MC2UC45SES3PE7Q, seg_pos={0 0}, upload=334781]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy): protocol: expected piece hash; context canceled; EOF Mar 18 21:54:25.619578476 (298h54m23.21634951s) [noise=true , address=145.79.227.46:4604 , piece_id=DAGTJ24IRMED5FNGQHRYTXEIXYTDVSRW2IT23MC2UC45SES3PE7Q, seg_pos={0 0}, upload=334781]: Long tail cancellation due to optimal threshold Mar 18 21:54:25.620166659 (298h54m23.216937744s) [noise=true , address=145.79.227.19:4080 , piece_id=ANOMNO3HLII5GPWIT2RDX6P7WOBTVJXMOJM5PAPDMK7ST2NYJWMA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KK51wudKiZb39Ku7wZhgBYEYmgsZTHfjiNNLmpBU9uuNFapt3): protocol: expected piece hash; context canceled; EOF Mar 18 21:54:25.620178720 (298h54m23.216949746s) [noise=true , address=145.79.227.19:4080 , piece_id=ANOMNO3HLII5GPWIT2RDX6P7WOBTVJXMOJM5PAPDMK7ST2NYJWMA, seg_pos={0 0}, upload=334781]: Long tail cancellation due to optimal threshold Mar 18 21:54:25.620702987 (298h54m23.217474021s) [noise=true , address=145.79.227.37:4569 , piece_id=PTE4YT3YJ5M3NDKXKQOUFTGQ3T5FDAUZB74QHJBVH6HQL53QXLUA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:18TqoejkuHLqZWhEPe3QPH1weVXnWJhaeoi8BGkvjCTLW6t4qT): protocol: expected piece hash; context canceled; EOF Mar 18 21:54:25.620718441 (298h54m23.217489474s) [noise=true , address=145.79.227.37:4569 , piece_id=PTE4YT3YJ5M3NDKXKQOUFTGQ3T5FDAUZB74QHJBVH6HQL53QXLUA, seg_pos={0 0}, upload=334781]: Long tail cancellation due to optimal threshold Mar 18 21:54:25.620748329 (298h54m23.21751937s) [seg_pos={0 0}, upload=334781]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Mar 18 21:54:25.620912612 (298h54m23.21768364s) [upload=334781]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 18 21:54:25.620920854 (298h54m23.217691883s) [upload=334781]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 21:54:38.136688455 (298h54m35.733459485s) [upload=334781]: Done uploading segments... Mar 18 21:54:38.136713048 (298h54m35.73348408s) [upload=334781]: Upload finished. err: <nil> Duration: 449.051624ms core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)