Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [XXX.XXX.....................................................] duration: 507.659892ms trace: https://jaeger.dev.storj.io/trace/6dceff78c86f0774 FAIL (2026-03-18 22:33:58 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 18 21:54:17.190275520 (298h54m14.787046534s) [upload=334781]: Starting upload Mar 18 21:54:17.190367052 (298h54m14.787138068s) [upload=334781]: Done starting upload Mar 18 21:54:17.190448332 (298h54m14.787219348s) [upload=334781]: Uploading segments... Mar 18 21:54:17.190705184 (298h54m14.787476207s) [upload=334781]: Got next segment. Inline: false Mar 18 21:54:17.190783980 (298h54m14.787555002s) [upload=334781]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 21:54:17.190804859 (298h54m14.787575882s) [upload=334781]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 18 21:54:25.452304457 (298h54m23.049075487s) [seg_pos={0 0}, upload=334781]: Begin upload segment... Mar 18 21:54:25.452374690 (298h54m23.049145713s) [seg_pos={0 0}, upload=334781]: Done begin upload segment. Mar 18 21:54:25.452463335 (298h54m23.049234366s) [noise=true , address=145.79.227.46:4604 , piece_id=DAGTJ24IRMED5FNGQHRYTXEIXYTDVSRW2IT23MC2UC45SES3PE7Q, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.452479705 (298h54m23.049250739s) [noise=true , address=45.91.116.9:4264 , piece_id=EE3TXU5KSPHKMNNCI3BHIGFKQNYZQRBDO4YT3TTR5U66YJMFT6XQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.453224160 (298h54m23.049995188s) [noise=true , address=79.127.219.46:4892 , piece_id=533AWUNWA66VX4AZF7Y3AANCF65GMESWZD3IJWBC5IOKB4QMVTZA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.454147610 (298h54m23.050918643s) [noise=true , address=145.79.227.48:4307 , piece_id=D6WSRVCL6UO4KQ542EKZWA6IP7FLA3ZUGSBXCSZUMLPFCJ5G5CQA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.454790080 (298h54m23.051561121s) [noise=true , address=145.79.227.2:4116 , piece_id=KEKK3M6M37G7Y7HXPULI2RSX5CFGUNCM3LILYNQOZ2PASM742ZGQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.455678920 (298h54m23.052449952s) [noise=true , address=145.79.227.19:4080 , piece_id=ANOMNO3HLII5GPWIT2RDX6P7WOBTVJXMOJM5PAPDMK7ST2NYJWMA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.456387102 (298h54m23.053158136s) [noise=true , address=145.79.227.37:4569 , piece_id=PTE4YT3YJ5M3NDKXKQOUFTGQ3T5FDAUZB74QHJBVH6HQL53QXLUA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.456987483 (298h54m23.053758515s) [noise=true , address=145.79.227.4:4190 , piece_id=LDID6UGFJ4GWYK635S5OE6334DEULOPVSOLOCLYQCBOFTXRZEKRQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.457980795 (298h54m23.054751829s) [noise=true , address=79.127.219.45:4855 , piece_id=644Z2MAXJ423ZF4JGZ2YLPM6XJRZT7IUJCZRRE7ZJMHB3EW7KB4Q, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.459806473 (298h54m23.056577503s) [noise=true , address=145.79.227.50:4376 , piece_id=BHABIBKA3ZENQUNU2OVBVUJZVWEQRZD435EHA3MEB25KMNBVFQIA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.460305248 (298h54m23.057076281s) [noise=true , address=77.90.31.16:4737 , piece_id=SMQOC6BKSIEIE5FXEYZGOIWRZ6GORC7GCOVEOFZXPQX4FGILVIPA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.461552822 (298h54m23.058323848s) [noise=true , address=79.127.219.41:4926 , piece_id=AVSCKJ5WDK4OBWWUSLUQNOPZI7BGTTDGT4XJYSLQAJPUHRYY4KVQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.463144575 (298h54m23.059915608s) [noise=true , address=145.79.227.36:4518 , piece_id=R5PDKAATCPFI3LOOUX6HJ4OP64NQ72N3RAOZS3GYFH5VSWPFU6OQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.464837828 (298h54m23.06160886s) [noise=true , address=145.79.227.26:4656 , piece_id=DDNFTYMUJMFHTZXFHJTFUURSVOJBQX2KVT6UWLBCZ5TIOVBDVOVQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.466411504 (298h54m23.063182537s) [noise=true , address=145.79.227.6:4261 , piece_id=CYOWHNR7X3Y4YE5SFHPNDBOU5QQ6M6DZIRAO3VX56KQBL3OP2FLQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.469091278 (298h54m23.065862311s) [noise=true , address=45.91.116.11:4118 , piece_id=CVKN5WPP2OTRAHYWJKDLCJV3OZ3I6G5CLYXTSEKJDQD7LRTJ55EA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.586976347 (298h54m23.183747361s) [upload=334781]: Next returned nil segment Mar 18 21:54:25.587000505 (298h54m23.183771534s) [upload=334781]: Waiting for error group managing segments... Mar 18 21:54:25.596319798 (298h54m23.193090825s) [noise=true , address=79.127.219.41:4926 , piece_id=AVSCKJ5WDK4OBWWUSLUQNOPZI7BGTTDGT4XJYSLQAJPUHRYY4KVQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.596985876 (298h54m23.1937569s) [noise=true , address=45.91.116.9:4264 , piece_id=EE3TXU5KSPHKMNNCI3BHIGFKQNYZQRBDO4YT3TTR5U66YJMFT6XQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.598026901 (298h54m23.194797934s) [noise=true , address=145.79.227.26:4656 , piece_id=DDNFTYMUJMFHTZXFHJTFUURSVOJBQX2KVT6UWLBCZ5TIOVBDVOVQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.598566162 (298h54m23.195337186s) [noise=true , address=145.79.227.50:4376 , piece_id=BHABIBKA3ZENQUNU2OVBVUJZVWEQRZD435EHA3MEB25KMNBVFQIA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.598781105 (298h54m23.19555213s) [noise=true , address=77.90.31.16:4737 , piece_id=SMQOC6BKSIEIE5FXEYZGOIWRZ6GORC7GCOVEOFZXPQX4FGILVIPA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.600260475 (298h54m23.197031509s) [noise=true , address=145.79.227.6:4261 , piece_id=CYOWHNR7X3Y4YE5SFHPNDBOU5QQ6M6DZIRAO3VX56KQBL3OP2FLQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.600344398 (298h54m23.197115411s) [noise=true , address=145.79.227.4:4190 , piece_id=LDID6UGFJ4GWYK635S5OE6334DEULOPVSOLOCLYQCBOFTXRZEKRQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.602467676 (298h54m23.19923871s) [noise=true , address=145.79.227.2:4116 , piece_id=KEKK3M6M37G7Y7HXPULI2RSX5CFGUNCM3LILYNQOZ2PASM742ZGQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.602977247 (298h54m23.199748272s) [noise=true , address=145.79.227.48:4307 , piece_id=D6WSRVCL6UO4KQ542EKZWA6IP7FLA3ZUGSBXCSZUMLPFCJ5G5CQA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.603520888 (298h54m23.200291919s) [noise=true , address=79.127.219.45:4855 , piece_id=644Z2MAXJ423ZF4JGZ2YLPM6XJRZT7IUJCZRRE7ZJMHB3EW7KB4Q, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.603907549 (298h54m23.200678573s) [noise=true , address=145.79.227.36:4518 , piece_id=R5PDKAATCPFI3LOOUX6HJ4OP64NQ72N3RAOZS3GYFH5VSWPFU6OQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.604438080 (298h54m23.201209113s) [noise=true , address=45.91.116.11:4118 , piece_id=CVKN5WPP2OTRAHYWJKDLCJV3OZ3I6G5CLYXTSEKJDQD7LRTJ55EA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.618924566 (298h54m23.21569559s) [noise=true , address=79.127.219.46:4892 , piece_id=533AWUNWA66VX4AZF7Y3AANCF65GMESWZD3IJWBC5IOKB4QMVTZA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.619077190 (298h54m23.215848214s) [seg_pos={0 0}, upload=334781]: Satisfied cohort requirements after 13 pieces Mar 18 21:54:25.619546929 (298h54m23.216317961s) [noise=true , address=145.79.227.46:4604 , piece_id=DAGTJ24IRMED5FNGQHRYTXEIXYTDVSRW2IT23MC2UC45SES3PE7Q, seg_pos={0 0}, upload=334781]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy): protocol: expected piece hash; context canceled; EOF Mar 18 21:54:25.619578476 (298h54m23.21634951s) [noise=true , address=145.79.227.46:4604 , piece_id=DAGTJ24IRMED5FNGQHRYTXEIXYTDVSRW2IT23MC2UC45SES3PE7Q, seg_pos={0 0}, upload=334781]: Long tail cancellation due to optimal threshold Mar 18 21:54:25.620166659 (298h54m23.216937744s) [noise=true , address=145.79.227.19:4080 , piece_id=ANOMNO3HLII5GPWIT2RDX6P7WOBTVJXMOJM5PAPDMK7ST2NYJWMA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KK51wudKiZb39Ku7wZhgBYEYmgsZTHfjiNNLmpBU9uuNFapt3): protocol: expected piece hash; context canceled; EOF Mar 18 21:54:25.620178720 (298h54m23.216949746s) [noise=true , address=145.79.227.19:4080 , piece_id=ANOMNO3HLII5GPWIT2RDX6P7WOBTVJXMOJM5PAPDMK7ST2NYJWMA, seg_pos={0 0}, upload=334781]: Long tail cancellation due to optimal threshold Mar 18 21:54:25.620702987 (298h54m23.217474021s) [noise=true , address=145.79.227.37:4569 , piece_id=PTE4YT3YJ5M3NDKXKQOUFTGQ3T5FDAUZB74QHJBVH6HQL53QXLUA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:18TqoejkuHLqZWhEPe3QPH1weVXnWJhaeoi8BGkvjCTLW6t4qT): protocol: expected piece hash; context canceled; EOF Mar 18 21:54:25.620718441 (298h54m23.217489474s) [noise=true , address=145.79.227.37:4569 , piece_id=PTE4YT3YJ5M3NDKXKQOUFTGQ3T5FDAUZB74QHJBVH6HQL53QXLUA, seg_pos={0 0}, upload=334781]: Long tail cancellation due to optimal threshold Mar 18 21:54:25.620748329 (298h54m23.21751937s) [seg_pos={0 0}, upload=334781]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Mar 18 21:54:25.620912612 (298h54m23.21768364s) [upload=334781]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 18 21:54:25.620920854 (298h54m23.217691883s) [upload=334781]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 21:54:38.136688455 (298h54m35.733459485s) [upload=334781]: Done uploading segments... Mar 18 21:54:38.136713048 (298h54m35.73348408s) [upload=334781]: Upload finished. err: <nil> Duration: 507.69958ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 4 out of 60 is failed (threshold=3) [...XX.X.X...................................................] duration: 283.672018ms trace: https://jaeger.dev.storj.io/trace/47004c0815b844e7 FAIL (2026-03-18 22:33:43 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 18 21:53:36.090826124 (298h53m33.687597155s) [upload=334767]: Starting upload Mar 18 21:53:36.090977797 (298h53m33.687748822s) [upload=334767]: Done starting upload Mar 18 21:53:36.091078093 (298h53m33.687849119s) [upload=334767]: Uploading segments... Mar 18 21:53:36.091268776 (298h53m33.688040083s) [upload=334767]: Got next segment. Inline: true Mar 18 21:53:36.091279074 (298h53m33.688050103s) [upload=334767]: Waiting for error group managing segments... Mar 18 21:53:36.091372978 (298h53m33.688144001s) [upload=334767]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 21:53:36.091379985 (298h53m33.688151008s) [upload=334767]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Mar 18 21:53:36.091384265 (298h53m33.688155287s) [upload=334767]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 21:54:04.718146694 (298h54m2.314917717s) [upload=334767]: Done uploading segments... Mar 18 21:54:04.718169125 (298h54m2.314940155s) [upload=334767]: Upload finished. err: <nil> Duration:* 283.712608ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 4 out of 60 is failed (threshold=3) [X..XXX......................................................] duration: 1.737145205s trace: https://jaeger.dev.storj.io/trace/27ed2876d5f3e85d FAIL (2026-03-18 22:33:16 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 1.737180848s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [XXXX.XXX....................................................] duration: 467.090178ms trace: https://jaeger.dev.storj.io/trace/6dceff78c86f0774 FAIL (2026-03-18 22:33:13 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 18 21:54:17.190275520 (298h54m14.787046534s) [upload=334781]: Starting upload Mar 18 21:54:17.190367052 (298h54m14.787138068s) [upload=334781]: Done starting upload Mar 18 21:54:17.190448332 (298h54m14.787219348s) [upload=334781]: Uploading segments... Mar 18 21:54:17.190705184 (298h54m14.787476207s) [upload=334781]: Got next segment. Inline: false Mar 18 21:54:17.190783980 (298h54m14.787555002s) [upload=334781]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 21:54:17.190804859 (298h54m14.787575882s) [upload=334781]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 18 21:54:25.452304457 (298h54m23.049075487s) [seg_pos={0 0}, upload=334781]: Begin upload segment... Mar 18 21:54:25.452374690 (298h54m23.049145713s) [seg_pos={0 0}, upload=334781]: Done begin upload segment. Mar 18 21:54:25.452463335 (298h54m23.049234366s) [noise=true , address=145.79.227.46:4604 , piece_id=DAGTJ24IRMED5FNGQHRYTXEIXYTDVSRW2IT23MC2UC45SES3PE7Q, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.452479705 (298h54m23.049250739s) [noise=true , address=45.91.116.9:4264 , piece_id=EE3TXU5KSPHKMNNCI3BHIGFKQNYZQRBDO4YT3TTR5U66YJMFT6XQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.453224160 (298h54m23.049995188s) [noise=true , address=79.127.219.46:4892 , piece_id=533AWUNWA66VX4AZF7Y3AANCF65GMESWZD3IJWBC5IOKB4QMVTZA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.454147610 (298h54m23.050918643s) [noise=true , address=145.79.227.48:4307 , piece_id=D6WSRVCL6UO4KQ542EKZWA6IP7FLA3ZUGSBXCSZUMLPFCJ5G5CQA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.454790080 (298h54m23.051561121s) [noise=true , address=145.79.227.2:4116 , piece_id=KEKK3M6M37G7Y7HXPULI2RSX5CFGUNCM3LILYNQOZ2PASM742ZGQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.455678920 (298h54m23.052449952s) [noise=true , address=145.79.227.19:4080 , piece_id=ANOMNO3HLII5GPWIT2RDX6P7WOBTVJXMOJM5PAPDMK7ST2NYJWMA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.456387102 (298h54m23.053158136s) [noise=true , address=145.79.227.37:4569 , piece_id=PTE4YT3YJ5M3NDKXKQOUFTGQ3T5FDAUZB74QHJBVH6HQL53QXLUA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.456987483 (298h54m23.053758515s) [noise=true , address=145.79.227.4:4190 , piece_id=LDID6UGFJ4GWYK635S5OE6334DEULOPVSOLOCLYQCBOFTXRZEKRQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.457980795 (298h54m23.054751829s) [noise=true , address=79.127.219.45:4855 , piece_id=644Z2MAXJ423ZF4JGZ2YLPM6XJRZT7IUJCZRRE7ZJMHB3EW7KB4Q, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.459806473 (298h54m23.056577503s) [noise=true , address=145.79.227.50:4376 , piece_id=BHABIBKA3ZENQUNU2OVBVUJZVWEQRZD435EHA3MEB25KMNBVFQIA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.460305248 (298h54m23.057076281s) [noise=true , address=77.90.31.16:4737 , piece_id=SMQOC6BKSIEIE5FXEYZGOIWRZ6GORC7GCOVEOFZXPQX4FGILVIPA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.461552822 (298h54m23.058323848s) [noise=true , address=79.127.219.41:4926 , piece_id=AVSCKJ5WDK4OBWWUSLUQNOPZI7BGTTDGT4XJYSLQAJPUHRYY4KVQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.463144575 (298h54m23.059915608s) [noise=true , address=145.79.227.36:4518 , piece_id=R5PDKAATCPFI3LOOUX6HJ4OP64NQ72N3RAOZS3GYFH5VSWPFU6OQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.464837828 (298h54m23.06160886s) [noise=true , address=145.79.227.26:4656 , piece_id=DDNFTYMUJMFHTZXFHJTFUURSVOJBQX2KVT6UWLBCZ5TIOVBDVOVQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.466411504 (298h54m23.063182537s) [noise=true , address=145.79.227.6:4261 , piece_id=CYOWHNR7X3Y4YE5SFHPNDBOU5QQ6M6DZIRAO3VX56KQBL3OP2FLQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.469091278 (298h54m23.065862311s) [noise=true , address=45.91.116.11:4118 , piece_id=CVKN5WPP2OTRAHYWJKDLCJV3OZ3I6G5CLYXTSEKJDQD7LRTJ55EA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.586976347 (298h54m23.183747361s) [upload=334781]: Next returned nil segment Mar 18 21:54:25.587000505 (298h54m23.183771534s) [upload=334781]: Waiting for error group managing segments... Mar 18 21:54:25.596319798 (298h54m23.193090825s) [noise=true , address=79.127.219.41:4926 , piece_id=AVSCKJ5WDK4OBWWUSLUQNOPZI7BGTTDGT4XJYSLQAJPUHRYY4KVQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.596985876 (298h54m23.1937569s) [noise=true , address=45.91.116.9:4264 , piece_id=EE3TXU5KSPHKMNNCI3BHIGFKQNYZQRBDO4YT3TTR5U66YJMFT6XQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.598026901 (298h54m23.194797934s) [noise=true , address=145.79.227.26:4656 , piece_id=DDNFTYMUJMFHTZXFHJTFUURSVOJBQX2KVT6UWLBCZ5TIOVBDVOVQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.598566162 (298h54m23.195337186s) [noise=true , address=145.79.227.50:4376 , piece_id=BHABIBKA3ZENQUNU2OVBVUJZVWEQRZD435EHA3MEB25KMNBVFQIA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.598781105 (298h54m23.19555213s) [noise=true , address=77.90.31.16:4737 , piece_id=SMQOC6BKSIEIE5FXEYZGOIWRZ6GORC7GCOVEOFZXPQX4FGILVIPA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.600260475 (298h54m23.197031509s) [noise=true , address=145.79.227.6:4261 , piece_id=CYOWHNR7X3Y4YE5SFHPNDBOU5QQ6M6DZIRAO3VX56KQBL3OP2FLQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.600344398 (298h54m23.197115411s) [noise=true , address=145.79.227.4:4190 , piece_id=LDID6UGFJ4GWYK635S5OE6334DEULOPVSOLOCLYQCBOFTXRZEKRQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.602467676 (298h54m23.19923871s) [noise=true , address=145.79.227.2:4116 , piece_id=KEKK3M6M37G7Y7HXPULI2RSX5CFGUNCM3LILYNQOZ2PASM742ZGQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.602977247 (298h54m23.199748272s) [noise=true , address=145.79.227.48:4307 , piece_id=D6WSRVCL6UO4KQ542EKZWA6IP7FLA3ZUGSBXCSZUMLPFCJ5G5CQA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.603520888 (298h54m23.200291919s) [noise=true , address=79.127.219.45:4855 , piece_id=644Z2MAXJ423ZF4JGZ2YLPM6XJRZT7IUJCZRRE7ZJMHB3EW7KB4Q, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.603907549 (298h54m23.200678573s) [noise=true , address=145.79.227.36:4518 , piece_id=R5PDKAATCPFI3LOOUX6HJ4OP64NQ72N3RAOZS3GYFH5VSWPFU6OQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.604438080 (298h54m23.201209113s) [noise=true , address=45.91.116.11:4118 , piece_id=CVKN5WPP2OTRAHYWJKDLCJV3OZ3I6G5CLYXTSEKJDQD7LRTJ55EA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.618924566 (298h54m23.21569559s) [noise=true , address=79.127.219.46:4892 , piece_id=533AWUNWA66VX4AZF7Y3AANCF65GMESWZD3IJWBC5IOKB4QMVTZA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.619077190 (298h54m23.215848214s) [seg_pos={0 0}, upload=334781]: Satisfied cohort requirements after 13 pieces Mar 18 21:54:25.619546929 (298h54m23.216317961s) [noise=true , address=145.79.227.46:4604 , piece_id=DAGTJ24IRMED5FNGQHRYTXEIXYTDVSRW2IT23MC2UC45SES3PE7Q, seg_pos={0 0}, upload=334781]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy): protocol: expected piece hash; context canceled; EOF Mar 18 21:54:25.619578476 (298h54m23.21634951s) [noise=true , address=145.79.227.46:4604 , piece_id=DAGTJ24IRMED5FNGQHRYTXEIXYTDVSRW2IT23MC2UC45SES3PE7Q, seg_pos={0 0}, upload=334781]: Long tail cancellation due to optimal threshold Mar 18 21:54:25.620166659 (298h54m23.216937744s) [noise=true , address=145.79.227.19:4080 , piece_id=ANOMNO3HLII5GPWIT2RDX6P7WOBTVJXMOJM5PAPDMK7ST2NYJWMA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KK51wudKiZb39Ku7wZhgBYEYmgsZTHfjiNNLmpBU9uuNFapt3): protocol: expected piece hash; context canceled; EOF Mar 18 21:54:25.620178720 (298h54m23.216949746s) [noise=true , address=145.79.227.19:4080 , piece_id=ANOMNO3HLII5GPWIT2RDX6P7WOBTVJXMOJM5PAPDMK7ST2NYJWMA, seg_pos={0 0}, upload=334781]: Long tail cancellation due to optimal threshold Mar 18 21:54:25.620702987 (298h54m23.217474021s) [noise=true , address=145.79.227.37:4569 , piece_id=PTE4YT3YJ5M3NDKXKQOUFTGQ3T5FDAUZB74QHJBVH6HQL53QXLUA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:18TqoejkuHLqZWhEPe3QPH1weVXnWJhaeoi8BGkvjCTLW6t4qT): protocol: expected piece hash; context canceled; EOF Mar 18 21:54:25.620718441 (298h54m23.217489474s) [noise=true , address=145.79.227.37:4569 , piece_id=PTE4YT3YJ5M3NDKXKQOUFTGQ3T5FDAUZB74QHJBVH6HQL53QXLUA, seg_pos={0 0}, upload=334781]: Long tail cancellation due to optimal threshold Mar 18 21:54:25.620748329 (298h54m23.21751937s) [seg_pos={0 0}, upload=334781]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Mar 18 21:54:25.620912612 (298h54m23.21768364s) [upload=334781]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 18 21:54:25.620920854 (298h54m23.217691883s) [upload=334781]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 21:54:38.136688455 (298h54m35.733459485s) [upload=334781]: Done uploading segments... Mar 18 21:54:38.136713048 (298h54m35.73348408s) [upload=334781]: Upload finished. err: <nil> Duration: 467.119561ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 5 out of 60 is failed (threshold=3) [.X..X..X.XX.................................................] duration: 244.356879ms trace: https://jaeger.dev.storj.io/trace/7c4371cfa8f7834c FAIL (2026-03-18 22:33:05 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 244.3847ms core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [.XXXX.XXX...................................................] duration: 443.002745ms trace: https://jaeger.dev.storj.io/trace/6dceff78c86f0774 FAIL (2026-03-18 22:32:55 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 18 21:54:17.190275520 (298h54m14.787046534s) [upload=334781]: Starting upload Mar 18 21:54:17.190367052 (298h54m14.787138068s) [upload=334781]: Done starting upload Mar 18 21:54:17.190448332 (298h54m14.787219348s) [upload=334781]: Uploading segments... Mar 18 21:54:17.190705184 (298h54m14.787476207s) [upload=334781]: Got next segment. Inline: false Mar 18 21:54:17.190783980 (298h54m14.787555002s) [upload=334781]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 21:54:17.190804859 (298h54m14.787575882s) [upload=334781]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 18 21:54:25.452304457 (298h54m23.049075487s) [seg_pos={0 0}, upload=334781]: Begin upload segment... Mar 18 21:54:25.452374690 (298h54m23.049145713s) [seg_pos={0 0}, upload=334781]: Done begin upload segment. Mar 18 21:54:25.452463335 (298h54m23.049234366s) [noise=true , address=145.79.227.46:4604 , piece_id=DAGTJ24IRMED5FNGQHRYTXEIXYTDVSRW2IT23MC2UC45SES3PE7Q, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.452479705 (298h54m23.049250739s) [noise=true , address=45.91.116.9:4264 , piece_id=EE3TXU5KSPHKMNNCI3BHIGFKQNYZQRBDO4YT3TTR5U66YJMFT6XQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.453224160 (298h54m23.049995188s) [noise=true , address=79.127.219.46:4892 , piece_id=533AWUNWA66VX4AZF7Y3AANCF65GMESWZD3IJWBC5IOKB4QMVTZA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.454147610 (298h54m23.050918643s) [noise=true , address=145.79.227.48:4307 , piece_id=D6WSRVCL6UO4KQ542EKZWA6IP7FLA3ZUGSBXCSZUMLPFCJ5G5CQA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.454790080 (298h54m23.051561121s) [noise=true , address=145.79.227.2:4116 , piece_id=KEKK3M6M37G7Y7HXPULI2RSX5CFGUNCM3LILYNQOZ2PASM742ZGQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.455678920 (298h54m23.052449952s) [noise=true , address=145.79.227.19:4080 , piece_id=ANOMNO3HLII5GPWIT2RDX6P7WOBTVJXMOJM5PAPDMK7ST2NYJWMA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.456387102 (298h54m23.053158136s) [noise=true , address=145.79.227.37:4569 , piece_id=PTE4YT3YJ5M3NDKXKQOUFTGQ3T5FDAUZB74QHJBVH6HQL53QXLUA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.456987483 (298h54m23.053758515s) [noise=true , address=145.79.227.4:4190 , piece_id=LDID6UGFJ4GWYK635S5OE6334DEULOPVSOLOCLYQCBOFTXRZEKRQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.457980795 (298h54m23.054751829s) [noise=true , address=79.127.219.45:4855 , piece_id=644Z2MAXJ423ZF4JGZ2YLPM6XJRZT7IUJCZRRE7ZJMHB3EW7KB4Q, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.459806473 (298h54m23.056577503s) [noise=true , address=145.79.227.50:4376 , piece_id=BHABIBKA3ZENQUNU2OVBVUJZVWEQRZD435EHA3MEB25KMNBVFQIA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.460305248 (298h54m23.057076281s) [noise=true , address=77.90.31.16:4737 , piece_id=SMQOC6BKSIEIE5FXEYZGOIWRZ6GORC7GCOVEOFZXPQX4FGILVIPA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.461552822 (298h54m23.058323848s) [noise=true , address=79.127.219.41:4926 , piece_id=AVSCKJ5WDK4OBWWUSLUQNOPZI7BGTTDGT4XJYSLQAJPUHRYY4KVQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.463144575 (298h54m23.059915608s) [noise=true , address=145.79.227.36:4518 , piece_id=R5PDKAATCPFI3LOOUX6HJ4OP64NQ72N3RAOZS3GYFH5VSWPFU6OQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.464837828 (298h54m23.06160886s) [noise=true , address=145.79.227.26:4656 , piece_id=DDNFTYMUJMFHTZXFHJTFUURSVOJBQX2KVT6UWLBCZ5TIOVBDVOVQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.466411504 (298h54m23.063182537s) [noise=true , address=145.79.227.6:4261 , piece_id=CYOWHNR7X3Y4YE5SFHPNDBOU5QQ6M6DZIRAO3VX56KQBL3OP2FLQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.469091278 (298h54m23.065862311s) [noise=true , address=45.91.116.11:4118 , piece_id=CVKN5WPP2OTRAHYWJKDLCJV3OZ3I6G5CLYXTSEKJDQD7LRTJ55EA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.586976347 (298h54m23.183747361s) [upload=334781]: Next returned nil segment Mar 18 21:54:25.587000505 (298h54m23.183771534s) [upload=334781]: Waiting for error group managing segments... Mar 18 21:54:25.596319798 (298h54m23.193090825s) [noise=true , address=79.127.219.41:4926 , piece_id=AVSCKJ5WDK4OBWWUSLUQNOPZI7BGTTDGT4XJYSLQAJPUHRYY4KVQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.596985876 (298h54m23.1937569s) [noise=true , address=45.91.116.9:4264 , piece_id=EE3TXU5KSPHKMNNCI3BHIGFKQNYZQRBDO4YT3TTR5U66YJMFT6XQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.598026901 (298h54m23.194797934s) [noise=true , address=145.79.227.26:4656 , piece_id=DDNFTYMUJMFHTZXFHJTFUURSVOJBQX2KVT6UWLBCZ5TIOVBDVOVQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.598566162 (298h54m23.195337186s) [noise=true , address=145.79.227.50:4376 , piece_id=BHABIBKA3ZENQUNU2OVBVUJZVWEQRZD435EHA3MEB25KMNBVFQIA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.598781105 (298h54m23.19555213s) [noise=true , address=77.90.31.16:4737 , piece_id=SMQOC6BKSIEIE5FXEYZGOIWRZ6GORC7GCOVEOFZXPQX4FGILVIPA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.600260475 (298h54m23.197031509s) [noise=true , address=145.79.227.6:4261 , piece_id=CYOWHNR7X3Y4YE5SFHPNDBOU5QQ6M6DZIRAO3VX56KQBL3OP2FLQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.600344398 (298h54m23.197115411s) [noise=true , address=145.79.227.4:4190 , piece_id=LDID6UGFJ4GWYK635S5OE6334DEULOPVSOLOCLYQCBOFTXRZEKRQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.602467676 (298h54m23.19923871s) [noise=true , address=145.79.227.2:4116 , piece_id=KEKK3M6M37G7Y7HXPULI2RSX5CFGUNCM3LILYNQOZ2PASM742ZGQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.602977247 (298h54m23.199748272s) [noise=true , address=145.79.227.48:4307 , piece_id=D6WSRVCL6UO4KQ542EKZWA6IP7FLA3ZUGSBXCSZUMLPFCJ5G5CQA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.603520888 (298h54m23.200291919s) [noise=true , address=79.127.219.45:4855 , piece_id=644Z2MAXJ423ZF4JGZ2YLPM6XJRZT7IUJCZRRE7ZJMHB3EW7KB4Q, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.603907549 (298h54m23.200678573s) [noise=true , address=145.79.227.36:4518 , piece_id=R5PDKAATCPFI3LOOUX6HJ4OP64NQ72N3RAOZS3GYFH5VSWPFU6OQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.604438080 (298h54m23.201209113s) [noise=true , address=45.91.116.11:4118 , piece_id=CVKN5WPP2OTRAHYWJKDLCJV3OZ3I6G5CLYXTSEKJDQD7LRTJ55EA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.618924566 (298h54m23.21569559s) [noise=true , address=79.127.219.46:4892 , piece_id=533AWUNWA66VX4AZF7Y3AANCF65GMESWZD3IJWBC5IOKB4QMVTZA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.619077190 (298h54m23.215848214s) [seg_pos={0 0}, upload=334781]: Satisfied cohort requirements after 13 pieces Mar 18 21:54:25.619546929 (298h54m23.216317961s) [noise=true , address=145.79.227.46:4604 , piece_id=DAGTJ24IRMED5FNGQHRYTXEIXYTDVSRW2IT23MC2UC45SES3PE7Q, seg_pos={0 0}, upload=334781]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy): protocol: expected piece hash; context canceled; EOF Mar 18 21:54:25.619578476 (298h54m23.21634951s) [noise=true , address=145.79.227.46:4604 , piece_id=DAGTJ24IRMED5FNGQHRYTXEIXYTDVSRW2IT23MC2UC45SES3PE7Q, seg_pos={0 0}, upload=334781]: Long tail cancellation due to optimal threshold Mar 18 21:54:25.620166659 (298h54m23.216937744s) [noise=true , address=145.79.227.19:4080 , piece_id=ANOMNO3HLII5GPWIT2RDX6P7WOBTVJXMOJM5PAPDMK7ST2NYJWMA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KK51wudKiZb39Ku7wZhgBYEYmgsZTHfjiNNLmpBU9uuNFapt3): protocol: expected piece hash; context canceled; EOF Mar 18 21:54:25.620178720 (298h54m23.216949746s) [noise=true , address=145.79.227.19:4080 , piece_id=ANOMNO3HLII5GPWIT2RDX6P7WOBTVJXMOJM5PAPDMK7ST2NYJWMA, seg_pos={0 0}, upload=334781]: Long tail cancellation due to optimal threshold Mar 18 21:54:25.620702987 (298h54m23.217474021s) [noise=true , address=145.79.227.37:4569 , piece_id=PTE4YT3YJ5M3NDKXKQOUFTGQ3T5FDAUZB74QHJBVH6HQL53QXLUA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:18TqoejkuHLqZWhEPe3QPH1weVXnWJhaeoi8BGkvjCTLW6t4qT): protocol: expected piece hash; context canceled; EOF Mar 18 21:54:25.620718441 (298h54m23.217489474s) [noise=true , address=145.79.227.37:4569 , piece_id=PTE4YT3YJ5M3NDKXKQOUFTGQ3T5FDAUZB74QHJBVH6HQL53QXLUA, seg_pos={0 0}, upload=334781]: Long tail cancellation due to optimal threshold Mar 18 21:54:25.620748329 (298h54m23.21751937s) [seg_pos={0 0}, upload=334781]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Mar 18 21:54:25.620912612 (298h54m23.21768364s) [upload=334781]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 18 21:54:25.620920854 (298h54m23.217691883s) [upload=334781]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 21:54:38.136688455 (298h54m35.733459485s) [upload=334781]: Done uploading segments... Mar 18 21:54:38.136713048 (298h54m35.73348408s) [upload=334781]: Upload finished. err: <nil> Duration: 443.037543ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 9 out of 60 is failed (threshold=6) [XXXXXXXXX...................................................] duration: 735.765524ms trace: https://jaeger.dev.storj.io/trace/6cabdd7d47c9cb44 FAIL (2026-03-18 22:32:47 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 735.796835ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 5 out of 60 is failed (threshold=3) [X...XX.X.X..................................................] duration: 257.298773ms trace: https://jaeger.dev.storj.io/trace/47004c0815b844e7 FAIL (2026-03-18 22:32:34 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 18 21:53:36.090826124 (298h53m33.687597155s) [upload=334767]: Starting upload Mar 18 21:53:36.090977797 (298h53m33.687748822s) [upload=334767]: Done starting upload Mar 18 21:53:36.091078093 (298h53m33.687849119s) [upload=334767]: Uploading segments... Mar 18 21:53:36.091268776 (298h53m33.688040083s) [upload=334767]: Got next segment. Inline: true Mar 18 21:53:36.091279074 (298h53m33.688050103s) [upload=334767]: Waiting for error group managing segments... Mar 18 21:53:36.091372978 (298h53m33.688144001s) [upload=334767]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 21:53:36.091379985 (298h53m33.688151008s) [upload=334767]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Mar 18 21:53:36.091384265 (298h53m33.688155287s) [upload=334767]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 21:54:04.718146694 (298h54m2.314917717s) [upload=334767]: Done uploading segments... Mar 18 21:54:04.718169125 (298h54m2.314940155s) [upload=334767]: Upload finished. err: <nil> Duration:* 257.325916ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.X..XXX.....................................................] duration: 1.683500717s trace: https://jaeger.dev.storj.io/trace/27ed2876d5f3e85d FAIL (2026-03-18 22:32:15 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 1.683534278s core uplink us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 10 out of 60 is failed (threshold=6) [XXXXXXXXXX..................................................] duration: 553.940723ms trace: https://jaeger.dev.storj.io/trace/6cabdd7d47c9cb44 FAIL (2026-03-18 22:32:13 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 553.969362ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 5 out of 60 is failed (threshold=3) [.X...XX.X.X.................................................] duration: 237.281648ms trace: https://jaeger.dev.storj.io/trace/47004c0815b844e7 FAIL (2026-03-18 22:32:07 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 18 21:53:36.090826124 (298h53m33.687597155s) [upload=334767]: Starting upload Mar 18 21:53:36.090977797 (298h53m33.687748822s) [upload=334767]: Done starting upload Mar 18 21:53:36.091078093 (298h53m33.687849119s) [upload=334767]: Uploading segments... Mar 18 21:53:36.091268776 (298h53m33.688040083s) [upload=334767]: Got next segment. Inline: true Mar 18 21:53:36.091279074 (298h53m33.688050103s) [upload=334767]: Waiting for error group managing segments... Mar 18 21:53:36.091372978 (298h53m33.688144001s) [upload=334767]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 21:53:36.091379985 (298h53m33.688151008s) [upload=334767]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Mar 18 21:53:36.091384265 (298h53m33.688155287s) [upload=334767]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 21:54:04.718146694 (298h54m2.314917717s) [upload=334767]: Done uploading segments... Mar 18 21:54:04.718169125 (298h54m2.314940155s) [upload=334767]: Upload finished. err: <nil> Duration:* 237.311861ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 6 out of 60 is failed (threshold=3) [X.X..X..X.XX................................................] duration: 225.979377ms trace: https://jaeger.dev.storj.io/trace/7c4371cfa8f7834c FAIL (2026-03-18 22:31:52 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 226.02443ms core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [..XXXX.XXX..................................................] duration: 466.640611ms trace: https://jaeger.dev.storj.io/trace/6dceff78c86f0774 FAIL (2026-03-18 22:31:50 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 18 21:54:17.190275520 (298h54m14.787046534s) [upload=334781]: Starting upload Mar 18 21:54:17.190367052 (298h54m14.787138068s) [upload=334781]: Done starting upload Mar 18 21:54:17.190448332 (298h54m14.787219348s) [upload=334781]: Uploading segments... Mar 18 21:54:17.190705184 (298h54m14.787476207s) [upload=334781]: Got next segment. Inline: false Mar 18 21:54:17.190783980 (298h54m14.787555002s) [upload=334781]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 21:54:17.190804859 (298h54m14.787575882s) [upload=334781]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 18 21:54:25.452304457 (298h54m23.049075487s) [seg_pos={0 0}, upload=334781]: Begin upload segment... Mar 18 21:54:25.452374690 (298h54m23.049145713s) [seg_pos={0 0}, upload=334781]: Done begin upload segment. Mar 18 21:54:25.452463335 (298h54m23.049234366s) [noise=true , address=145.79.227.46:4604 , piece_id=DAGTJ24IRMED5FNGQHRYTXEIXYTDVSRW2IT23MC2UC45SES3PE7Q, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.452479705 (298h54m23.049250739s) [noise=true , address=45.91.116.9:4264 , piece_id=EE3TXU5KSPHKMNNCI3BHIGFKQNYZQRBDO4YT3TTR5U66YJMFT6XQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.453224160 (298h54m23.049995188s) [noise=true , address=79.127.219.46:4892 , piece_id=533AWUNWA66VX4AZF7Y3AANCF65GMESWZD3IJWBC5IOKB4QMVTZA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.454147610 (298h54m23.050918643s) [noise=true , address=145.79.227.48:4307 , piece_id=D6WSRVCL6UO4KQ542EKZWA6IP7FLA3ZUGSBXCSZUMLPFCJ5G5CQA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.454790080 (298h54m23.051561121s) [noise=true , address=145.79.227.2:4116 , piece_id=KEKK3M6M37G7Y7HXPULI2RSX5CFGUNCM3LILYNQOZ2PASM742ZGQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.455678920 (298h54m23.052449952s) [noise=true , address=145.79.227.19:4080 , piece_id=ANOMNO3HLII5GPWIT2RDX6P7WOBTVJXMOJM5PAPDMK7ST2NYJWMA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.456387102 (298h54m23.053158136s) [noise=true , address=145.79.227.37:4569 , piece_id=PTE4YT3YJ5M3NDKXKQOUFTGQ3T5FDAUZB74QHJBVH6HQL53QXLUA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.456987483 (298h54m23.053758515s) [noise=true , address=145.79.227.4:4190 , piece_id=LDID6UGFJ4GWYK635S5OE6334DEULOPVSOLOCLYQCBOFTXRZEKRQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.457980795 (298h54m23.054751829s) [noise=true , address=79.127.219.45:4855 , piece_id=644Z2MAXJ423ZF4JGZ2YLPM6XJRZT7IUJCZRRE7ZJMHB3EW7KB4Q, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.459806473 (298h54m23.056577503s) [noise=true , address=145.79.227.50:4376 , piece_id=BHABIBKA3ZENQUNU2OVBVUJZVWEQRZD435EHA3MEB25KMNBVFQIA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.460305248 (298h54m23.057076281s) [noise=true , address=77.90.31.16:4737 , piece_id=SMQOC6BKSIEIE5FXEYZGOIWRZ6GORC7GCOVEOFZXPQX4FGILVIPA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.461552822 (298h54m23.058323848s) [noise=true , address=79.127.219.41:4926 , piece_id=AVSCKJ5WDK4OBWWUSLUQNOPZI7BGTTDGT4XJYSLQAJPUHRYY4KVQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.463144575 (298h54m23.059915608s) [noise=true , address=145.79.227.36:4518 , piece_id=R5PDKAATCPFI3LOOUX6HJ4OP64NQ72N3RAOZS3GYFH5VSWPFU6OQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.464837828 (298h54m23.06160886s) [noise=true , address=145.79.227.26:4656 , piece_id=DDNFTYMUJMFHTZXFHJTFUURSVOJBQX2KVT6UWLBCZ5TIOVBDVOVQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.466411504 (298h54m23.063182537s) [noise=true , address=145.79.227.6:4261 , piece_id=CYOWHNR7X3Y4YE5SFHPNDBOU5QQ6M6DZIRAO3VX56KQBL3OP2FLQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.469091278 (298h54m23.065862311s) [noise=true , address=45.91.116.11:4118 , piece_id=CVKN5WPP2OTRAHYWJKDLCJV3OZ3I6G5CLYXTSEKJDQD7LRTJ55EA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.586976347 (298h54m23.183747361s) [upload=334781]: Next returned nil segment Mar 18 21:54:25.587000505 (298h54m23.183771534s) [upload=334781]: Waiting for error group managing segments... Mar 18 21:54:25.596319798 (298h54m23.193090825s) [noise=true , address=79.127.219.41:4926 , piece_id=AVSCKJ5WDK4OBWWUSLUQNOPZI7BGTTDGT4XJYSLQAJPUHRYY4KVQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.596985876 (298h54m23.1937569s) [noise=true , address=45.91.116.9:4264 , piece_id=EE3TXU5KSPHKMNNCI3BHIGFKQNYZQRBDO4YT3TTR5U66YJMFT6XQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.598026901 (298h54m23.194797934s) [noise=true , address=145.79.227.26:4656 , piece_id=DDNFTYMUJMFHTZXFHJTFUURSVOJBQX2KVT6UWLBCZ5TIOVBDVOVQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.598566162 (298h54m23.195337186s) [noise=true , address=145.79.227.50:4376 , piece_id=BHABIBKA3ZENQUNU2OVBVUJZVWEQRZD435EHA3MEB25KMNBVFQIA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.598781105 (298h54m23.19555213s) [noise=true , address=77.90.31.16:4737 , piece_id=SMQOC6BKSIEIE5FXEYZGOIWRZ6GORC7GCOVEOFZXPQX4FGILVIPA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.600260475 (298h54m23.197031509s) [noise=true , address=145.79.227.6:4261 , piece_id=CYOWHNR7X3Y4YE5SFHPNDBOU5QQ6M6DZIRAO3VX56KQBL3OP2FLQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.600344398 (298h54m23.197115411s) [noise=true , address=145.79.227.4:4190 , piece_id=LDID6UGFJ4GWYK635S5OE6334DEULOPVSOLOCLYQCBOFTXRZEKRQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.602467676 (298h54m23.19923871s) [noise=true , address=145.79.227.2:4116 , piece_id=KEKK3M6M37G7Y7HXPULI2RSX5CFGUNCM3LILYNQOZ2PASM742ZGQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.602977247 (298h54m23.199748272s) [noise=true , address=145.79.227.48:4307 , piece_id=D6WSRVCL6UO4KQ542EKZWA6IP7FLA3ZUGSBXCSZUMLPFCJ5G5CQA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.603520888 (298h54m23.200291919s) [noise=true , address=79.127.219.45:4855 , piece_id=644Z2MAXJ423ZF4JGZ2YLPM6XJRZT7IUJCZRRE7ZJMHB3EW7KB4Q, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.603907549 (298h54m23.200678573s) [noise=true , address=145.79.227.36:4518 , piece_id=R5PDKAATCPFI3LOOUX6HJ4OP64NQ72N3RAOZS3GYFH5VSWPFU6OQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.604438080 (298h54m23.201209113s) [noise=true , address=45.91.116.11:4118 , piece_id=CVKN5WPP2OTRAHYWJKDLCJV3OZ3I6G5CLYXTSEKJDQD7LRTJ55EA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.618924566 (298h54m23.21569559s) [noise=true , address=79.127.219.46:4892 , piece_id=533AWUNWA66VX4AZF7Y3AANCF65GMESWZD3IJWBC5IOKB4QMVTZA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.619077190 (298h54m23.215848214s) [seg_pos={0 0}, upload=334781]: Satisfied cohort requirements after 13 pieces Mar 18 21:54:25.619546929 (298h54m23.216317961s) [noise=true , address=145.79.227.46:4604 , piece_id=DAGTJ24IRMED5FNGQHRYTXEIXYTDVSRW2IT23MC2UC45SES3PE7Q, seg_pos={0 0}, upload=334781]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy): protocol: expected piece hash; context canceled; EOF Mar 18 21:54:25.619578476 (298h54m23.21634951s) [noise=true , address=145.79.227.46:4604 , piece_id=DAGTJ24IRMED5FNGQHRYTXEIXYTDVSRW2IT23MC2UC45SES3PE7Q, seg_pos={0 0}, upload=334781]: Long tail cancellation due to optimal threshold Mar 18 21:54:25.620166659 (298h54m23.216937744s) [noise=true , address=145.79.227.19:4080 , piece_id=ANOMNO3HLII5GPWIT2RDX6P7WOBTVJXMOJM5PAPDMK7ST2NYJWMA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KK51wudKiZb39Ku7wZhgBYEYmgsZTHfjiNNLmpBU9uuNFapt3): protocol: expected piece hash; context canceled; EOF Mar 18 21:54:25.620178720 (298h54m23.216949746s) [noise=true , address=145.79.227.19:4080 , piece_id=ANOMNO3HLII5GPWIT2RDX6P7WOBTVJXMOJM5PAPDMK7ST2NYJWMA, seg_pos={0 0}, upload=334781]: Long tail cancellation due to optimal threshold Mar 18 21:54:25.620702987 (298h54m23.217474021s) [noise=true , address=145.79.227.37:4569 , piece_id=PTE4YT3YJ5M3NDKXKQOUFTGQ3T5FDAUZB74QHJBVH6HQL53QXLUA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:18TqoejkuHLqZWhEPe3QPH1weVXnWJhaeoi8BGkvjCTLW6t4qT): protocol: expected piece hash; context canceled; EOF Mar 18 21:54:25.620718441 (298h54m23.217489474s) [noise=true , address=145.79.227.37:4569 , piece_id=PTE4YT3YJ5M3NDKXKQOUFTGQ3T5FDAUZB74QHJBVH6HQL53QXLUA, seg_pos={0 0}, upload=334781]: Long tail cancellation due to optimal threshold Mar 18 21:54:25.620748329 (298h54m23.21751937s) [seg_pos={0 0}, upload=334781]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Mar 18 21:54:25.620912612 (298h54m23.21768364s) [upload=334781]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 18 21:54:25.620920854 (298h54m23.217691883s) [upload=334781]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 21:54:38.136688455 (298h54m35.733459485s) [upload=334781]: Done uploading segments... Mar 18 21:54:38.136713048 (298h54m35.73348408s) [upload=334781]: Upload finished. err: <nil> Duration: 466.670802ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 5 out of 60 is failed (threshold=3) [X.X..XXX....................................................] duration: 1.755461048s trace: https://jaeger.dev.storj.io/trace/27ed2876d5f3e85d FAIL (2026-03-18 22:31:32 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 1.755524185s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [...XXXX.XXX.................................................] duration: 458.765069ms trace: https://jaeger.dev.storj.io/trace/6dceff78c86f0774 FAIL (2026-03-18 22:31:23 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 18 21:54:17.190275520 (298h54m14.787046534s) [upload=334781]: Starting upload Mar 18 21:54:17.190367052 (298h54m14.787138068s) [upload=334781]: Done starting upload Mar 18 21:54:17.190448332 (298h54m14.787219348s) [upload=334781]: Uploading segments... Mar 18 21:54:17.190705184 (298h54m14.787476207s) [upload=334781]: Got next segment. Inline: false Mar 18 21:54:17.190783980 (298h54m14.787555002s) [upload=334781]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 21:54:17.190804859 (298h54m14.787575882s) [upload=334781]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 18 21:54:25.452304457 (298h54m23.049075487s) [seg_pos={0 0}, upload=334781]: Begin upload segment... Mar 18 21:54:25.452374690 (298h54m23.049145713s) [seg_pos={0 0}, upload=334781]: Done begin upload segment. Mar 18 21:54:25.452463335 (298h54m23.049234366s) [noise=true , address=145.79.227.46:4604 , piece_id=DAGTJ24IRMED5FNGQHRYTXEIXYTDVSRW2IT23MC2UC45SES3PE7Q, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.452479705 (298h54m23.049250739s) [noise=true , address=45.91.116.9:4264 , piece_id=EE3TXU5KSPHKMNNCI3BHIGFKQNYZQRBDO4YT3TTR5U66YJMFT6XQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.453224160 (298h54m23.049995188s) [noise=true , address=79.127.219.46:4892 , piece_id=533AWUNWA66VX4AZF7Y3AANCF65GMESWZD3IJWBC5IOKB4QMVTZA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.454147610 (298h54m23.050918643s) [noise=true , address=145.79.227.48:4307 , piece_id=D6WSRVCL6UO4KQ542EKZWA6IP7FLA3ZUGSBXCSZUMLPFCJ5G5CQA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.454790080 (298h54m23.051561121s) [noise=true , address=145.79.227.2:4116 , piece_id=KEKK3M6M37G7Y7HXPULI2RSX5CFGUNCM3LILYNQOZ2PASM742ZGQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.455678920 (298h54m23.052449952s) [noise=true , address=145.79.227.19:4080 , piece_id=ANOMNO3HLII5GPWIT2RDX6P7WOBTVJXMOJM5PAPDMK7ST2NYJWMA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.456387102 (298h54m23.053158136s) [noise=true , address=145.79.227.37:4569 , piece_id=PTE4YT3YJ5M3NDKXKQOUFTGQ3T5FDAUZB74QHJBVH6HQL53QXLUA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.456987483 (298h54m23.053758515s) [noise=true , address=145.79.227.4:4190 , piece_id=LDID6UGFJ4GWYK635S5OE6334DEULOPVSOLOCLYQCBOFTXRZEKRQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.457980795 (298h54m23.054751829s) [noise=true , address=79.127.219.45:4855 , piece_id=644Z2MAXJ423ZF4JGZ2YLPM6XJRZT7IUJCZRRE7ZJMHB3EW7KB4Q, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.459806473 (298h54m23.056577503s) [noise=true , address=145.79.227.50:4376 , piece_id=BHABIBKA3ZENQUNU2OVBVUJZVWEQRZD435EHA3MEB25KMNBVFQIA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.460305248 (298h54m23.057076281s) [noise=true , address=77.90.31.16:4737 , piece_id=SMQOC6BKSIEIE5FXEYZGOIWRZ6GORC7GCOVEOFZXPQX4FGILVIPA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.461552822 (298h54m23.058323848s) [noise=true , address=79.127.219.41:4926 , piece_id=AVSCKJ5WDK4OBWWUSLUQNOPZI7BGTTDGT4XJYSLQAJPUHRYY4KVQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.463144575 (298h54m23.059915608s) [noise=true , address=145.79.227.36:4518 , piece_id=R5PDKAATCPFI3LOOUX6HJ4OP64NQ72N3RAOZS3GYFH5VSWPFU6OQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.464837828 (298h54m23.06160886s) [noise=true , address=145.79.227.26:4656 , piece_id=DDNFTYMUJMFHTZXFHJTFUURSVOJBQX2KVT6UWLBCZ5TIOVBDVOVQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.466411504 (298h54m23.063182537s) [noise=true , address=145.79.227.6:4261 , piece_id=CYOWHNR7X3Y4YE5SFHPNDBOU5QQ6M6DZIRAO3VX56KQBL3OP2FLQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.469091278 (298h54m23.065862311s) [noise=true , address=45.91.116.11:4118 , piece_id=CVKN5WPP2OTRAHYWJKDLCJV3OZ3I6G5CLYXTSEKJDQD7LRTJ55EA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.586976347 (298h54m23.183747361s) [upload=334781]: Next returned nil segment Mar 18 21:54:25.587000505 (298h54m23.183771534s) [upload=334781]: Waiting for error group managing segments... Mar 18 21:54:25.596319798 (298h54m23.193090825s) [noise=true , address=79.127.219.41:4926 , piece_id=AVSCKJ5WDK4OBWWUSLUQNOPZI7BGTTDGT4XJYSLQAJPUHRYY4KVQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.596985876 (298h54m23.1937569s) [noise=true , address=45.91.116.9:4264 , piece_id=EE3TXU5KSPHKMNNCI3BHIGFKQNYZQRBDO4YT3TTR5U66YJMFT6XQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.598026901 (298h54m23.194797934s) [noise=true , address=145.79.227.26:4656 , piece_id=DDNFTYMUJMFHTZXFHJTFUURSVOJBQX2KVT6UWLBCZ5TIOVBDVOVQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.598566162 (298h54m23.195337186s) [noise=true , address=145.79.227.50:4376 , piece_id=BHABIBKA3ZENQUNU2OVBVUJZVWEQRZD435EHA3MEB25KMNBVFQIA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.598781105 (298h54m23.19555213s) [noise=true , address=77.90.31.16:4737 , piece_id=SMQOC6BKSIEIE5FXEYZGOIWRZ6GORC7GCOVEOFZXPQX4FGILVIPA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.600260475 (298h54m23.197031509s) [noise=true , address=145.79.227.6:4261 , piece_id=CYOWHNR7X3Y4YE5SFHPNDBOU5QQ6M6DZIRAO3VX56KQBL3OP2FLQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.600344398 (298h54m23.197115411s) [noise=true , address=145.79.227.4:4190 , piece_id=LDID6UGFJ4GWYK635S5OE6334DEULOPVSOLOCLYQCBOFTXRZEKRQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.602467676 (298h54m23.19923871s) [noise=true , address=145.79.227.2:4116 , piece_id=KEKK3M6M37G7Y7HXPULI2RSX5CFGUNCM3LILYNQOZ2PASM742ZGQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.602977247 (298h54m23.199748272s) [noise=true , address=145.79.227.48:4307 , piece_id=D6WSRVCL6UO4KQ542EKZWA6IP7FLA3ZUGSBXCSZUMLPFCJ5G5CQA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.603520888 (298h54m23.200291919s) [noise=true , address=79.127.219.45:4855 , piece_id=644Z2MAXJ423ZF4JGZ2YLPM6XJRZT7IUJCZRRE7ZJMHB3EW7KB4Q, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.603907549 (298h54m23.200678573s) [noise=true , address=145.79.227.36:4518 , piece_id=R5PDKAATCPFI3LOOUX6HJ4OP64NQ72N3RAOZS3GYFH5VSWPFU6OQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.604438080 (298h54m23.201209113s) [noise=true , address=45.91.116.11:4118 , piece_id=CVKN5WPP2OTRAHYWJKDLCJV3OZ3I6G5CLYXTSEKJDQD7LRTJ55EA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.618924566 (298h54m23.21569559s) [noise=true , address=79.127.219.46:4892 , piece_id=533AWUNWA66VX4AZF7Y3AANCF65GMESWZD3IJWBC5IOKB4QMVTZA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.619077190 (298h54m23.215848214s) [seg_pos={0 0}, upload=334781]: Satisfied cohort requirements after 13 pieces Mar 18 21:54:25.619546929 (298h54m23.216317961s) [noise=true , address=145.79.227.46:4604 , piece_id=DAGTJ24IRMED5FNGQHRYTXEIXYTDVSRW2IT23MC2UC45SES3PE7Q, seg_pos={0 0}, upload=334781]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy): protocol: expected piece hash; context canceled; EOF Mar 18 21:54:25.619578476 (298h54m23.21634951s) [noise=true , address=145.79.227.46:4604 , piece_id=DAGTJ24IRMED5FNGQHRYTXEIXYTDVSRW2IT23MC2UC45SES3PE7Q, seg_pos={0 0}, upload=334781]: Long tail cancellation due to optimal threshold Mar 18 21:54:25.620166659 (298h54m23.216937744s) [noise=true , address=145.79.227.19:4080 , piece_id=ANOMNO3HLII5GPWIT2RDX6P7WOBTVJXMOJM5PAPDMK7ST2NYJWMA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KK51wudKiZb39Ku7wZhgBYEYmgsZTHfjiNNLmpBU9uuNFapt3): protocol: expected piece hash; context canceled; EOF Mar 18 21:54:25.620178720 (298h54m23.216949746s) [noise=true , address=145.79.227.19:4080 , piece_id=ANOMNO3HLII5GPWIT2RDX6P7WOBTVJXMOJM5PAPDMK7ST2NYJWMA, seg_pos={0 0}, upload=334781]: Long tail cancellation due to optimal threshold Mar 18 21:54:25.620702987 (298h54m23.217474021s) [noise=true , address=145.79.227.37:4569 , piece_id=PTE4YT3YJ5M3NDKXKQOUFTGQ3T5FDAUZB74QHJBVH6HQL53QXLUA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:18TqoejkuHLqZWhEPe3QPH1weVXnWJhaeoi8BGkvjCTLW6t4qT): protocol: expected piece hash; context canceled; EOF Mar 18 21:54:25.620718441 (298h54m23.217489474s) [noise=true , address=145.79.227.37:4569 , piece_id=PTE4YT3YJ5M3NDKXKQOUFTGQ3T5FDAUZB74QHJBVH6HQL53QXLUA, seg_pos={0 0}, upload=334781]: Long tail cancellation due to optimal threshold Mar 18 21:54:25.620748329 (298h54m23.21751937s) [seg_pos={0 0}, upload=334781]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Mar 18 21:54:25.620912612 (298h54m23.21768364s) [upload=334781]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 18 21:54:25.620920854 (298h54m23.217691883s) [upload=334781]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 21:54:38.136688455 (298h54m35.733459485s) [upload=334781]: Done uploading segments... Mar 18 21:54:38.136713048 (298h54m35.73348408s) [upload=334781]: Upload finished. err: <nil> Duration: 458.797735ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 7 out of 60 is failed (threshold=3) [XX.X..X..X.XX...............................................] duration: 256.139054ms trace: https://jaeger.dev.storj.io/trace/7c4371cfa8f7834c FAIL (2026-03-18 22:31:13 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 256.175665ms core uplink us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 11 out of 60 is failed (threshold=6) [XXXXXXXXXXX.................................................] duration: 538.972071ms trace: https://jaeger.dev.storj.io/trace/6cabdd7d47c9cb44 FAIL (2026-03-18 22:31:11 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 539.009574ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 6 out of 60 is failed (threshold=3) [X.X...XX.X.X................................................] duration: 242.936144ms trace: https://jaeger.dev.storj.io/trace/47004c0815b844e7 FAIL (2026-03-18 22:31:02 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 18 21:53:36.090826124 (298h53m33.687597155s) [upload=334767]: Starting upload Mar 18 21:53:36.090977797 (298h53m33.687748822s) [upload=334767]: Done starting upload Mar 18 21:53:36.091078093 (298h53m33.687849119s) [upload=334767]: Uploading segments... Mar 18 21:53:36.091268776 (298h53m33.688040083s) [upload=334767]: Got next segment. Inline: true Mar 18 21:53:36.091279074 (298h53m33.688050103s) [upload=334767]: Waiting for error group managing segments... Mar 18 21:53:36.091372978 (298h53m33.688144001s) [upload=334767]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 21:53:36.091379985 (298h53m33.688151008s) [upload=334767]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Mar 18 21:53:36.091384265 (298h53m33.688155287s) [upload=334767]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 21:54:04.718146694 (298h54m2.314917717s) [upload=334767]: Done uploading segments... Mar 18 21:54:04.718169125 (298h54m2.314940155s) [upload=334767]: Upload finished. err: <nil> Duration:* 242.993195ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 12 out of 60 is failed (threshold=6) [XXXXXXXXXXXX................................................] duration: 537.150344ms trace: https://jaeger.dev.storj.io/trace/6cabdd7d47c9cb44 FAIL (2026-03-18 22:30:54 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 537.178725ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 6 out of 60 is failed (threshold=3) [XX.X..XXX...................................................] duration: 1.733513438s trace: https://jaeger.dev.storj.io/trace/27ed2876d5f3e85d FAIL (2026-03-18 22:30:51 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 1.733548784s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 7 out of 60 is failed (threshold=3) [.XX.X..X..X.XX..............................................] duration: 260.706267ms trace: https://jaeger.dev.storj.io/trace/7c4371cfa8f7834c FAIL (2026-03-18 22:30:25 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 260.761086ms core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 7 out of 60 is failed (threshold=3) [XX.X...XX.X.X...............................................] duration: 277.908874ms trace: https://jaeger.dev.storj.io/trace/47004c0815b844e7 FAIL (2026-03-18 22:30:16 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 18 21:53:36.090826124 (298h53m33.687597155s) [upload=334767]: Starting upload Mar 18 21:53:36.090977797 (298h53m33.687748822s) [upload=334767]: Done starting upload Mar 18 21:53:36.091078093 (298h53m33.687849119s) [upload=334767]: Uploading segments... Mar 18 21:53:36.091268776 (298h53m33.688040083s) [upload=334767]: Got next segment. Inline: true Mar 18 21:53:36.091279074 (298h53m33.688050103s) [upload=334767]: Waiting for error group managing segments... Mar 18 21:53:36.091372978 (298h53m33.688144001s) [upload=334767]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 21:53:36.091379985 (298h53m33.688151008s) [upload=334767]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Mar 18 21:53:36.091384265 (298h53m33.688155287s) [upload=334767]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 21:54:04.718146694 (298h54m2.314917717s) [upload=334767]: Done uploading segments... Mar 18 21:54:04.718169125 (298h54m2.314940155s) [upload=334767]: Upload finished. err: <nil> Duration:* 277.983964ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [....XXXX.XXX................................................] duration: 463.240321ms trace: https://jaeger.dev.storj.io/trace/6dceff78c86f0774 FAIL (2026-03-18 22:30:10 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 18 21:54:17.190275520 (298h54m14.787046534s) [upload=334781]: Starting upload Mar 18 21:54:17.190367052 (298h54m14.787138068s) [upload=334781]: Done starting upload Mar 18 21:54:17.190448332 (298h54m14.787219348s) [upload=334781]: Uploading segments... Mar 18 21:54:17.190705184 (298h54m14.787476207s) [upload=334781]: Got next segment. Inline: false Mar 18 21:54:17.190783980 (298h54m14.787555002s) [upload=334781]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 21:54:17.190804859 (298h54m14.787575882s) [upload=334781]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 18 21:54:25.452304457 (298h54m23.049075487s) [seg_pos={0 0}, upload=334781]: Begin upload segment... Mar 18 21:54:25.452374690 (298h54m23.049145713s) [seg_pos={0 0}, upload=334781]: Done begin upload segment. Mar 18 21:54:25.452463335 (298h54m23.049234366s) [noise=true , address=145.79.227.46:4604 , piece_id=DAGTJ24IRMED5FNGQHRYTXEIXYTDVSRW2IT23MC2UC45SES3PE7Q, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.452479705 (298h54m23.049250739s) [noise=true , address=45.91.116.9:4264 , piece_id=EE3TXU5KSPHKMNNCI3BHIGFKQNYZQRBDO4YT3TTR5U66YJMFT6XQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.453224160 (298h54m23.049995188s) [noise=true , address=79.127.219.46:4892 , piece_id=533AWUNWA66VX4AZF7Y3AANCF65GMESWZD3IJWBC5IOKB4QMVTZA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.454147610 (298h54m23.050918643s) [noise=true , address=145.79.227.48:4307 , piece_id=D6WSRVCL6UO4KQ542EKZWA6IP7FLA3ZUGSBXCSZUMLPFCJ5G5CQA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.454790080 (298h54m23.051561121s) [noise=true , address=145.79.227.2:4116 , piece_id=KEKK3M6M37G7Y7HXPULI2RSX5CFGUNCM3LILYNQOZ2PASM742ZGQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.455678920 (298h54m23.052449952s) [noise=true , address=145.79.227.19:4080 , piece_id=ANOMNO3HLII5GPWIT2RDX6P7WOBTVJXMOJM5PAPDMK7ST2NYJWMA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.456387102 (298h54m23.053158136s) [noise=true , address=145.79.227.37:4569 , piece_id=PTE4YT3YJ5M3NDKXKQOUFTGQ3T5FDAUZB74QHJBVH6HQL53QXLUA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.456987483 (298h54m23.053758515s) [noise=true , address=145.79.227.4:4190 , piece_id=LDID6UGFJ4GWYK635S5OE6334DEULOPVSOLOCLYQCBOFTXRZEKRQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.457980795 (298h54m23.054751829s) [noise=true , address=79.127.219.45:4855 , piece_id=644Z2MAXJ423ZF4JGZ2YLPM6XJRZT7IUJCZRRE7ZJMHB3EW7KB4Q, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.459806473 (298h54m23.056577503s) [noise=true , address=145.79.227.50:4376 , piece_id=BHABIBKA3ZENQUNU2OVBVUJZVWEQRZD435EHA3MEB25KMNBVFQIA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.460305248 (298h54m23.057076281s) [noise=true , address=77.90.31.16:4737 , piece_id=SMQOC6BKSIEIE5FXEYZGOIWRZ6GORC7GCOVEOFZXPQX4FGILVIPA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.461552822 (298h54m23.058323848s) [noise=true , address=79.127.219.41:4926 , piece_id=AVSCKJ5WDK4OBWWUSLUQNOPZI7BGTTDGT4XJYSLQAJPUHRYY4KVQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.463144575 (298h54m23.059915608s) [noise=true , address=145.79.227.36:4518 , piece_id=R5PDKAATCPFI3LOOUX6HJ4OP64NQ72N3RAOZS3GYFH5VSWPFU6OQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.464837828 (298h54m23.06160886s) [noise=true , address=145.79.227.26:4656 , piece_id=DDNFTYMUJMFHTZXFHJTFUURSVOJBQX2KVT6UWLBCZ5TIOVBDVOVQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.466411504 (298h54m23.063182537s) [noise=true , address=145.79.227.6:4261 , piece_id=CYOWHNR7X3Y4YE5SFHPNDBOU5QQ6M6DZIRAO3VX56KQBL3OP2FLQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.469091278 (298h54m23.065862311s) [noise=true , address=45.91.116.11:4118 , piece_id=CVKN5WPP2OTRAHYWJKDLCJV3OZ3I6G5CLYXTSEKJDQD7LRTJ55EA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.586976347 (298h54m23.183747361s) [upload=334781]: Next returned nil segment Mar 18 21:54:25.587000505 (298h54m23.183771534s) [upload=334781]: Waiting for error group managing segments... Mar 18 21:54:25.596319798 (298h54m23.193090825s) [noise=true , address=79.127.219.41:4926 , piece_id=AVSCKJ5WDK4OBWWUSLUQNOPZI7BGTTDGT4XJYSLQAJPUHRYY4KVQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.596985876 (298h54m23.1937569s) [noise=true , address=45.91.116.9:4264 , piece_id=EE3TXU5KSPHKMNNCI3BHIGFKQNYZQRBDO4YT3TTR5U66YJMFT6XQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.598026901 (298h54m23.194797934s) [noise=true , address=145.79.227.26:4656 , piece_id=DDNFTYMUJMFHTZXFHJTFUURSVOJBQX2KVT6UWLBCZ5TIOVBDVOVQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.598566162 (298h54m23.195337186s) [noise=true , address=145.79.227.50:4376 , piece_id=BHABIBKA3ZENQUNU2OVBVUJZVWEQRZD435EHA3MEB25KMNBVFQIA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.598781105 (298h54m23.19555213s) [noise=true , address=77.90.31.16:4737 , piece_id=SMQOC6BKSIEIE5FXEYZGOIWRZ6GORC7GCOVEOFZXPQX4FGILVIPA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.600260475 (298h54m23.197031509s) [noise=true , address=145.79.227.6:4261 , piece_id=CYOWHNR7X3Y4YE5SFHPNDBOU5QQ6M6DZIRAO3VX56KQBL3OP2FLQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.600344398 (298h54m23.197115411s) [noise=true , address=145.79.227.4:4190 , piece_id=LDID6UGFJ4GWYK635S5OE6334DEULOPVSOLOCLYQCBOFTXRZEKRQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.602467676 (298h54m23.19923871s) [noise=true , address=145.79.227.2:4116 , piece_id=KEKK3M6M37G7Y7HXPULI2RSX5CFGUNCM3LILYNQOZ2PASM742ZGQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.602977247 (298h54m23.199748272s) [noise=true , address=145.79.227.48:4307 , piece_id=D6WSRVCL6UO4KQ542EKZWA6IP7FLA3ZUGSBXCSZUMLPFCJ5G5CQA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.603520888 (298h54m23.200291919s) [noise=true , address=79.127.219.45:4855 , piece_id=644Z2MAXJ423ZF4JGZ2YLPM6XJRZT7IUJCZRRE7ZJMHB3EW7KB4Q, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.603907549 (298h54m23.200678573s) [noise=true , address=145.79.227.36:4518 , piece_id=R5PDKAATCPFI3LOOUX6HJ4OP64NQ72N3RAOZS3GYFH5VSWPFU6OQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.604438080 (298h54m23.201209113s) [noise=true , address=45.91.116.11:4118 , piece_id=CVKN5WPP2OTRAHYWJKDLCJV3OZ3I6G5CLYXTSEKJDQD7LRTJ55EA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.618924566 (298h54m23.21569559s) [noise=true , address=79.127.219.46:4892 , piece_id=533AWUNWA66VX4AZF7Y3AANCF65GMESWZD3IJWBC5IOKB4QMVTZA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.619077190 (298h54m23.215848214s) [seg_pos={0 0}, upload=334781]: Satisfied cohort requirements after 13 pieces Mar 18 21:54:25.619546929 (298h54m23.216317961s) [noise=true , address=145.79.227.46:4604 , piece_id=DAGTJ24IRMED5FNGQHRYTXEIXYTDVSRW2IT23MC2UC45SES3PE7Q, seg_pos={0 0}, upload=334781]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy): protocol: expected piece hash; context canceled; EOF Mar 18 21:54:25.619578476 (298h54m23.21634951s) [noise=true , address=145.79.227.46:4604 , piece_id=DAGTJ24IRMED5FNGQHRYTXEIXYTDVSRW2IT23MC2UC45SES3PE7Q, seg_pos={0 0}, upload=334781]: Long tail cancellation due to optimal threshold Mar 18 21:54:25.620166659 (298h54m23.216937744s) [noise=true , address=145.79.227.19:4080 , piece_id=ANOMNO3HLII5GPWIT2RDX6P7WOBTVJXMOJM5PAPDMK7ST2NYJWMA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KK51wudKiZb39Ku7wZhgBYEYmgsZTHfjiNNLmpBU9uuNFapt3): protocol: expected piece hash; context canceled; EOF Mar 18 21:54:25.620178720 (298h54m23.216949746s) [noise=true , address=145.79.227.19:4080 , piece_id=ANOMNO3HLII5GPWIT2RDX6P7WOBTVJXMOJM5PAPDMK7ST2NYJWMA, seg_pos={0 0}, upload=334781]: Long tail cancellation due to optimal threshold Mar 18 21:54:25.620702987 (298h54m23.217474021s) [noise=true , address=145.79.227.37:4569 , piece_id=PTE4YT3YJ5M3NDKXKQOUFTGQ3T5FDAUZB74QHJBVH6HQL53QXLUA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:18TqoejkuHLqZWhEPe3QPH1weVXnWJhaeoi8BGkvjCTLW6t4qT): protocol: expected piece hash; context canceled; EOF Mar 18 21:54:25.620718441 (298h54m23.217489474s) [noise=true , address=145.79.227.37:4569 , piece_id=PTE4YT3YJ5M3NDKXKQOUFTGQ3T5FDAUZB74QHJBVH6HQL53QXLUA, seg_pos={0 0}, upload=334781]: Long tail cancellation due to optimal threshold Mar 18 21:54:25.620748329 (298h54m23.21751937s) [seg_pos={0 0}, upload=334781]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Mar 18 21:54:25.620912612 (298h54m23.21768364s) [upload=334781]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 18 21:54:25.620920854 (298h54m23.217691883s) [upload=334781]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 21:54:38.136688455 (298h54m35.733459485s) [upload=334781]: Done uploading segments... Mar 18 21:54:38.136713048 (298h54m35.73348408s) [upload=334781]: Upload finished. err: <nil> Duration: 463.270594ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 13 out of 60 is failed (threshold=6) [XXXXXXXXXXXXX...............................................] duration: 637.655111ms trace: https://jaeger.dev.storj.io/trace/6cabdd7d47c9cb44 FAIL (2026-03-18 22:30:05 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 638.356782ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 7 out of 60 is failed (threshold=3) [..XX.X..X..X.XX.............................................] duration: 321.832285ms trace: https://jaeger.dev.storj.io/trace/7c4371cfa8f7834c FAIL (2026-03-18 22:30:00 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 321.863015ms core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [.....XXXX.XXX...............................................] duration: 484.241964ms trace: https://jaeger.dev.storj.io/trace/6dceff78c86f0774 FAIL (2026-03-18 22:29:54 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 18 21:54:17.190275520 (298h54m14.787046534s) [upload=334781]: Starting upload Mar 18 21:54:17.190367052 (298h54m14.787138068s) [upload=334781]: Done starting upload Mar 18 21:54:17.190448332 (298h54m14.787219348s) [upload=334781]: Uploading segments... Mar 18 21:54:17.190705184 (298h54m14.787476207s) [upload=334781]: Got next segment. Inline: false Mar 18 21:54:17.190783980 (298h54m14.787555002s) [upload=334781]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 21:54:17.190804859 (298h54m14.787575882s) [upload=334781]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 18 21:54:25.452304457 (298h54m23.049075487s) [seg_pos={0 0}, upload=334781]: Begin upload segment... Mar 18 21:54:25.452374690 (298h54m23.049145713s) [seg_pos={0 0}, upload=334781]: Done begin upload segment. Mar 18 21:54:25.452463335 (298h54m23.049234366s) [noise=true , address=145.79.227.46:4604 , piece_id=DAGTJ24IRMED5FNGQHRYTXEIXYTDVSRW2IT23MC2UC45SES3PE7Q, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.452479705 (298h54m23.049250739s) [noise=true , address=45.91.116.9:4264 , piece_id=EE3TXU5KSPHKMNNCI3BHIGFKQNYZQRBDO4YT3TTR5U66YJMFT6XQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.453224160 (298h54m23.049995188s) [noise=true , address=79.127.219.46:4892 , piece_id=533AWUNWA66VX4AZF7Y3AANCF65GMESWZD3IJWBC5IOKB4QMVTZA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.454147610 (298h54m23.050918643s) [noise=true , address=145.79.227.48:4307 , piece_id=D6WSRVCL6UO4KQ542EKZWA6IP7FLA3ZUGSBXCSZUMLPFCJ5G5CQA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.454790080 (298h54m23.051561121s) [noise=true , address=145.79.227.2:4116 , piece_id=KEKK3M6M37G7Y7HXPULI2RSX5CFGUNCM3LILYNQOZ2PASM742ZGQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.455678920 (298h54m23.052449952s) [noise=true , address=145.79.227.19:4080 , piece_id=ANOMNO3HLII5GPWIT2RDX6P7WOBTVJXMOJM5PAPDMK7ST2NYJWMA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.456387102 (298h54m23.053158136s) [noise=true , address=145.79.227.37:4569 , piece_id=PTE4YT3YJ5M3NDKXKQOUFTGQ3T5FDAUZB74QHJBVH6HQL53QXLUA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.456987483 (298h54m23.053758515s) [noise=true , address=145.79.227.4:4190 , piece_id=LDID6UGFJ4GWYK635S5OE6334DEULOPVSOLOCLYQCBOFTXRZEKRQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.457980795 (298h54m23.054751829s) [noise=true , address=79.127.219.45:4855 , piece_id=644Z2MAXJ423ZF4JGZ2YLPM6XJRZT7IUJCZRRE7ZJMHB3EW7KB4Q, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.459806473 (298h54m23.056577503s) [noise=true , address=145.79.227.50:4376 , piece_id=BHABIBKA3ZENQUNU2OVBVUJZVWEQRZD435EHA3MEB25KMNBVFQIA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.460305248 (298h54m23.057076281s) [noise=true , address=77.90.31.16:4737 , piece_id=SMQOC6BKSIEIE5FXEYZGOIWRZ6GORC7GCOVEOFZXPQX4FGILVIPA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.461552822 (298h54m23.058323848s) [noise=true , address=79.127.219.41:4926 , piece_id=AVSCKJ5WDK4OBWWUSLUQNOPZI7BGTTDGT4XJYSLQAJPUHRYY4KVQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.463144575 (298h54m23.059915608s) [noise=true , address=145.79.227.36:4518 , piece_id=R5PDKAATCPFI3LOOUX6HJ4OP64NQ72N3RAOZS3GYFH5VSWPFU6OQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.464837828 (298h54m23.06160886s) [noise=true , address=145.79.227.26:4656 , piece_id=DDNFTYMUJMFHTZXFHJTFUURSVOJBQX2KVT6UWLBCZ5TIOVBDVOVQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.466411504 (298h54m23.063182537s) [noise=true , address=145.79.227.6:4261 , piece_id=CYOWHNR7X3Y4YE5SFHPNDBOU5QQ6M6DZIRAO3VX56KQBL3OP2FLQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.469091278 (298h54m23.065862311s) [noise=true , address=45.91.116.11:4118 , piece_id=CVKN5WPP2OTRAHYWJKDLCJV3OZ3I6G5CLYXTSEKJDQD7LRTJ55EA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.586976347 (298h54m23.183747361s) [upload=334781]: Next returned nil segment Mar 18 21:54:25.587000505 (298h54m23.183771534s) [upload=334781]: Waiting for error group managing segments... Mar 18 21:54:25.596319798 (298h54m23.193090825s) [noise=true , address=79.127.219.41:4926 , piece_id=AVSCKJ5WDK4OBWWUSLUQNOPZI7BGTTDGT4XJYSLQAJPUHRYY4KVQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.596985876 (298h54m23.1937569s) [noise=true , address=45.91.116.9:4264 , piece_id=EE3TXU5KSPHKMNNCI3BHIGFKQNYZQRBDO4YT3TTR5U66YJMFT6XQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.598026901 (298h54m23.194797934s) [noise=true , address=145.79.227.26:4656 , piece_id=DDNFTYMUJMFHTZXFHJTFUURSVOJBQX2KVT6UWLBCZ5TIOVBDVOVQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.598566162 (298h54m23.195337186s) [noise=true , address=145.79.227.50:4376 , piece_id=BHABIBKA3ZENQUNU2OVBVUJZVWEQRZD435EHA3MEB25KMNBVFQIA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.598781105 (298h54m23.19555213s) [noise=true , address=77.90.31.16:4737 , piece_id=SMQOC6BKSIEIE5FXEYZGOIWRZ6GORC7GCOVEOFZXPQX4FGILVIPA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.600260475 (298h54m23.197031509s) [noise=true , address=145.79.227.6:4261 , piece_id=CYOWHNR7X3Y4YE5SFHPNDBOU5QQ6M6DZIRAO3VX56KQBL3OP2FLQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.600344398 (298h54m23.197115411s) [noise=true , address=145.79.227.4:4190 , piece_id=LDID6UGFJ4GWYK635S5OE6334DEULOPVSOLOCLYQCBOFTXRZEKRQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.602467676 (298h54m23.19923871s) [noise=true , address=145.79.227.2:4116 , piece_id=KEKK3M6M37G7Y7HXPULI2RSX5CFGUNCM3LILYNQOZ2PASM742ZGQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.602977247 (298h54m23.199748272s) [noise=true , address=145.79.227.48:4307 , piece_id=D6WSRVCL6UO4KQ542EKZWA6IP7FLA3ZUGSBXCSZUMLPFCJ5G5CQA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.603520888 (298h54m23.200291919s) [noise=true , address=79.127.219.45:4855 , piece_id=644Z2MAXJ423ZF4JGZ2YLPM6XJRZT7IUJCZRRE7ZJMHB3EW7KB4Q, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.603907549 (298h54m23.200678573s) [noise=true , address=145.79.227.36:4518 , piece_id=R5PDKAATCPFI3LOOUX6HJ4OP64NQ72N3RAOZS3GYFH5VSWPFU6OQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.604438080 (298h54m23.201209113s) [noise=true , address=45.91.116.11:4118 , piece_id=CVKN5WPP2OTRAHYWJKDLCJV3OZ3I6G5CLYXTSEKJDQD7LRTJ55EA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.618924566 (298h54m23.21569559s) [noise=true , address=79.127.219.46:4892 , piece_id=533AWUNWA66VX4AZF7Y3AANCF65GMESWZD3IJWBC5IOKB4QMVTZA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.619077190 (298h54m23.215848214s) [seg_pos={0 0}, upload=334781]: Satisfied cohort requirements after 13 pieces Mar 18 21:54:25.619546929 (298h54m23.216317961s) [noise=true , address=145.79.227.46:4604 , piece_id=DAGTJ24IRMED5FNGQHRYTXEIXYTDVSRW2IT23MC2UC45SES3PE7Q, seg_pos={0 0}, upload=334781]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy): protocol: expected piece hash; context canceled; EOF Mar 18 21:54:25.619578476 (298h54m23.21634951s) [noise=true , address=145.79.227.46:4604 , piece_id=DAGTJ24IRMED5FNGQHRYTXEIXYTDVSRW2IT23MC2UC45SES3PE7Q, seg_pos={0 0}, upload=334781]: Long tail cancellation due to optimal threshold Mar 18 21:54:25.620166659 (298h54m23.216937744s) [noise=true , address=145.79.227.19:4080 , piece_id=ANOMNO3HLII5GPWIT2RDX6P7WOBTVJXMOJM5PAPDMK7ST2NYJWMA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KK51wudKiZb39Ku7wZhgBYEYmgsZTHfjiNNLmpBU9uuNFapt3): protocol: expected piece hash; context canceled; EOF Mar 18 21:54:25.620178720 (298h54m23.216949746s) [noise=true , address=145.79.227.19:4080 , piece_id=ANOMNO3HLII5GPWIT2RDX6P7WOBTVJXMOJM5PAPDMK7ST2NYJWMA, seg_pos={0 0}, upload=334781]: Long tail cancellation due to optimal threshold Mar 18 21:54:25.620702987 (298h54m23.217474021s) [noise=true , address=145.79.227.37:4569 , piece_id=PTE4YT3YJ5M3NDKXKQOUFTGQ3T5FDAUZB74QHJBVH6HQL53QXLUA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:18TqoejkuHLqZWhEPe3QPH1weVXnWJhaeoi8BGkvjCTLW6t4qT): protocol: expected piece hash; context canceled; EOF Mar 18 21:54:25.620718441 (298h54m23.217489474s) [noise=true , address=145.79.227.37:4569 , piece_id=PTE4YT3YJ5M3NDKXKQOUFTGQ3T5FDAUZB74QHJBVH6HQL53QXLUA, seg_pos={0 0}, upload=334781]: Long tail cancellation due to optimal threshold Mar 18 21:54:25.620748329 (298h54m23.21751937s) [seg_pos={0 0}, upload=334781]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Mar 18 21:54:25.620912612 (298h54m23.21768364s) [upload=334781]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 18 21:54:25.620920854 (298h54m23.217691883s) [upload=334781]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 21:54:38.136688455 (298h54m35.733459485s) [upload=334781]: Done uploading segments... Mar 18 21:54:38.136713048 (298h54m35.73348408s) [upload=334781]: Upload finished. err: <nil> Duration: 484.271276ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 14 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXX..............................................] duration: 1.055700003s trace: https://jaeger.dev.storj.io/trace/6cabdd7d47c9cb44 FAIL (2026-03-18 22:29:48 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.055735123s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 7 out of 60 is failed (threshold=3) [XXX.X..XXX..................................................] duration: 1.752735496s trace: https://jaeger.dev.storj.io/trace/27ed2876d5f3e85d FAIL (2026-03-18 22:29:39 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 1.752779788s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 8 out of 60 is failed (threshold=3) [XXX.X...XX.X.X..............................................] duration: 247.841649ms trace: https://jaeger.dev.storj.io/trace/47004c0815b844e7 FAIL (2026-03-18 22:29:34 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 18 21:53:36.090826124 (298h53m33.687597155s) [upload=334767]: Starting upload Mar 18 21:53:36.090977797 (298h53m33.687748822s) [upload=334767]: Done starting upload Mar 18 21:53:36.091078093 (298h53m33.687849119s) [upload=334767]: Uploading segments... Mar 18 21:53:36.091268776 (298h53m33.688040083s) [upload=334767]: Got next segment. Inline: true Mar 18 21:53:36.091279074 (298h53m33.688050103s) [upload=334767]: Waiting for error group managing segments... Mar 18 21:53:36.091372978 (298h53m33.688144001s) [upload=334767]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 21:53:36.091379985 (298h53m33.688151008s) [upload=334767]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Mar 18 21:53:36.091384265 (298h53m33.688155287s) [upload=334767]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 21:54:04.718146694 (298h54m2.314917717s) [upload=334767]: Done uploading segments... Mar 18 21:54:04.718169125 (298h54m2.314940155s) [upload=334767]: Upload finished. err: <nil> Duration:* 247.867751ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 8 out of 60 is failed (threshold=3) [XXXX.X..XXX.................................................] duration: 1.808046108s trace: https://jaeger.dev.storj.io/trace/27ed2876d5f3e85d FAIL (2026-03-18 22:29:22 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 1.808099617s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 8 out of 60 is failed (threshold=3) [X..XX.X..X..X.XX............................................] duration: 447.648808ms trace: https://jaeger.dev.storj.io/trace/7c4371cfa8f7834c FAIL (2026-03-18 22:29:14 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 447.695432ms core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 9 out of 60 is failed (threshold=3) [XXXXX.X..XXX................................................] duration: 1.786861864s trace: https://jaeger.dev.storj.io/trace/27ed2876d5f3e85d FAIL (2026-03-18 22:28:55 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 1.786892654s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [X.....XXXX.XXX..............................................] duration: 483.840871ms trace: https://jaeger.dev.storj.io/trace/6dceff78c86f0774 FAIL (2026-03-18 22:28:52 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 18 21:54:17.190275520 (298h54m14.787046534s) [upload=334781]: Starting upload Mar 18 21:54:17.190367052 (298h54m14.787138068s) [upload=334781]: Done starting upload Mar 18 21:54:17.190448332 (298h54m14.787219348s) [upload=334781]: Uploading segments... Mar 18 21:54:17.190705184 (298h54m14.787476207s) [upload=334781]: Got next segment. Inline: false Mar 18 21:54:17.190783980 (298h54m14.787555002s) [upload=334781]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 21:54:17.190804859 (298h54m14.787575882s) [upload=334781]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 18 21:54:25.452304457 (298h54m23.049075487s) [seg_pos={0 0}, upload=334781]: Begin upload segment... Mar 18 21:54:25.452374690 (298h54m23.049145713s) [seg_pos={0 0}, upload=334781]: Done begin upload segment. Mar 18 21:54:25.452463335 (298h54m23.049234366s) [noise=true , address=145.79.227.46:4604 , piece_id=DAGTJ24IRMED5FNGQHRYTXEIXYTDVSRW2IT23MC2UC45SES3PE7Q, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.452479705 (298h54m23.049250739s) [noise=true , address=45.91.116.9:4264 , piece_id=EE3TXU5KSPHKMNNCI3BHIGFKQNYZQRBDO4YT3TTR5U66YJMFT6XQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.453224160 (298h54m23.049995188s) [noise=true , address=79.127.219.46:4892 , piece_id=533AWUNWA66VX4AZF7Y3AANCF65GMESWZD3IJWBC5IOKB4QMVTZA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.454147610 (298h54m23.050918643s) [noise=true , address=145.79.227.48:4307 , piece_id=D6WSRVCL6UO4KQ542EKZWA6IP7FLA3ZUGSBXCSZUMLPFCJ5G5CQA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.454790080 (298h54m23.051561121s) [noise=true , address=145.79.227.2:4116 , piece_id=KEKK3M6M37G7Y7HXPULI2RSX5CFGUNCM3LILYNQOZ2PASM742ZGQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.455678920 (298h54m23.052449952s) [noise=true , address=145.79.227.19:4080 , piece_id=ANOMNO3HLII5GPWIT2RDX6P7WOBTVJXMOJM5PAPDMK7ST2NYJWMA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.456387102 (298h54m23.053158136s) [noise=true , address=145.79.227.37:4569 , piece_id=PTE4YT3YJ5M3NDKXKQOUFTGQ3T5FDAUZB74QHJBVH6HQL53QXLUA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.456987483 (298h54m23.053758515s) [noise=true , address=145.79.227.4:4190 , piece_id=LDID6UGFJ4GWYK635S5OE6334DEULOPVSOLOCLYQCBOFTXRZEKRQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.457980795 (298h54m23.054751829s) [noise=true , address=79.127.219.45:4855 , piece_id=644Z2MAXJ423ZF4JGZ2YLPM6XJRZT7IUJCZRRE7ZJMHB3EW7KB4Q, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.459806473 (298h54m23.056577503s) [noise=true , address=145.79.227.50:4376 , piece_id=BHABIBKA3ZENQUNU2OVBVUJZVWEQRZD435EHA3MEB25KMNBVFQIA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.460305248 (298h54m23.057076281s) [noise=true , address=77.90.31.16:4737 , piece_id=SMQOC6BKSIEIE5FXEYZGOIWRZ6GORC7GCOVEOFZXPQX4FGILVIPA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.461552822 (298h54m23.058323848s) [noise=true , address=79.127.219.41:4926 , piece_id=AVSCKJ5WDK4OBWWUSLUQNOPZI7BGTTDGT4XJYSLQAJPUHRYY4KVQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.463144575 (298h54m23.059915608s) [noise=true , address=145.79.227.36:4518 , piece_id=R5PDKAATCPFI3LOOUX6HJ4OP64NQ72N3RAOZS3GYFH5VSWPFU6OQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.464837828 (298h54m23.06160886s) [noise=true , address=145.79.227.26:4656 , piece_id=DDNFTYMUJMFHTZXFHJTFUURSVOJBQX2KVT6UWLBCZ5TIOVBDVOVQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.466411504 (298h54m23.063182537s) [noise=true , address=145.79.227.6:4261 , piece_id=CYOWHNR7X3Y4YE5SFHPNDBOU5QQ6M6DZIRAO3VX56KQBL3OP2FLQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.469091278 (298h54m23.065862311s) [noise=true , address=45.91.116.11:4118 , piece_id=CVKN5WPP2OTRAHYWJKDLCJV3OZ3I6G5CLYXTSEKJDQD7LRTJ55EA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.586976347 (298h54m23.183747361s) [upload=334781]: Next returned nil segment Mar 18 21:54:25.587000505 (298h54m23.183771534s) [upload=334781]: Waiting for error group managing segments... Mar 18 21:54:25.596319798 (298h54m23.193090825s) [noise=true , address=79.127.219.41:4926 , piece_id=AVSCKJ5WDK4OBWWUSLUQNOPZI7BGTTDGT4XJYSLQAJPUHRYY4KVQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.596985876 (298h54m23.1937569s) [noise=true , address=45.91.116.9:4264 , piece_id=EE3TXU5KSPHKMNNCI3BHIGFKQNYZQRBDO4YT3TTR5U66YJMFT6XQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.598026901 (298h54m23.194797934s) [noise=true , address=145.79.227.26:4656 , piece_id=DDNFTYMUJMFHTZXFHJTFUURSVOJBQX2KVT6UWLBCZ5TIOVBDVOVQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.598566162 (298h54m23.195337186s) [noise=true , address=145.79.227.50:4376 , piece_id=BHABIBKA3ZENQUNU2OVBVUJZVWEQRZD435EHA3MEB25KMNBVFQIA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.598781105 (298h54m23.19555213s) [noise=true , address=77.90.31.16:4737 , piece_id=SMQOC6BKSIEIE5FXEYZGOIWRZ6GORC7GCOVEOFZXPQX4FGILVIPA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.600260475 (298h54m23.197031509s) [noise=true , address=145.79.227.6:4261 , piece_id=CYOWHNR7X3Y4YE5SFHPNDBOU5QQ6M6DZIRAO3VX56KQBL3OP2FLQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.600344398 (298h54m23.197115411s) [noise=true , address=145.79.227.4:4190 , piece_id=LDID6UGFJ4GWYK635S5OE6334DEULOPVSOLOCLYQCBOFTXRZEKRQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.602467676 (298h54m23.19923871s) [noise=true , address=145.79.227.2:4116 , piece_id=KEKK3M6M37G7Y7HXPULI2RSX5CFGUNCM3LILYNQOZ2PASM742ZGQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.602977247 (298h54m23.199748272s) [noise=true , address=145.79.227.48:4307 , piece_id=D6WSRVCL6UO4KQ542EKZWA6IP7FLA3ZUGSBXCSZUMLPFCJ5G5CQA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.603520888 (298h54m23.200291919s) [noise=true , address=79.127.219.45:4855 , piece_id=644Z2MAXJ423ZF4JGZ2YLPM6XJRZT7IUJCZRRE7ZJMHB3EW7KB4Q, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.603907549 (298h54m23.200678573s) [noise=true , address=145.79.227.36:4518 , piece_id=R5PDKAATCPFI3LOOUX6HJ4OP64NQ72N3RAOZS3GYFH5VSWPFU6OQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.604438080 (298h54m23.201209113s) [noise=true , address=45.91.116.11:4118 , piece_id=CVKN5WPP2OTRAHYWJKDLCJV3OZ3I6G5CLYXTSEKJDQD7LRTJ55EA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.618924566 (298h54m23.21569559s) [noise=true , address=79.127.219.46:4892 , piece_id=533AWUNWA66VX4AZF7Y3AANCF65GMESWZD3IJWBC5IOKB4QMVTZA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.619077190 (298h54m23.215848214s) [seg_pos={0 0}, upload=334781]: Satisfied cohort requirements after 13 pieces Mar 18 21:54:25.619546929 (298h54m23.216317961s) [noise=true , address=145.79.227.46:4604 , piece_id=DAGTJ24IRMED5FNGQHRYTXEIXYTDVSRW2IT23MC2UC45SES3PE7Q, seg_pos={0 0}, upload=334781]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy): protocol: expected piece hash; context canceled; EOF Mar 18 21:54:25.619578476 (298h54m23.21634951s) [noise=true , address=145.79.227.46:4604 , piece_id=DAGTJ24IRMED5FNGQHRYTXEIXYTDVSRW2IT23MC2UC45SES3PE7Q, seg_pos={0 0}, upload=334781]: Long tail cancellation due to optimal threshold Mar 18 21:54:25.620166659 (298h54m23.216937744s) [noise=true , address=145.79.227.19:4080 , piece_id=ANOMNO3HLII5GPWIT2RDX6P7WOBTVJXMOJM5PAPDMK7ST2NYJWMA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KK51wudKiZb39Ku7wZhgBYEYmgsZTHfjiNNLmpBU9uuNFapt3): protocol: expected piece hash; context canceled; EOF Mar 18 21:54:25.620178720 (298h54m23.216949746s) [noise=true , address=145.79.227.19:4080 , piece_id=ANOMNO3HLII5GPWIT2RDX6P7WOBTVJXMOJM5PAPDMK7ST2NYJWMA, seg_pos={0 0}, upload=334781]: Long tail cancellation due to optimal threshold Mar 18 21:54:25.620702987 (298h54m23.217474021s) [noise=true , address=145.79.227.37:4569 , piece_id=PTE4YT3YJ5M3NDKXKQOUFTGQ3T5FDAUZB74QHJBVH6HQL53QXLUA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:18TqoejkuHLqZWhEPe3QPH1weVXnWJhaeoi8BGkvjCTLW6t4qT): protocol: expected piece hash; context canceled; EOF Mar 18 21:54:25.620718441 (298h54m23.217489474s) [noise=true , address=145.79.227.37:4569 , piece_id=PTE4YT3YJ5M3NDKXKQOUFTGQ3T5FDAUZB74QHJBVH6HQL53QXLUA, seg_pos={0 0}, upload=334781]: Long tail cancellation due to optimal threshold Mar 18 21:54:25.620748329 (298h54m23.21751937s) [seg_pos={0 0}, upload=334781]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Mar 18 21:54:25.620912612 (298h54m23.21768364s) [upload=334781]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 18 21:54:25.620920854 (298h54m23.217691883s) [upload=334781]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 21:54:38.136688455 (298h54m35.733459485s) [upload=334781]: Done uploading segments... Mar 18 21:54:38.136713048 (298h54m35.73348408s) [upload=334781]: Upload finished. err: <nil> Duration: 483.869112ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 15 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXX.............................................] duration: 513.080504ms trace: https://jaeger.dev.storj.io/trace/6cabdd7d47c9cb44 FAIL (2026-03-18 22:28:36 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 513.130446ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 10 out of 60 is failed (threshold=3) [XXXXXX.X..XXX...............................................] duration: 1.870258184s trace: https://jaeger.dev.storj.io/trace/27ed2876d5f3e85d FAIL (2026-03-18 22:28:34 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 1.87035021s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 9 out of 60 is failed (threshold=3) [XXXX.X...XX.X.X.............................................] duration: 323.306266ms trace: https://jaeger.dev.storj.io/trace/47004c0815b844e7 FAIL (2026-03-18 22:28:33 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 18 21:53:36.090826124 (298h53m33.687597155s) [upload=334767]: Starting upload Mar 18 21:53:36.090977797 (298h53m33.687748822s) [upload=334767]: Done starting upload Mar 18 21:53:36.091078093 (298h53m33.687849119s) [upload=334767]: Uploading segments... Mar 18 21:53:36.091268776 (298h53m33.688040083s) [upload=334767]: Got next segment. Inline: true Mar 18 21:53:36.091279074 (298h53m33.688050103s) [upload=334767]: Waiting for error group managing segments... Mar 18 21:53:36.091372978 (298h53m33.688144001s) [upload=334767]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 21:53:36.091379985 (298h53m33.688151008s) [upload=334767]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Mar 18 21:53:36.091384265 (298h53m33.688155287s) [upload=334767]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 21:54:04.718146694 (298h54m2.314917717s) [upload=334767]: Done uploading segments... Mar 18 21:54:04.718169125 (298h54m2.314940155s) [upload=334767]: Upload finished. err: <nil> Duration:* 323.325387ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 9 out of 60 is failed (threshold=3) [XX..XX.X..X..X.XX...........................................] duration: 252.631936ms trace: https://jaeger.dev.storj.io/trace/7c4371cfa8f7834c FAIL (2026-03-18 22:28:08 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 252.662283ms core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 10 out of 60 is failed (threshold=3) [XXXXX.X...XX.X.X............................................] duration: 282.086786ms trace: https://jaeger.dev.storj.io/trace/47004c0815b844e7 FAIL (2026-03-18 22:27:50 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 18 21:53:36.090826124 (298h53m33.687597155s) [upload=334767]: Starting upload Mar 18 21:53:36.090977797 (298h53m33.687748822s) [upload=334767]: Done starting upload Mar 18 21:53:36.091078093 (298h53m33.687849119s) [upload=334767]: Uploading segments... Mar 18 21:53:36.091268776 (298h53m33.688040083s) [upload=334767]: Got next segment. Inline: true Mar 18 21:53:36.091279074 (298h53m33.688050103s) [upload=334767]: Waiting for error group managing segments... Mar 18 21:53:36.091372978 (298h53m33.688144001s) [upload=334767]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 21:53:36.091379985 (298h53m33.688151008s) [upload=334767]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Mar 18 21:53:36.091384265 (298h53m33.688155287s) [upload=334767]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 21:54:04.718146694 (298h54m2.314917717s) [upload=334767]: Done uploading segments... Mar 18 21:54:04.718169125 (298h54m2.314940155s) [upload=334767]: Upload finished. err: <nil> Duration:* 282.115464ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [XX.....XXXX.XXX.............................................] duration: 468.182889ms trace: https://jaeger.dev.storj.io/trace/6dceff78c86f0774 FAIL (2026-03-18 22:27:46 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 18 21:54:17.190275520 (298h54m14.787046534s) [upload=334781]: Starting upload Mar 18 21:54:17.190367052 (298h54m14.787138068s) [upload=334781]: Done starting upload Mar 18 21:54:17.190448332 (298h54m14.787219348s) [upload=334781]: Uploading segments... Mar 18 21:54:17.190705184 (298h54m14.787476207s) [upload=334781]: Got next segment. Inline: false Mar 18 21:54:17.190783980 (298h54m14.787555002s) [upload=334781]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 21:54:17.190804859 (298h54m14.787575882s) [upload=334781]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 18 21:54:25.452304457 (298h54m23.049075487s) [seg_pos={0 0}, upload=334781]: Begin upload segment... Mar 18 21:54:25.452374690 (298h54m23.049145713s) [seg_pos={0 0}, upload=334781]: Done begin upload segment. Mar 18 21:54:25.452463335 (298h54m23.049234366s) [noise=true , address=145.79.227.46:4604 , piece_id=DAGTJ24IRMED5FNGQHRYTXEIXYTDVSRW2IT23MC2UC45SES3PE7Q, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.452479705 (298h54m23.049250739s) [noise=true , address=45.91.116.9:4264 , piece_id=EE3TXU5KSPHKMNNCI3BHIGFKQNYZQRBDO4YT3TTR5U66YJMFT6XQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.453224160 (298h54m23.049995188s) [noise=true , address=79.127.219.46:4892 , piece_id=533AWUNWA66VX4AZF7Y3AANCF65GMESWZD3IJWBC5IOKB4QMVTZA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.454147610 (298h54m23.050918643s) [noise=true , address=145.79.227.48:4307 , piece_id=D6WSRVCL6UO4KQ542EKZWA6IP7FLA3ZUGSBXCSZUMLPFCJ5G5CQA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.454790080 (298h54m23.051561121s) [noise=true , address=145.79.227.2:4116 , piece_id=KEKK3M6M37G7Y7HXPULI2RSX5CFGUNCM3LILYNQOZ2PASM742ZGQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.455678920 (298h54m23.052449952s) [noise=true , address=145.79.227.19:4080 , piece_id=ANOMNO3HLII5GPWIT2RDX6P7WOBTVJXMOJM5PAPDMK7ST2NYJWMA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.456387102 (298h54m23.053158136s) [noise=true , address=145.79.227.37:4569 , piece_id=PTE4YT3YJ5M3NDKXKQOUFTGQ3T5FDAUZB74QHJBVH6HQL53QXLUA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.456987483 (298h54m23.053758515s) [noise=true , address=145.79.227.4:4190 , piece_id=LDID6UGFJ4GWYK635S5OE6334DEULOPVSOLOCLYQCBOFTXRZEKRQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.457980795 (298h54m23.054751829s) [noise=true , address=79.127.219.45:4855 , piece_id=644Z2MAXJ423ZF4JGZ2YLPM6XJRZT7IUJCZRRE7ZJMHB3EW7KB4Q, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.459806473 (298h54m23.056577503s) [noise=true , address=145.79.227.50:4376 , piece_id=BHABIBKA3ZENQUNU2OVBVUJZVWEQRZD435EHA3MEB25KMNBVFQIA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.460305248 (298h54m23.057076281s) [noise=true , address=77.90.31.16:4737 , piece_id=SMQOC6BKSIEIE5FXEYZGOIWRZ6GORC7GCOVEOFZXPQX4FGILVIPA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.461552822 (298h54m23.058323848s) [noise=true , address=79.127.219.41:4926 , piece_id=AVSCKJ5WDK4OBWWUSLUQNOPZI7BGTTDGT4XJYSLQAJPUHRYY4KVQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.463144575 (298h54m23.059915608s) [noise=true , address=145.79.227.36:4518 , piece_id=R5PDKAATCPFI3LOOUX6HJ4OP64NQ72N3RAOZS3GYFH5VSWPFU6OQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.464837828 (298h54m23.06160886s) [noise=true , address=145.79.227.26:4656 , piece_id=DDNFTYMUJMFHTZXFHJTFUURSVOJBQX2KVT6UWLBCZ5TIOVBDVOVQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.466411504 (298h54m23.063182537s) [noise=true , address=145.79.227.6:4261 , piece_id=CYOWHNR7X3Y4YE5SFHPNDBOU5QQ6M6DZIRAO3VX56KQBL3OP2FLQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.469091278 (298h54m23.065862311s) [noise=true , address=45.91.116.11:4118 , piece_id=CVKN5WPP2OTRAHYWJKDLCJV3OZ3I6G5CLYXTSEKJDQD7LRTJ55EA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.586976347 (298h54m23.183747361s) [upload=334781]: Next returned nil segment Mar 18 21:54:25.587000505 (298h54m23.183771534s) [upload=334781]: Waiting for error group managing segments... Mar 18 21:54:25.596319798 (298h54m23.193090825s) [noise=true , address=79.127.219.41:4926 , piece_id=AVSCKJ5WDK4OBWWUSLUQNOPZI7BGTTDGT4XJYSLQAJPUHRYY4KVQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.596985876 (298h54m23.1937569s) [noise=true , address=45.91.116.9:4264 , piece_id=EE3TXU5KSPHKMNNCI3BHIGFKQNYZQRBDO4YT3TTR5U66YJMFT6XQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.598026901 (298h54m23.194797934s) [noise=true , address=145.79.227.26:4656 , piece_id=DDNFTYMUJMFHTZXFHJTFUURSVOJBQX2KVT6UWLBCZ5TIOVBDVOVQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.598566162 (298h54m23.195337186s) [noise=true , address=145.79.227.50:4376 , piece_id=BHABIBKA3ZENQUNU2OVBVUJZVWEQRZD435EHA3MEB25KMNBVFQIA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.598781105 (298h54m23.19555213s) [noise=true , address=77.90.31.16:4737 , piece_id=SMQOC6BKSIEIE5FXEYZGOIWRZ6GORC7GCOVEOFZXPQX4FGILVIPA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.600260475 (298h54m23.197031509s) [noise=true , address=145.79.227.6:4261 , piece_id=CYOWHNR7X3Y4YE5SFHPNDBOU5QQ6M6DZIRAO3VX56KQBL3OP2FLQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.600344398 (298h54m23.197115411s) [noise=true , address=145.79.227.4:4190 , piece_id=LDID6UGFJ4GWYK635S5OE6334DEULOPVSOLOCLYQCBOFTXRZEKRQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.602467676 (298h54m23.19923871s) [noise=true , address=145.79.227.2:4116 , piece_id=KEKK3M6M37G7Y7HXPULI2RSX5CFGUNCM3LILYNQOZ2PASM742ZGQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.602977247 (298h54m23.199748272s) [noise=true , address=145.79.227.48:4307 , piece_id=D6WSRVCL6UO4KQ542EKZWA6IP7FLA3ZUGSBXCSZUMLPFCJ5G5CQA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.603520888 (298h54m23.200291919s) [noise=true , address=79.127.219.45:4855 , piece_id=644Z2MAXJ423ZF4JGZ2YLPM6XJRZT7IUJCZRRE7ZJMHB3EW7KB4Q, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.603907549 (298h54m23.200678573s) [noise=true , address=145.79.227.36:4518 , piece_id=R5PDKAATCPFI3LOOUX6HJ4OP64NQ72N3RAOZS3GYFH5VSWPFU6OQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.604438080 (298h54m23.201209113s) [noise=true , address=45.91.116.11:4118 , piece_id=CVKN5WPP2OTRAHYWJKDLCJV3OZ3I6G5CLYXTSEKJDQD7LRTJ55EA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.618924566 (298h54m23.21569559s) [noise=true , address=79.127.219.46:4892 , piece_id=533AWUNWA66VX4AZF7Y3AANCF65GMESWZD3IJWBC5IOKB4QMVTZA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.619077190 (298h54m23.215848214s) [seg_pos={0 0}, upload=334781]: Satisfied cohort requirements after 13 pieces Mar 18 21:54:25.619546929 (298h54m23.216317961s) [noise=true , address=145.79.227.46:4604 , piece_id=DAGTJ24IRMED5FNGQHRYTXEIXYTDVSRW2IT23MC2UC45SES3PE7Q, seg_pos={0 0}, upload=334781]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy): protocol: expected piece hash; context canceled; EOF Mar 18 21:54:25.619578476 (298h54m23.21634951s) [noise=true , address=145.79.227.46:4604 , piece_id=DAGTJ24IRMED5FNGQHRYTXEIXYTDVSRW2IT23MC2UC45SES3PE7Q, seg_pos={0 0}, upload=334781]: Long tail cancellation due to optimal threshold Mar 18 21:54:25.620166659 (298h54m23.216937744s) [noise=true , address=145.79.227.19:4080 , piece_id=ANOMNO3HLII5GPWIT2RDX6P7WOBTVJXMOJM5PAPDMK7ST2NYJWMA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KK51wudKiZb39Ku7wZhgBYEYmgsZTHfjiNNLmpBU9uuNFapt3): protocol: expected piece hash; context canceled; EOF Mar 18 21:54:25.620178720 (298h54m23.216949746s) [noise=true , address=145.79.227.19:4080 , piece_id=ANOMNO3HLII5GPWIT2RDX6P7WOBTVJXMOJM5PAPDMK7ST2NYJWMA, seg_pos={0 0}, upload=334781]: Long tail cancellation due to optimal threshold Mar 18 21:54:25.620702987 (298h54m23.217474021s) [noise=true , address=145.79.227.37:4569 , piece_id=PTE4YT3YJ5M3NDKXKQOUFTGQ3T5FDAUZB74QHJBVH6HQL53QXLUA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:18TqoejkuHLqZWhEPe3QPH1weVXnWJhaeoi8BGkvjCTLW6t4qT): protocol: expected piece hash; context canceled; EOF Mar 18 21:54:25.620718441 (298h54m23.217489474s) [noise=true , address=145.79.227.37:4569 , piece_id=PTE4YT3YJ5M3NDKXKQOUFTGQ3T5FDAUZB74QHJBVH6HQL53QXLUA, seg_pos={0 0}, upload=334781]: Long tail cancellation due to optimal threshold Mar 18 21:54:25.620748329 (298h54m23.21751937s) [seg_pos={0 0}, upload=334781]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Mar 18 21:54:25.620912612 (298h54m23.21768364s) [upload=334781]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 18 21:54:25.620920854 (298h54m23.217691883s) [upload=334781]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 21:54:38.136688455 (298h54m35.733459485s) [upload=334781]: Done uploading segments... Mar 18 21:54:38.136713048 (298h54m35.73348408s) [upload=334781]: Upload finished. err: <nil> Duration: 468.226557ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 228.697369ms trace: https://jaeger.dev.storj.io/trace/2ab94940f3da3187 FAIL (2026-03-18 22:27:45 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 18 21:41:41.272084084 (298h41m38.868855112s) [upload=334559]: Starting upload Mar 18 21:41:41.272169036 (298h41m38.868940066s) [upload=334559]: Done starting upload Mar 18 21:41:41.272251155 (298h41m38.869022168s) [upload=334559]: Uploading segments... Mar 18 21:41:41.272358183 (298h41m38.869129211s) [upload=334559]: Got next segment. Inline: true Mar 18 21:41:41.272367512 (298h41m38.869138542s) [upload=334559]: Waiting for error group managing segments... Mar 18 21:41:41.272525860 (298h41m38.869296906s) [upload=334559]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 21:41:41.272550748 (298h41m38.869321776s) [upload=334559]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Mar 18 21:41:41.272555523 (298h41m38.86932655s) [upload=334559]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 21:42:30.253452076 (298h42m27.850223099s) [upload=334559]: Done uploading segments... Mar 18 21:42:30.253475139 (298h42m27.850246163s) [upload=334559]: Upload finished. err: <nil> Duration:* 228.726223ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 10 out of 60 is failed (threshold=3) [.XXXXXX.X..XXX..............................................] duration: 1.6874302s trace: https://jaeger.dev.storj.io/trace/27ed2876d5f3e85d FAIL (2026-03-18 22:27:30 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 1.687461339s core uplink us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 16 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXX............................................] duration: 735.453165ms trace: https://jaeger.dev.storj.io/trace/6cabdd7d47c9cb44 FAIL (2026-03-18 22:27:28 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 735.52412ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 213.149146ms trace: https://jaeger.dev.storj.io/trace/2ab94940f3da3187 FAIL (2026-03-18 22:27:21 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 18 21:41:41.272084084 (298h41m38.868855112s) [upload=334559]: Starting upload Mar 18 21:41:41.272169036 (298h41m38.868940066s) [upload=334559]: Done starting upload Mar 18 21:41:41.272251155 (298h41m38.869022168s) [upload=334559]: Uploading segments... Mar 18 21:41:41.272358183 (298h41m38.869129211s) [upload=334559]: Got next segment. Inline: true Mar 18 21:41:41.272367512 (298h41m38.869138542s) [upload=334559]: Waiting for error group managing segments... Mar 18 21:41:41.272525860 (298h41m38.869296906s) [upload=334559]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 21:41:41.272550748 (298h41m38.869321776s) [upload=334559]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Mar 18 21:41:41.272555523 (298h41m38.86932655s) [upload=334559]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 21:42:30.253452076 (298h42m27.850223099s) [upload=334559]: Done uploading segments... Mar 18 21:42:30.253475139 (298h42m27.850246163s) [upload=334559]: Upload finished. err: <nil> Duration:* 213.176219ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 11 out of 60 is failed (threshold=3) [XXXXXX.X...XX.X.X...........................................] duration: 293.012157ms trace: https://jaeger.dev.storj.io/trace/47004c0815b844e7 FAIL (2026-03-18 22:27:13 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 18 21:53:36.090826124 (298h53m33.687597155s) [upload=334767]: Starting upload Mar 18 21:53:36.090977797 (298h53m33.687748822s) [upload=334767]: Done starting upload Mar 18 21:53:36.091078093 (298h53m33.687849119s) [upload=334767]: Uploading segments... Mar 18 21:53:36.091268776 (298h53m33.688040083s) [upload=334767]: Got next segment. Inline: true Mar 18 21:53:36.091279074 (298h53m33.688050103s) [upload=334767]: Waiting for error group managing segments... Mar 18 21:53:36.091372978 (298h53m33.688144001s) [upload=334767]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 21:53:36.091379985 (298h53m33.688151008s) [upload=334767]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Mar 18 21:53:36.091384265 (298h53m33.688155287s) [upload=334767]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 21:54:04.718146694 (298h54m2.314917717s) [upload=334767]: Done uploading segments... Mar 18 21:54:04.718169125 (298h54m2.314940155s) [upload=334767]: Upload finished. err: <nil> Duration:* 293.125585ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 17 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXX...........................................] duration: 510.682711ms trace: https://jaeger.dev.storj.io/trace/6cabdd7d47c9cb44 FAIL (2026-03-18 22:27:08 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 510.742148ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [XXX.....XXXX.XXX............................................] duration: 495.134568ms trace: https://jaeger.dev.storj.io/trace/6dceff78c86f0774 FAIL (2026-03-18 22:27:06 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 18 21:54:17.190275520 (298h54m14.787046534s) [upload=334781]: Starting upload Mar 18 21:54:17.190367052 (298h54m14.787138068s) [upload=334781]: Done starting upload Mar 18 21:54:17.190448332 (298h54m14.787219348s) [upload=334781]: Uploading segments... Mar 18 21:54:17.190705184 (298h54m14.787476207s) [upload=334781]: Got next segment. Inline: false Mar 18 21:54:17.190783980 (298h54m14.787555002s) [upload=334781]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 21:54:17.190804859 (298h54m14.787575882s) [upload=334781]: Flush batch item: pb.BatchRequestItem_SegmentBegin Mar 18 21:54:25.452304457 (298h54m23.049075487s) [seg_pos={0 0}, upload=334781]: Begin upload segment... Mar 18 21:54:25.452374690 (298h54m23.049145713s) [seg_pos={0 0}, upload=334781]: Done begin upload segment. Mar 18 21:54:25.452463335 (298h54m23.049234366s) [noise=true , address=145.79.227.46:4604 , piece_id=DAGTJ24IRMED5FNGQHRYTXEIXYTDVSRW2IT23MC2UC45SES3PE7Q, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.452479705 (298h54m23.049250739s) [noise=true , address=45.91.116.9:4264 , piece_id=EE3TXU5KSPHKMNNCI3BHIGFKQNYZQRBDO4YT3TTR5U66YJMFT6XQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.453224160 (298h54m23.049995188s) [noise=true , address=79.127.219.46:4892 , piece_id=533AWUNWA66VX4AZF7Y3AANCF65GMESWZD3IJWBC5IOKB4QMVTZA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.454147610 (298h54m23.050918643s) [noise=true , address=145.79.227.48:4307 , piece_id=D6WSRVCL6UO4KQ542EKZWA6IP7FLA3ZUGSBXCSZUMLPFCJ5G5CQA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.454790080 (298h54m23.051561121s) [noise=true , address=145.79.227.2:4116 , piece_id=KEKK3M6M37G7Y7HXPULI2RSX5CFGUNCM3LILYNQOZ2PASM742ZGQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.455678920 (298h54m23.052449952s) [noise=true , address=145.79.227.19:4080 , piece_id=ANOMNO3HLII5GPWIT2RDX6P7WOBTVJXMOJM5PAPDMK7ST2NYJWMA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.456387102 (298h54m23.053158136s) [noise=true , address=145.79.227.37:4569 , piece_id=PTE4YT3YJ5M3NDKXKQOUFTGQ3T5FDAUZB74QHJBVH6HQL53QXLUA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.456987483 (298h54m23.053758515s) [noise=true , address=145.79.227.4:4190 , piece_id=LDID6UGFJ4GWYK635S5OE6334DEULOPVSOLOCLYQCBOFTXRZEKRQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.457980795 (298h54m23.054751829s) [noise=true , address=79.127.219.45:4855 , piece_id=644Z2MAXJ423ZF4JGZ2YLPM6XJRZT7IUJCZRRE7ZJMHB3EW7KB4Q, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.459806473 (298h54m23.056577503s) [noise=true , address=145.79.227.50:4376 , piece_id=BHABIBKA3ZENQUNU2OVBVUJZVWEQRZD435EHA3MEB25KMNBVFQIA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.460305248 (298h54m23.057076281s) [noise=true , address=77.90.31.16:4737 , piece_id=SMQOC6BKSIEIE5FXEYZGOIWRZ6GORC7GCOVEOFZXPQX4FGILVIPA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.461552822 (298h54m23.058323848s) [noise=true , address=79.127.219.41:4926 , piece_id=AVSCKJ5WDK4OBWWUSLUQNOPZI7BGTTDGT4XJYSLQAJPUHRYY4KVQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.463144575 (298h54m23.059915608s) [noise=true , address=145.79.227.36:4518 , piece_id=R5PDKAATCPFI3LOOUX6HJ4OP64NQ72N3RAOZS3GYFH5VSWPFU6OQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.464837828 (298h54m23.06160886s) [noise=true , address=145.79.227.26:4656 , piece_id=DDNFTYMUJMFHTZXFHJTFUURSVOJBQX2KVT6UWLBCZ5TIOVBDVOVQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.466411504 (298h54m23.063182537s) [noise=true , address=145.79.227.6:4261 , piece_id=CYOWHNR7X3Y4YE5SFHPNDBOU5QQ6M6DZIRAO3VX56KQBL3OP2FLQ, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.469091278 (298h54m23.065862311s) [noise=true , address=45.91.116.11:4118 , piece_id=CVKN5WPP2OTRAHYWJKDLCJV3OZ3I6G5CLYXTSEKJDQD7LRTJ55EA, seg_pos={0 0}, upload=334781]: Uploading piece... Mar 18 21:54:25.586976347 (298h54m23.183747361s) [upload=334781]: Next returned nil segment Mar 18 21:54:25.587000505 (298h54m23.183771534s) [upload=334781]: Waiting for error group managing segments... Mar 18 21:54:25.596319798 (298h54m23.193090825s) [noise=true , address=79.127.219.41:4926 , piece_id=AVSCKJ5WDK4OBWWUSLUQNOPZI7BGTTDGT4XJYSLQAJPUHRYY4KVQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.596985876 (298h54m23.1937569s) [noise=true , address=45.91.116.9:4264 , piece_id=EE3TXU5KSPHKMNNCI3BHIGFKQNYZQRBDO4YT3TTR5U66YJMFT6XQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.598026901 (298h54m23.194797934s) [noise=true , address=145.79.227.26:4656 , piece_id=DDNFTYMUJMFHTZXFHJTFUURSVOJBQX2KVT6UWLBCZ5TIOVBDVOVQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.598566162 (298h54m23.195337186s) [noise=true , address=145.79.227.50:4376 , piece_id=BHABIBKA3ZENQUNU2OVBVUJZVWEQRZD435EHA3MEB25KMNBVFQIA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.598781105 (298h54m23.19555213s) [noise=true , address=77.90.31.16:4737 , piece_id=SMQOC6BKSIEIE5FXEYZGOIWRZ6GORC7GCOVEOFZXPQX4FGILVIPA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.600260475 (298h54m23.197031509s) [noise=true , address=145.79.227.6:4261 , piece_id=CYOWHNR7X3Y4YE5SFHPNDBOU5QQ6M6DZIRAO3VX56KQBL3OP2FLQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.600344398 (298h54m23.197115411s) [noise=true , address=145.79.227.4:4190 , piece_id=LDID6UGFJ4GWYK635S5OE6334DEULOPVSOLOCLYQCBOFTXRZEKRQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.602467676 (298h54m23.19923871s) [noise=true , address=145.79.227.2:4116 , piece_id=KEKK3M6M37G7Y7HXPULI2RSX5CFGUNCM3LILYNQOZ2PASM742ZGQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.602977247 (298h54m23.199748272s) [noise=true , address=145.79.227.48:4307 , piece_id=D6WSRVCL6UO4KQ542EKZWA6IP7FLA3ZUGSBXCSZUMLPFCJ5G5CQA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.603520888 (298h54m23.200291919s) [noise=true , address=79.127.219.45:4855 , piece_id=644Z2MAXJ423ZF4JGZ2YLPM6XJRZT7IUJCZRRE7ZJMHB3EW7KB4Q, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.603907549 (298h54m23.200678573s) [noise=true , address=145.79.227.36:4518 , piece_id=R5PDKAATCPFI3LOOUX6HJ4OP64NQ72N3RAOZS3GYFH5VSWPFU6OQ, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.604438080 (298h54m23.201209113s) [noise=true , address=45.91.116.11:4118 , piece_id=CVKN5WPP2OTRAHYWJKDLCJV3OZ3I6G5CLYXTSEKJDQD7LRTJ55EA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.618924566 (298h54m23.21569559s) [noise=true , address=79.127.219.46:4892 , piece_id=533AWUNWA66VX4AZF7Y3AANCF65GMESWZD3IJWBC5IOKB4QMVTZA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: <nil> Mar 18 21:54:25.619077190 (298h54m23.215848214s) [seg_pos={0 0}, upload=334781]: Satisfied cohort requirements after 13 pieces Mar 18 21:54:25.619546929 (298h54m23.216317961s) [noise=true , address=145.79.227.46:4604 , piece_id=DAGTJ24IRMED5FNGQHRYTXEIXYTDVSRW2IT23MC2UC45SES3PE7Q, seg_pos={0 0}, upload=334781]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy): protocol: expected piece hash; context canceled; EOF Mar 18 21:54:25.619578476 (298h54m23.21634951s) [noise=true , address=145.79.227.46:4604 , piece_id=DAGTJ24IRMED5FNGQHRYTXEIXYTDVSRW2IT23MC2UC45SES3PE7Q, seg_pos={0 0}, upload=334781]: Long tail cancellation due to optimal threshold Mar 18 21:54:25.620166659 (298h54m23.216937744s) [noise=true , address=145.79.227.19:4080 , piece_id=ANOMNO3HLII5GPWIT2RDX6P7WOBTVJXMOJM5PAPDMK7ST2NYJWMA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KK51wudKiZb39Ku7wZhgBYEYmgsZTHfjiNNLmpBU9uuNFapt3): protocol: expected piece hash; context canceled; EOF Mar 18 21:54:25.620178720 (298h54m23.216949746s) [noise=true , address=145.79.227.19:4080 , piece_id=ANOMNO3HLII5GPWIT2RDX6P7WOBTVJXMOJM5PAPDMK7ST2NYJWMA, seg_pos={0 0}, upload=334781]: Long tail cancellation due to optimal threshold Mar 18 21:54:25.620702987 (298h54m23.217474021s) [noise=true , address=145.79.227.37:4569 , piece_id=PTE4YT3YJ5M3NDKXKQOUFTGQ3T5FDAUZB74QHJBVH6HQL53QXLUA, seg_pos={0 0}, upload=334781]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:18TqoejkuHLqZWhEPe3QPH1weVXnWJhaeoi8BGkvjCTLW6t4qT): protocol: expected piece hash; context canceled; EOF Mar 18 21:54:25.620718441 (298h54m23.217489474s) [noise=true , address=145.79.227.37:4569 , piece_id=PTE4YT3YJ5M3NDKXKQOUFTGQ3T5FDAUZB74QHJBVH6HQL53QXLUA, seg_pos={0 0}, upload=334781]: Long tail cancellation due to optimal threshold Mar 18 21:54:25.620748329 (298h54m23.21751937s) [seg_pos={0 0}, upload=334781]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Mar 18 21:54:25.620912612 (298h54m23.21768364s) [upload=334781]: Flush batch item: pb.BatchRequestItem_SegmentCommit Mar 18 21:54:25.620920854 (298h54m23.217691883s) [upload=334781]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 21:54:38.136688455 (298h54m35.733459485s) [upload=334781]: Done uploading segments... Mar 18 21:54:38.136713048 (298h54m35.73348408s) [upload=334781]: Upload finished. err: <nil> Duration: 495.165194ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 11 out of 60 is failed (threshold=3) [X.XXXXXX.X..XXX.............................................] duration: 1.633209041s trace: https://jaeger.dev.storj.io/trace/27ed2876d5f3e85d FAIL (2026-03-18 22:27:02 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 1.633295952s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 10 out of 60 is failed (threshold=3) [XXX..XX.X..X..X.XX..........................................] duration: 229.551924ms trace: https://jaeger.dev.storj.io/trace/7c4371cfa8f7834c FAIL (2026-03-18 22:26:54 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 229.639758ms core uplink us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 18 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXX..........................................] duration: 549.392398ms trace: https://jaeger.dev.storj.io/trace/6cabdd7d47c9cb44 FAIL (2026-03-18 22:26:45 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 549.434365ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 193.920113ms trace: https://jaeger.dev.storj.io/trace/2ab94940f3da3187 FAIL (2026-03-18 22:26:40 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Mar 18 21:41:41.272084084 (298h41m38.868855112s) [upload=334559]: Starting upload Mar 18 21:41:41.272169036 (298h41m38.868940066s) [upload=334559]: Done starting upload Mar 18 21:41:41.272251155 (298h41m38.869022168s) [upload=334559]: Uploading segments... Mar 18 21:41:41.272358183 (298h41m38.869129211s) [upload=334559]: Got next segment. Inline: true Mar 18 21:41:41.272367512 (298h41m38.869138542s) [upload=334559]: Waiting for error group managing segments... Mar 18 21:41:41.272525860 (298h41m38.869296906s) [upload=334559]: Flush batch item: pb.BatchRequestItem_ObjectBegin Mar 18 21:41:41.272550748 (298h41m38.869321776s) [upload=334559]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Mar 18 21:41:41.272555523 (298h41m38.86932655s) [upload=334559]: Flush batch item: pb.BatchRequestItem_ObjectCommit Mar 18 21:42:30.253452076 (298h42m27.850223099s) [upload=334559]: Done uploading segments... Mar 18 21:42:30.253475139 (298h42m27.850246163s) [upload=334559]: Upload finished. err: <nil> Duration:* 193.952505ms core uplink us-select-1 us1-placement-20 metainfo-projects

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)