Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
1.649316374s trace:
https://jaeger.dev.storj.io/trace/62357518aac8ca5f
FAIL (2026-03-25 15:10:16 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 1.649429402s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................] duration:
1.411949151s trace:
https://jaeger.dev.storj.io/trace/62357518aac8ca5f
FAIL (2026-03-25 15:09:25 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 1.412018214s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
1.363542424s trace:
https://jaeger.dev.storj.io/trace/62357518aac8ca5f
FAIL (2026-03-25 15:09:00 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 1.363577688s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................] duration:
1.278721184s trace:
https://jaeger.dev.storj.io/trace/62357518aac8ca5f
FAIL (2026-03-25 15:08:34 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 1.278867261s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [......X.....................................................] duration:
1.830059476s trace:
https://jaeger.dev.storj.io/trace/62357518aac8ca5f
FAIL (2026-03-25 15:07:27 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 1.830128185s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
1.609352743s trace:
https://jaeger.dev.storj.io/trace/1d1f0bc4e4438495
FAIL (2026-03-25 15:07:19 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
upload=514680]: Uploading piece...
Mar 25 14:21:03.327216643 (459h21m0.923987655s) [noise=true , address=45.91.116.12:4254 , piece_id=UIQ3FWZSZ52RJYWFVWNH3OGNQPZSDYEZEDX35K3R7SDKRIW3WOAQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.327580267 (459h21m0.924351301s) [noise=true , address=193.149.164.70:4120 , piece_id=HMFOJCRQTG3PAQDIZERNEYWKFFBUIRKRXP56OQUQWH6R2YX6NYLQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.327966258 (459h21m0.924737281s) [noise=true , address=77.90.31.16:4739 , piece_id=LEV3TZ2XFAYLQHULRJOG3DDBFMP3XRWX4EU2CQ2YOFICYXLIR3SA, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.328304150 (459h21m0.925075204s) [noise=true , address=77.90.31.14:4688 , piece_id=XDS6GE2YZK3LWUTBOMHLWTHM4ASCQJI25WTENUU75TJGFULYPOCQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.328528343 (459h21m0.925299366s) [noise=true , address=64.49.13.138:4487 , piece_id=NMZRB7X7MX2KJRDEUBROG3F6ZPQOGYJLTJ5WHBBFOUQCKXIHIZKQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.328832656 (459h21m0.925603704s) [noise=true , address=145.79.227.44:4201 , piece_id=SHQIEKFZSOKO7SUBGDC7GSJQNL47IEWENP5KN5ZN5ZOS4K3R7Y7A, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.328990765 (459h21m0.925761788s) [noise=true , address=77.90.31.17:4791 , piece_id=L5H6QECJGEQPTPRB6INQWHZPMK7ZXXDYYFFG5ANE37HAE33OTVXQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.329286413 (459h21m0.926057443s) [noise=true , address=64.49.13.27:4964 , piece_id=UOBMO3UGR2SVYEXRSW2SQMHSDJV47OXOZRZJHCOHVWNXMIIDSA2A, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.329346877 (459h21m0.9261179s) [noise=true , address=193.149.164.66:4984 , piece_id=CGM44RIHZGETSCKRZQOFL3VYIJTSB57RQDXTG3OPX3F3HJNQDYZA, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.329706811 (459h21m0.926477834s) [noise=true , address=45.91.116.7:4023 , piece_id=GYWVVXKLOAZSDJIS64INTFQIPQZCLBH7YPLTWY7QJSB766A3QTGQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.329728635 (459h21m0.926499661s) [noise=true , address=64.49.13.140:4509 , piece_id=7HN3FXTKC4RRUJZ34AHFDJHMUBIOWIQVFST626CR7X2ZYQS3CFCQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.330142941 (459h21m0.926913971s) [noise=true , address=145.79.227.34:4916 , piece_id=645H2KOPV4YB2HTP3XZF2RF5ZICIR34QNYRNRKX5KHH2VANVVZOQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.330171985 (459h21m0.926943015s) [noise=true , address=193.149.164.71:4296 , piece_id=RXZXV3FCVDEFHBJQVGIWEKS57EJ3IKAJFTDGMOEZHRRBHV3NRDSA, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.330543280 (459h21m0.927314303s) [noise=true , address=193.149.164.67:4030 , piece_id=PBIDHXYQ22HHK3HBJLFPC5X4L74Q332LNNEDYPFDV6SHWCQKRCNQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.330675401 (459h21m0.927446433s) [noise=true , address=64.49.13.26:4955 , piece_id=VIN3RVGRSUUHYCHPXCDNVSPRLFSCH3JXIDERBV7NIM4K33ZUBUGQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.330936609 (459h21m0.927707633s) [noise=true , address=193.149.164.68:4050 , piece_id=F4AQMCLJL6ML2LQKWUIKJAN6C2Y3M7UVYXRCRVRON5BC6CLTEK6Q, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.331355022 (459h21m0.928126051s) [noise=true , address=77.90.31.11:4567 , piece_id=ZSITPEN2OCYCHLSV3PDXCAXBAHEAQMS3YFCCOOE2SISJMUFRPHOA, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.402821184 (459h21m0.999592291s) [upload=514680]: Next returned nil segment
Mar 25 14:21:03.402848744 (459h21m0.99961978s) [upload=514680]: Waiting for error group managing segments...
Mar 25 14:21:03.408851473 (459h21m1.005622504s) [noise=true , address=77.90.31.16:4739 , piece_id=LEV3TZ2XFAYLQHULRJOG3DDBFMP3XRWX4EU2CQ2YOFICYXLIR3SA, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.409008646 (459h21m1.005779679s) [noise=true , address=145.79.227.34:4916 , piece_id=645H2KOPV4YB2HTP3XZF2RF5ZICIR34QNYRNRKX5KHH2VANVVZOQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.418584053 (459h21m1.015355084s) [noise=true , address=77.90.31.17:4791 , piece_id=L5H6QECJGEQPTPRB6INQWHZPMK7ZXXDYYFFG5ANE37HAE33OTVXQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.446927214 (459h21m1.043698248s) [noise=true , address=45.91.116.7:4023 , piece_id=GYWVVXKLOAZSDJIS64INTFQIPQZCLBH7YPLTWY7QJSB766A3QTGQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.448923805 (459h21m1.045694815s) [noise=true , address=77.90.31.14:4688 , piece_id=XDS6GE2YZK3LWUTBOMHLWTHM4ASCQJI25WTENUU75TJGFULYPOCQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.458340727 (459h21m1.055111751s) [noise=true , address=45.91.116.12:4254 , piece_id=UIQ3FWZSZ52RJYWFVWNH3OGNQPZSDYEZEDX35K3R7SDKRIW3WOAQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.468833670 (459h21m1.065604702s) [noise=true , address=77.90.31.11:4567 , piece_id=ZSITPEN2OCYCHLSV3PDXCAXBAHEAQMS3YFCCOOE2SISJMUFRPHOA, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.925971986 (459h21m1.522743012s) [noise=true , address=193.149.164.66:4984 , piece_id=CGM44RIHZGETSCKRZQOFL3VYIJTSB57RQDXTG3OPX3F3HJNQDYZA, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.928271525 (459h21m1.525042552s) [noise=true , address=64.49.13.140:4509 , piece_id=7HN3FXTKC4RRUJZ34AHFDJHMUBIOWIQVFST626CR7X2ZYQS3CFCQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.929087375 (459h21m1.525858388s) [noise=true , address=193.149.164.70:4120 , piece_id=HMFOJCRQTG3PAQDIZERNEYWKFFBUIRKRXP56OQUQWH6R2YX6NYLQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.929851130 (459h21m1.526622145s) [noise=true , address=193.149.164.71:4296 , piece_id=RXZXV3FCVDEFHBJQVGIWEKS57EJ3IKAJFTDGMOEZHRRBHV3NRDSA, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.935607616 (459h21m1.532378643s) [noise=true , address=193.149.164.69:4085 , piece_id=KGCVBEK344OW573PN2Y55JA7AC3ZTQD36G3T6TUHF7ZMZ4WN7N5A, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.938874518 (459h21m1.535645546s) [noise=true , address=193.149.164.67:4030 , piece_id=PBIDHXYQ22HHK3HBJLFPC5X4L74Q332LNNEDYPFDV6SHWCQKRCNQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.950243460 (459h21m1.547014493s) [noise=true , address=64.49.13.26:4955 , piece_id=VIN3RVGRSUUHYCHPXCDNVSPRLFSCH3JXIDERBV7NIM4K33ZUBUGQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.950277580 (459h21m1.54704861s) [seg_pos={0 0}, upload=514680]: Satisfied cohort requirements after 14 pieces
Mar 25 14:21:03.950707500 (459h21m1.547478532s) [noise=true , address=193.149.164.68:4050 , piece_id=F4AQMCLJL6ML2LQKWUIKJAN6C2Y3M7UVYXRCRVRON5BC6CLTEK6Q, seg_pos={0 0}, upload=514680]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1WPfYrCJrhPTqZ2qgBvXFCUodKz6QeXCnkKgToFs5muNRSSLtK): protocol: expected piece hash; context canceled; EOF
Mar 25 14:21:03.950721648 (459h21m1.547492674s) [noise=true , address=193.149.164.68:4050 , piece_id=F4AQMCLJL6ML2LQKWUIKJAN6C2Y3M7UVYXRCRVRON5BC6CLTEK6Q, seg_pos={0 0}, upload=514680]: Long tail cancellation due to optimal threshold
Mar 25 14:21:03.950793002 (459h21m1.547564034s) [noise=true , address=64.49.13.27:4964 , piece_id=UOBMO3UGR2SVYEXRSW2SQMHSDJV47OXOZRZJHCOHVWNXMIIDSA2A, seg_pos={0 0}, upload=514680]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RcgfaeKP3XpZGuSB9GMURCCsjjc79WeZasUL2RRiQVDwxbnAk): protocol: expected piece hash; context canceled; EOF
Mar 25 14:21:03.950804838 (459h21m1.547575864s) [noise=true , address=64.49.13.27:4964 , piece_id=UOBMO3UGR2SVYEXRSW2SQMHSDJV47OXOZRZJHCOHVWNXMIIDSA2A, seg_pos={0 0}, upload=514680]: Long tail cancellation due to optimal threshold
Mar 25 14:21:03.950899119 (459h21m1.547670144s) [noise=true , address=145.79.227.44:4201 , piece_id=SHQIEKFZSOKO7SUBGDC7GSJQNL47IEWENP5KN5ZN5ZOS4K3R7Y7A, seg_pos={0 0}, upload=514680]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bPDvhDfu8Z58LxZcicg1vAiiu1sZYGpwQnvuhk4t5m6yR87Fi): protocol: expected piece hash; context canceled; EOF
Mar 25 14:21:03.950907089 (459h21m1.547678114s) [noise=true , address=145.79.227.44:4201 , piece_id=SHQIEKFZSOKO7SUBGDC7GSJQNL47IEWENP5KN5ZN5ZOS4K3R7Y7A, seg_pos={0 0}, upload=514680]: Long tail cancellation due to optimal threshold
Mar 25 14:21:03.951133312 (459h21m1.547904347s) [noise=true , address=64.49.13.138:4487 , piece_id=NMZRB7X7MX2KJRDEUBROG3F6ZPQOGYJLTJ5WHBBFOUQCKXIHIZKQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12taemzrausuc58d8Dyo8xaR7MqDf72f37rEheF32sUfy8Z8GZP): protocol: expected piece hash; context canceled; EOF
Mar 25 14:21:03.951146101 (459h21m1.547917132s) [noise=true , address=64.49.13.138:4487 , piece_id=NMZRB7X7MX2KJRDEUBROG3F6ZPQOGYJLTJ5WHBBFOUQCKXIHIZKQ, seg_pos={0 0}, upload=514680]: Long tail cancellation due to optimal threshold
Mar 25 14:21:03.951175027 (459h21m1.547946039s) [seg_pos={0 0}, upload=514680]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Mar 25 14:21:03.951282270 (459h21m1.548053293s) [upload=514680]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Mar 25 14:21:03.951290421 (459h21m1.548061457s) [upload=514680]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Mar 25 14:21:04.013599996 (459h21m1.610371022s) [upload=514680]: Done uploading segments...
Mar 25 14:21:04.013621043 (459h21m1.610392066s) [upload=514680]: Upload finished. err: <nil>
Duration: 1.609390417s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......X....................................................] duration:
1.48025583s trace:
https://jaeger.dev.storj.io/trace/62357518aac8ca5f
FAIL (2026-03-25 15:06:35 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 1.48028546s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
1.465778679s trace:
https://jaeger.dev.storj.io/trace/1d1f0bc4e4438495
FAIL (2026-03-25 15:06:23 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
upload=514680]: Uploading piece...
Mar 25 14:21:03.327216643 (459h21m0.923987655s) [noise=true , address=45.91.116.12:4254 , piece_id=UIQ3FWZSZ52RJYWFVWNH3OGNQPZSDYEZEDX35K3R7SDKRIW3WOAQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.327580267 (459h21m0.924351301s) [noise=true , address=193.149.164.70:4120 , piece_id=HMFOJCRQTG3PAQDIZERNEYWKFFBUIRKRXP56OQUQWH6R2YX6NYLQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.327966258 (459h21m0.924737281s) [noise=true , address=77.90.31.16:4739 , piece_id=LEV3TZ2XFAYLQHULRJOG3DDBFMP3XRWX4EU2CQ2YOFICYXLIR3SA, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.328304150 (459h21m0.925075204s) [noise=true , address=77.90.31.14:4688 , piece_id=XDS6GE2YZK3LWUTBOMHLWTHM4ASCQJI25WTENUU75TJGFULYPOCQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.328528343 (459h21m0.925299366s) [noise=true , address=64.49.13.138:4487 , piece_id=NMZRB7X7MX2KJRDEUBROG3F6ZPQOGYJLTJ5WHBBFOUQCKXIHIZKQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.328832656 (459h21m0.925603704s) [noise=true , address=145.79.227.44:4201 , piece_id=SHQIEKFZSOKO7SUBGDC7GSJQNL47IEWENP5KN5ZN5ZOS4K3R7Y7A, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.328990765 (459h21m0.925761788s) [noise=true , address=77.90.31.17:4791 , piece_id=L5H6QECJGEQPTPRB6INQWHZPMK7ZXXDYYFFG5ANE37HAE33OTVXQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.329286413 (459h21m0.926057443s) [noise=true , address=64.49.13.27:4964 , piece_id=UOBMO3UGR2SVYEXRSW2SQMHSDJV47OXOZRZJHCOHVWNXMIIDSA2A, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.329346877 (459h21m0.9261179s) [noise=true , address=193.149.164.66:4984 , piece_id=CGM44RIHZGETSCKRZQOFL3VYIJTSB57RQDXTG3OPX3F3HJNQDYZA, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.329706811 (459h21m0.926477834s) [noise=true , address=45.91.116.7:4023 , piece_id=GYWVVXKLOAZSDJIS64INTFQIPQZCLBH7YPLTWY7QJSB766A3QTGQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.329728635 (459h21m0.926499661s) [noise=true , address=64.49.13.140:4509 , piece_id=7HN3FXTKC4RRUJZ34AHFDJHMUBIOWIQVFST626CR7X2ZYQS3CFCQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.330142941 (459h21m0.926913971s) [noise=true , address=145.79.227.34:4916 , piece_id=645H2KOPV4YB2HTP3XZF2RF5ZICIR34QNYRNRKX5KHH2VANVVZOQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.330171985 (459h21m0.926943015s) [noise=true , address=193.149.164.71:4296 , piece_id=RXZXV3FCVDEFHBJQVGIWEKS57EJ3IKAJFTDGMOEZHRRBHV3NRDSA, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.330543280 (459h21m0.927314303s) [noise=true , address=193.149.164.67:4030 , piece_id=PBIDHXYQ22HHK3HBJLFPC5X4L74Q332LNNEDYPFDV6SHWCQKRCNQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.330675401 (459h21m0.927446433s) [noise=true , address=64.49.13.26:4955 , piece_id=VIN3RVGRSUUHYCHPXCDNVSPRLFSCH3JXIDERBV7NIM4K33ZUBUGQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.330936609 (459h21m0.927707633s) [noise=true , address=193.149.164.68:4050 , piece_id=F4AQMCLJL6ML2LQKWUIKJAN6C2Y3M7UVYXRCRVRON5BC6CLTEK6Q, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.331355022 (459h21m0.928126051s) [noise=true , address=77.90.31.11:4567 , piece_id=ZSITPEN2OCYCHLSV3PDXCAXBAHEAQMS3YFCCOOE2SISJMUFRPHOA, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.402821184 (459h21m0.999592291s) [upload=514680]: Next returned nil segment
Mar 25 14:21:03.402848744 (459h21m0.99961978s) [upload=514680]: Waiting for error group managing segments...
Mar 25 14:21:03.408851473 (459h21m1.005622504s) [noise=true , address=77.90.31.16:4739 , piece_id=LEV3TZ2XFAYLQHULRJOG3DDBFMP3XRWX4EU2CQ2YOFICYXLIR3SA, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.409008646 (459h21m1.005779679s) [noise=true , address=145.79.227.34:4916 , piece_id=645H2KOPV4YB2HTP3XZF2RF5ZICIR34QNYRNRKX5KHH2VANVVZOQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.418584053 (459h21m1.015355084s) [noise=true , address=77.90.31.17:4791 , piece_id=L5H6QECJGEQPTPRB6INQWHZPMK7ZXXDYYFFG5ANE37HAE33OTVXQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.446927214 (459h21m1.043698248s) [noise=true , address=45.91.116.7:4023 , piece_id=GYWVVXKLOAZSDJIS64INTFQIPQZCLBH7YPLTWY7QJSB766A3QTGQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.448923805 (459h21m1.045694815s) [noise=true , address=77.90.31.14:4688 , piece_id=XDS6GE2YZK3LWUTBOMHLWTHM4ASCQJI25WTENUU75TJGFULYPOCQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.458340727 (459h21m1.055111751s) [noise=true , address=45.91.116.12:4254 , piece_id=UIQ3FWZSZ52RJYWFVWNH3OGNQPZSDYEZEDX35K3R7SDKRIW3WOAQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.468833670 (459h21m1.065604702s) [noise=true , address=77.90.31.11:4567 , piece_id=ZSITPEN2OCYCHLSV3PDXCAXBAHEAQMS3YFCCOOE2SISJMUFRPHOA, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.925971986 (459h21m1.522743012s) [noise=true , address=193.149.164.66:4984 , piece_id=CGM44RIHZGETSCKRZQOFL3VYIJTSB57RQDXTG3OPX3F3HJNQDYZA, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.928271525 (459h21m1.525042552s) [noise=true , address=64.49.13.140:4509 , piece_id=7HN3FXTKC4RRUJZ34AHFDJHMUBIOWIQVFST626CR7X2ZYQS3CFCQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.929087375 (459h21m1.525858388s) [noise=true , address=193.149.164.70:4120 , piece_id=HMFOJCRQTG3PAQDIZERNEYWKFFBUIRKRXP56OQUQWH6R2YX6NYLQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.929851130 (459h21m1.526622145s) [noise=true , address=193.149.164.71:4296 , piece_id=RXZXV3FCVDEFHBJQVGIWEKS57EJ3IKAJFTDGMOEZHRRBHV3NRDSA, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.935607616 (459h21m1.532378643s) [noise=true , address=193.149.164.69:4085 , piece_id=KGCVBEK344OW573PN2Y55JA7AC3ZTQD36G3T6TUHF7ZMZ4WN7N5A, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.938874518 (459h21m1.535645546s) [noise=true , address=193.149.164.67:4030 , piece_id=PBIDHXYQ22HHK3HBJLFPC5X4L74Q332LNNEDYPFDV6SHWCQKRCNQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.950243460 (459h21m1.547014493s) [noise=true , address=64.49.13.26:4955 , piece_id=VIN3RVGRSUUHYCHPXCDNVSPRLFSCH3JXIDERBV7NIM4K33ZUBUGQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.950277580 (459h21m1.54704861s) [seg_pos={0 0}, upload=514680]: Satisfied cohort requirements after 14 pieces
Mar 25 14:21:03.950707500 (459h21m1.547478532s) [noise=true , address=193.149.164.68:4050 , piece_id=F4AQMCLJL6ML2LQKWUIKJAN6C2Y3M7UVYXRCRVRON5BC6CLTEK6Q, seg_pos={0 0}, upload=514680]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1WPfYrCJrhPTqZ2qgBvXFCUodKz6QeXCnkKgToFs5muNRSSLtK): protocol: expected piece hash; context canceled; EOF
Mar 25 14:21:03.950721648 (459h21m1.547492674s) [noise=true , address=193.149.164.68:4050 , piece_id=F4AQMCLJL6ML2LQKWUIKJAN6C2Y3M7UVYXRCRVRON5BC6CLTEK6Q, seg_pos={0 0}, upload=514680]: Long tail cancellation due to optimal threshold
Mar 25 14:21:03.950793002 (459h21m1.547564034s) [noise=true , address=64.49.13.27:4964 , piece_id=UOBMO3UGR2SVYEXRSW2SQMHSDJV47OXOZRZJHCOHVWNXMIIDSA2A, seg_pos={0 0}, upload=514680]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RcgfaeKP3XpZGuSB9GMURCCsjjc79WeZasUL2RRiQVDwxbnAk): protocol: expected piece hash; context canceled; EOF
Mar 25 14:21:03.950804838 (459h21m1.547575864s) [noise=true , address=64.49.13.27:4964 , piece_id=UOBMO3UGR2SVYEXRSW2SQMHSDJV47OXOZRZJHCOHVWNXMIIDSA2A, seg_pos={0 0}, upload=514680]: Long tail cancellation due to optimal threshold
Mar 25 14:21:03.950899119 (459h21m1.547670144s) [noise=true , address=145.79.227.44:4201 , piece_id=SHQIEKFZSOKO7SUBGDC7GSJQNL47IEWENP5KN5ZN5ZOS4K3R7Y7A, seg_pos={0 0}, upload=514680]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bPDvhDfu8Z58LxZcicg1vAiiu1sZYGpwQnvuhk4t5m6yR87Fi): protocol: expected piece hash; context canceled; EOF
Mar 25 14:21:03.950907089 (459h21m1.547678114s) [noise=true , address=145.79.227.44:4201 , piece_id=SHQIEKFZSOKO7SUBGDC7GSJQNL47IEWENP5KN5ZN5ZOS4K3R7Y7A, seg_pos={0 0}, upload=514680]: Long tail cancellation due to optimal threshold
Mar 25 14:21:03.951133312 (459h21m1.547904347s) [noise=true , address=64.49.13.138:4487 , piece_id=NMZRB7X7MX2KJRDEUBROG3F6ZPQOGYJLTJ5WHBBFOUQCKXIHIZKQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12taemzrausuc58d8Dyo8xaR7MqDf72f37rEheF32sUfy8Z8GZP): protocol: expected piece hash; context canceled; EOF
Mar 25 14:21:03.951146101 (459h21m1.547917132s) [noise=true , address=64.49.13.138:4487 , piece_id=NMZRB7X7MX2KJRDEUBROG3F6ZPQOGYJLTJ5WHBBFOUQCKXIHIZKQ, seg_pos={0 0}, upload=514680]: Long tail cancellation due to optimal threshold
Mar 25 14:21:03.951175027 (459h21m1.547946039s) [seg_pos={0 0}, upload=514680]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Mar 25 14:21:03.951282270 (459h21m1.548053293s) [upload=514680]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Mar 25 14:21:03.951290421 (459h21m1.548061457s) [upload=514680]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Mar 25 14:21:04.013599996 (459h21m1.610371022s) [upload=514680]: Done uploading segments...
Mar 25 14:21:04.013621043 (459h21m1.610392066s) [upload=514680]: Upload finished. err: <nil>
Duration: 1.465826201s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
35.69034ms target:
https://us1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/21096334d2fd8925
FAIL (2026-03-25 15:05:32 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.4.32.59:443: i/o timeout
Duration: 35.745092ms
core
satellite
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [........X...................................................] duration:
1.549082247s trace:
https://jaeger.dev.storj.io/trace/62357518aac8ca5f
FAIL (2026-03-25 15:05:20 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 1.549114467s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
1.431700594s trace:
https://jaeger.dev.storj.io/trace/1d1f0bc4e4438495
FAIL (2026-03-25 15:05:18 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
upload=514680]: Uploading piece...
Mar 25 14:21:03.327216643 (459h21m0.923987655s) [noise=true , address=45.91.116.12:4254 , piece_id=UIQ3FWZSZ52RJYWFVWNH3OGNQPZSDYEZEDX35K3R7SDKRIW3WOAQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.327580267 (459h21m0.924351301s) [noise=true , address=193.149.164.70:4120 , piece_id=HMFOJCRQTG3PAQDIZERNEYWKFFBUIRKRXP56OQUQWH6R2YX6NYLQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.327966258 (459h21m0.924737281s) [noise=true , address=77.90.31.16:4739 , piece_id=LEV3TZ2XFAYLQHULRJOG3DDBFMP3XRWX4EU2CQ2YOFICYXLIR3SA, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.328304150 (459h21m0.925075204s) [noise=true , address=77.90.31.14:4688 , piece_id=XDS6GE2YZK3LWUTBOMHLWTHM4ASCQJI25WTENUU75TJGFULYPOCQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.328528343 (459h21m0.925299366s) [noise=true , address=64.49.13.138:4487 , piece_id=NMZRB7X7MX2KJRDEUBROG3F6ZPQOGYJLTJ5WHBBFOUQCKXIHIZKQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.328832656 (459h21m0.925603704s) [noise=true , address=145.79.227.44:4201 , piece_id=SHQIEKFZSOKO7SUBGDC7GSJQNL47IEWENP5KN5ZN5ZOS4K3R7Y7A, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.328990765 (459h21m0.925761788s) [noise=true , address=77.90.31.17:4791 , piece_id=L5H6QECJGEQPTPRB6INQWHZPMK7ZXXDYYFFG5ANE37HAE33OTVXQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.329286413 (459h21m0.926057443s) [noise=true , address=64.49.13.27:4964 , piece_id=UOBMO3UGR2SVYEXRSW2SQMHSDJV47OXOZRZJHCOHVWNXMIIDSA2A, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.329346877 (459h21m0.9261179s) [noise=true , address=193.149.164.66:4984 , piece_id=CGM44RIHZGETSCKRZQOFL3VYIJTSB57RQDXTG3OPX3F3HJNQDYZA, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.329706811 (459h21m0.926477834s) [noise=true , address=45.91.116.7:4023 , piece_id=GYWVVXKLOAZSDJIS64INTFQIPQZCLBH7YPLTWY7QJSB766A3QTGQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.329728635 (459h21m0.926499661s) [noise=true , address=64.49.13.140:4509 , piece_id=7HN3FXTKC4RRUJZ34AHFDJHMUBIOWIQVFST626CR7X2ZYQS3CFCQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.330142941 (459h21m0.926913971s) [noise=true , address=145.79.227.34:4916 , piece_id=645H2KOPV4YB2HTP3XZF2RF5ZICIR34QNYRNRKX5KHH2VANVVZOQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.330171985 (459h21m0.926943015s) [noise=true , address=193.149.164.71:4296 , piece_id=RXZXV3FCVDEFHBJQVGIWEKS57EJ3IKAJFTDGMOEZHRRBHV3NRDSA, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.330543280 (459h21m0.927314303s) [noise=true , address=193.149.164.67:4030 , piece_id=PBIDHXYQ22HHK3HBJLFPC5X4L74Q332LNNEDYPFDV6SHWCQKRCNQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.330675401 (459h21m0.927446433s) [noise=true , address=64.49.13.26:4955 , piece_id=VIN3RVGRSUUHYCHPXCDNVSPRLFSCH3JXIDERBV7NIM4K33ZUBUGQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.330936609 (459h21m0.927707633s) [noise=true , address=193.149.164.68:4050 , piece_id=F4AQMCLJL6ML2LQKWUIKJAN6C2Y3M7UVYXRCRVRON5BC6CLTEK6Q, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.331355022 (459h21m0.928126051s) [noise=true , address=77.90.31.11:4567 , piece_id=ZSITPEN2OCYCHLSV3PDXCAXBAHEAQMS3YFCCOOE2SISJMUFRPHOA, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.402821184 (459h21m0.999592291s) [upload=514680]: Next returned nil segment
Mar 25 14:21:03.402848744 (459h21m0.99961978s) [upload=514680]: Waiting for error group managing segments...
Mar 25 14:21:03.408851473 (459h21m1.005622504s) [noise=true , address=77.90.31.16:4739 , piece_id=LEV3TZ2XFAYLQHULRJOG3DDBFMP3XRWX4EU2CQ2YOFICYXLIR3SA, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.409008646 (459h21m1.005779679s) [noise=true , address=145.79.227.34:4916 , piece_id=645H2KOPV4YB2HTP3XZF2RF5ZICIR34QNYRNRKX5KHH2VANVVZOQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.418584053 (459h21m1.015355084s) [noise=true , address=77.90.31.17:4791 , piece_id=L5H6QECJGEQPTPRB6INQWHZPMK7ZXXDYYFFG5ANE37HAE33OTVXQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.446927214 (459h21m1.043698248s) [noise=true , address=45.91.116.7:4023 , piece_id=GYWVVXKLOAZSDJIS64INTFQIPQZCLBH7YPLTWY7QJSB766A3QTGQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.448923805 (459h21m1.045694815s) [noise=true , address=77.90.31.14:4688 , piece_id=XDS6GE2YZK3LWUTBOMHLWTHM4ASCQJI25WTENUU75TJGFULYPOCQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.458340727 (459h21m1.055111751s) [noise=true , address=45.91.116.12:4254 , piece_id=UIQ3FWZSZ52RJYWFVWNH3OGNQPZSDYEZEDX35K3R7SDKRIW3WOAQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.468833670 (459h21m1.065604702s) [noise=true , address=77.90.31.11:4567 , piece_id=ZSITPEN2OCYCHLSV3PDXCAXBAHEAQMS3YFCCOOE2SISJMUFRPHOA, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.925971986 (459h21m1.522743012s) [noise=true , address=193.149.164.66:4984 , piece_id=CGM44RIHZGETSCKRZQOFL3VYIJTSB57RQDXTG3OPX3F3HJNQDYZA, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.928271525 (459h21m1.525042552s) [noise=true , address=64.49.13.140:4509 , piece_id=7HN3FXTKC4RRUJZ34AHFDJHMUBIOWIQVFST626CR7X2ZYQS3CFCQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.929087375 (459h21m1.525858388s) [noise=true , address=193.149.164.70:4120 , piece_id=HMFOJCRQTG3PAQDIZERNEYWKFFBUIRKRXP56OQUQWH6R2YX6NYLQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.929851130 (459h21m1.526622145s) [noise=true , address=193.149.164.71:4296 , piece_id=RXZXV3FCVDEFHBJQVGIWEKS57EJ3IKAJFTDGMOEZHRRBHV3NRDSA, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.935607616 (459h21m1.532378643s) [noise=true , address=193.149.164.69:4085 , piece_id=KGCVBEK344OW573PN2Y55JA7AC3ZTQD36G3T6TUHF7ZMZ4WN7N5A, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.938874518 (459h21m1.535645546s) [noise=true , address=193.149.164.67:4030 , piece_id=PBIDHXYQ22HHK3HBJLFPC5X4L74Q332LNNEDYPFDV6SHWCQKRCNQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.950243460 (459h21m1.547014493s) [noise=true , address=64.49.13.26:4955 , piece_id=VIN3RVGRSUUHYCHPXCDNVSPRLFSCH3JXIDERBV7NIM4K33ZUBUGQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.950277580 (459h21m1.54704861s) [seg_pos={0 0}, upload=514680]: Satisfied cohort requirements after 14 pieces
Mar 25 14:21:03.950707500 (459h21m1.547478532s) [noise=true , address=193.149.164.68:4050 , piece_id=F4AQMCLJL6ML2LQKWUIKJAN6C2Y3M7UVYXRCRVRON5BC6CLTEK6Q, seg_pos={0 0}, upload=514680]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1WPfYrCJrhPTqZ2qgBvXFCUodKz6QeXCnkKgToFs5muNRSSLtK): protocol: expected piece hash; context canceled; EOF
Mar 25 14:21:03.950721648 (459h21m1.547492674s) [noise=true , address=193.149.164.68:4050 , piece_id=F4AQMCLJL6ML2LQKWUIKJAN6C2Y3M7UVYXRCRVRON5BC6CLTEK6Q, seg_pos={0 0}, upload=514680]: Long tail cancellation due to optimal threshold
Mar 25 14:21:03.950793002 (459h21m1.547564034s) [noise=true , address=64.49.13.27:4964 , piece_id=UOBMO3UGR2SVYEXRSW2SQMHSDJV47OXOZRZJHCOHVWNXMIIDSA2A, seg_pos={0 0}, upload=514680]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RcgfaeKP3XpZGuSB9GMURCCsjjc79WeZasUL2RRiQVDwxbnAk): protocol: expected piece hash; context canceled; EOF
Mar 25 14:21:03.950804838 (459h21m1.547575864s) [noise=true , address=64.49.13.27:4964 , piece_id=UOBMO3UGR2SVYEXRSW2SQMHSDJV47OXOZRZJHCOHVWNXMIIDSA2A, seg_pos={0 0}, upload=514680]: Long tail cancellation due to optimal threshold
Mar 25 14:21:03.950899119 (459h21m1.547670144s) [noise=true , address=145.79.227.44:4201 , piece_id=SHQIEKFZSOKO7SUBGDC7GSJQNL47IEWENP5KN5ZN5ZOS4K3R7Y7A, seg_pos={0 0}, upload=514680]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bPDvhDfu8Z58LxZcicg1vAiiu1sZYGpwQnvuhk4t5m6yR87Fi): protocol: expected piece hash; context canceled; EOF
Mar 25 14:21:03.950907089 (459h21m1.547678114s) [noise=true , address=145.79.227.44:4201 , piece_id=SHQIEKFZSOKO7SUBGDC7GSJQNL47IEWENP5KN5ZN5ZOS4K3R7Y7A, seg_pos={0 0}, upload=514680]: Long tail cancellation due to optimal threshold
Mar 25 14:21:03.951133312 (459h21m1.547904347s) [noise=true , address=64.49.13.138:4487 , piece_id=NMZRB7X7MX2KJRDEUBROG3F6ZPQOGYJLTJ5WHBBFOUQCKXIHIZKQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12taemzrausuc58d8Dyo8xaR7MqDf72f37rEheF32sUfy8Z8GZP): protocol: expected piece hash; context canceled; EOF
Mar 25 14:21:03.951146101 (459h21m1.547917132s) [noise=true , address=64.49.13.138:4487 , piece_id=NMZRB7X7MX2KJRDEUBROG3F6ZPQOGYJLTJ5WHBBFOUQCKXIHIZKQ, seg_pos={0 0}, upload=514680]: Long tail cancellation due to optimal threshold
Mar 25 14:21:03.951175027 (459h21m1.547946039s) [seg_pos={0 0}, upload=514680]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Mar 25 14:21:03.951282270 (459h21m1.548053293s) [upload=514680]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Mar 25 14:21:03.951290421 (459h21m1.548061457s) [upload=514680]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Mar 25 14:21:04.013599996 (459h21m1.610371022s) [upload=514680]: Done uploading segments...
Mar 25 14:21:04.013621043 (459h21m1.610392066s) [upload=514680]: Upload finished. err: <nil>
Duration: 1.431850757s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........X..................................................] duration:
1.712824202s trace:
https://jaeger.dev.storj.io/trace/62357518aac8ca5f
FAIL (2026-03-25 15:04:48 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 1.712890066s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
71.296773ms target:
https://us1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/21096334d2fd8925
FAIL (2026-03-25 15:04:33 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.4.32.59:443: i/o timeout
Duration: 71.316078ms
core
satellite
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................] duration:
1.381700791s trace:
https://jaeger.dev.storj.io/trace/1d1f0bc4e4438495
FAIL (2026-03-25 15:04:09 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
upload=514680]: Uploading piece...
Mar 25 14:21:03.327216643 (459h21m0.923987655s) [noise=true , address=45.91.116.12:4254 , piece_id=UIQ3FWZSZ52RJYWFVWNH3OGNQPZSDYEZEDX35K3R7SDKRIW3WOAQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.327580267 (459h21m0.924351301s) [noise=true , address=193.149.164.70:4120 , piece_id=HMFOJCRQTG3PAQDIZERNEYWKFFBUIRKRXP56OQUQWH6R2YX6NYLQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.327966258 (459h21m0.924737281s) [noise=true , address=77.90.31.16:4739 , piece_id=LEV3TZ2XFAYLQHULRJOG3DDBFMP3XRWX4EU2CQ2YOFICYXLIR3SA, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.328304150 (459h21m0.925075204s) [noise=true , address=77.90.31.14:4688 , piece_id=XDS6GE2YZK3LWUTBOMHLWTHM4ASCQJI25WTENUU75TJGFULYPOCQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.328528343 (459h21m0.925299366s) [noise=true , address=64.49.13.138:4487 , piece_id=NMZRB7X7MX2KJRDEUBROG3F6ZPQOGYJLTJ5WHBBFOUQCKXIHIZKQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.328832656 (459h21m0.925603704s) [noise=true , address=145.79.227.44:4201 , piece_id=SHQIEKFZSOKO7SUBGDC7GSJQNL47IEWENP5KN5ZN5ZOS4K3R7Y7A, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.328990765 (459h21m0.925761788s) [noise=true , address=77.90.31.17:4791 , piece_id=L5H6QECJGEQPTPRB6INQWHZPMK7ZXXDYYFFG5ANE37HAE33OTVXQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.329286413 (459h21m0.926057443s) [noise=true , address=64.49.13.27:4964 , piece_id=UOBMO3UGR2SVYEXRSW2SQMHSDJV47OXOZRZJHCOHVWNXMIIDSA2A, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.329346877 (459h21m0.9261179s) [noise=true , address=193.149.164.66:4984 , piece_id=CGM44RIHZGETSCKRZQOFL3VYIJTSB57RQDXTG3OPX3F3HJNQDYZA, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.329706811 (459h21m0.926477834s) [noise=true , address=45.91.116.7:4023 , piece_id=GYWVVXKLOAZSDJIS64INTFQIPQZCLBH7YPLTWY7QJSB766A3QTGQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.329728635 (459h21m0.926499661s) [noise=true , address=64.49.13.140:4509 , piece_id=7HN3FXTKC4RRUJZ34AHFDJHMUBIOWIQVFST626CR7X2ZYQS3CFCQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.330142941 (459h21m0.926913971s) [noise=true , address=145.79.227.34:4916 , piece_id=645H2KOPV4YB2HTP3XZF2RF5ZICIR34QNYRNRKX5KHH2VANVVZOQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.330171985 (459h21m0.926943015s) [noise=true , address=193.149.164.71:4296 , piece_id=RXZXV3FCVDEFHBJQVGIWEKS57EJ3IKAJFTDGMOEZHRRBHV3NRDSA, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.330543280 (459h21m0.927314303s) [noise=true , address=193.149.164.67:4030 , piece_id=PBIDHXYQ22HHK3HBJLFPC5X4L74Q332LNNEDYPFDV6SHWCQKRCNQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.330675401 (459h21m0.927446433s) [noise=true , address=64.49.13.26:4955 , piece_id=VIN3RVGRSUUHYCHPXCDNVSPRLFSCH3JXIDERBV7NIM4K33ZUBUGQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.330936609 (459h21m0.927707633s) [noise=true , address=193.149.164.68:4050 , piece_id=F4AQMCLJL6ML2LQKWUIKJAN6C2Y3M7UVYXRCRVRON5BC6CLTEK6Q, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.331355022 (459h21m0.928126051s) [noise=true , address=77.90.31.11:4567 , piece_id=ZSITPEN2OCYCHLSV3PDXCAXBAHEAQMS3YFCCOOE2SISJMUFRPHOA, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.402821184 (459h21m0.999592291s) [upload=514680]: Next returned nil segment
Mar 25 14:21:03.402848744 (459h21m0.99961978s) [upload=514680]: Waiting for error group managing segments...
Mar 25 14:21:03.408851473 (459h21m1.005622504s) [noise=true , address=77.90.31.16:4739 , piece_id=LEV3TZ2XFAYLQHULRJOG3DDBFMP3XRWX4EU2CQ2YOFICYXLIR3SA, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.409008646 (459h21m1.005779679s) [noise=true , address=145.79.227.34:4916 , piece_id=645H2KOPV4YB2HTP3XZF2RF5ZICIR34QNYRNRKX5KHH2VANVVZOQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.418584053 (459h21m1.015355084s) [noise=true , address=77.90.31.17:4791 , piece_id=L5H6QECJGEQPTPRB6INQWHZPMK7ZXXDYYFFG5ANE37HAE33OTVXQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.446927214 (459h21m1.043698248s) [noise=true , address=45.91.116.7:4023 , piece_id=GYWVVXKLOAZSDJIS64INTFQIPQZCLBH7YPLTWY7QJSB766A3QTGQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.448923805 (459h21m1.045694815s) [noise=true , address=77.90.31.14:4688 , piece_id=XDS6GE2YZK3LWUTBOMHLWTHM4ASCQJI25WTENUU75TJGFULYPOCQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.458340727 (459h21m1.055111751s) [noise=true , address=45.91.116.12:4254 , piece_id=UIQ3FWZSZ52RJYWFVWNH3OGNQPZSDYEZEDX35K3R7SDKRIW3WOAQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.468833670 (459h21m1.065604702s) [noise=true , address=77.90.31.11:4567 , piece_id=ZSITPEN2OCYCHLSV3PDXCAXBAHEAQMS3YFCCOOE2SISJMUFRPHOA, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.925971986 (459h21m1.522743012s) [noise=true , address=193.149.164.66:4984 , piece_id=CGM44RIHZGETSCKRZQOFL3VYIJTSB57RQDXTG3OPX3F3HJNQDYZA, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.928271525 (459h21m1.525042552s) [noise=true , address=64.49.13.140:4509 , piece_id=7HN3FXTKC4RRUJZ34AHFDJHMUBIOWIQVFST626CR7X2ZYQS3CFCQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.929087375 (459h21m1.525858388s) [noise=true , address=193.149.164.70:4120 , piece_id=HMFOJCRQTG3PAQDIZERNEYWKFFBUIRKRXP56OQUQWH6R2YX6NYLQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.929851130 (459h21m1.526622145s) [noise=true , address=193.149.164.71:4296 , piece_id=RXZXV3FCVDEFHBJQVGIWEKS57EJ3IKAJFTDGMOEZHRRBHV3NRDSA, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.935607616 (459h21m1.532378643s) [noise=true , address=193.149.164.69:4085 , piece_id=KGCVBEK344OW573PN2Y55JA7AC3ZTQD36G3T6TUHF7ZMZ4WN7N5A, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.938874518 (459h21m1.535645546s) [noise=true , address=193.149.164.67:4030 , piece_id=PBIDHXYQ22HHK3HBJLFPC5X4L74Q332LNNEDYPFDV6SHWCQKRCNQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.950243460 (459h21m1.547014493s) [noise=true , address=64.49.13.26:4955 , piece_id=VIN3RVGRSUUHYCHPXCDNVSPRLFSCH3JXIDERBV7NIM4K33ZUBUGQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.950277580 (459h21m1.54704861s) [seg_pos={0 0}, upload=514680]: Satisfied cohort requirements after 14 pieces
Mar 25 14:21:03.950707500 (459h21m1.547478532s) [noise=true , address=193.149.164.68:4050 , piece_id=F4AQMCLJL6ML2LQKWUIKJAN6C2Y3M7UVYXRCRVRON5BC6CLTEK6Q, seg_pos={0 0}, upload=514680]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1WPfYrCJrhPTqZ2qgBvXFCUodKz6QeXCnkKgToFs5muNRSSLtK): protocol: expected piece hash; context canceled; EOF
Mar 25 14:21:03.950721648 (459h21m1.547492674s) [noise=true , address=193.149.164.68:4050 , piece_id=F4AQMCLJL6ML2LQKWUIKJAN6C2Y3M7UVYXRCRVRON5BC6CLTEK6Q, seg_pos={0 0}, upload=514680]: Long tail cancellation due to optimal threshold
Mar 25 14:21:03.950793002 (459h21m1.547564034s) [noise=true , address=64.49.13.27:4964 , piece_id=UOBMO3UGR2SVYEXRSW2SQMHSDJV47OXOZRZJHCOHVWNXMIIDSA2A, seg_pos={0 0}, upload=514680]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RcgfaeKP3XpZGuSB9GMURCCsjjc79WeZasUL2RRiQVDwxbnAk): protocol: expected piece hash; context canceled; EOF
Mar 25 14:21:03.950804838 (459h21m1.547575864s) [noise=true , address=64.49.13.27:4964 , piece_id=UOBMO3UGR2SVYEXRSW2SQMHSDJV47OXOZRZJHCOHVWNXMIIDSA2A, seg_pos={0 0}, upload=514680]: Long tail cancellation due to optimal threshold
Mar 25 14:21:03.950899119 (459h21m1.547670144s) [noise=true , address=145.79.227.44:4201 , piece_id=SHQIEKFZSOKO7SUBGDC7GSJQNL47IEWENP5KN5ZN5ZOS4K3R7Y7A, seg_pos={0 0}, upload=514680]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bPDvhDfu8Z58LxZcicg1vAiiu1sZYGpwQnvuhk4t5m6yR87Fi): protocol: expected piece hash; context canceled; EOF
Mar 25 14:21:03.950907089 (459h21m1.547678114s) [noise=true , address=145.79.227.44:4201 , piece_id=SHQIEKFZSOKO7SUBGDC7GSJQNL47IEWENP5KN5ZN5ZOS4K3R7Y7A, seg_pos={0 0}, upload=514680]: Long tail cancellation due to optimal threshold
Mar 25 14:21:03.951133312 (459h21m1.547904347s) [noise=true , address=64.49.13.138:4487 , piece_id=NMZRB7X7MX2KJRDEUBROG3F6ZPQOGYJLTJ5WHBBFOUQCKXIHIZKQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12taemzrausuc58d8Dyo8xaR7MqDf72f37rEheF32sUfy8Z8GZP): protocol: expected piece hash; context canceled; EOF
Mar 25 14:21:03.951146101 (459h21m1.547917132s) [noise=true , address=64.49.13.138:4487 , piece_id=NMZRB7X7MX2KJRDEUBROG3F6ZPQOGYJLTJ5WHBBFOUQCKXIHIZKQ, seg_pos={0 0}, upload=514680]: Long tail cancellation due to optimal threshold
Mar 25 14:21:03.951175027 (459h21m1.547946039s) [seg_pos={0 0}, upload=514680]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Mar 25 14:21:03.951282270 (459h21m1.548053293s) [upload=514680]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Mar 25 14:21:03.951290421 (459h21m1.548061457s) [upload=514680]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Mar 25 14:21:04.013599996 (459h21m1.610371022s) [upload=514680]: Done uploading segments...
Mar 25 14:21:04.013621043 (459h21m1.610392066s) [upload=514680]: Upload finished. err: <nil>
Duration: 1.381856479s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
29.899011ms target:
https://us1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/21096334d2fd8925
FAIL (2026-03-25 15:03:40 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.4.32.59:443: i/o timeout
Duration: 29.923852ms
core
satellite
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........X.................................................] duration:
1.243754236s trace:
https://jaeger.dev.storj.io/trace/62357518aac8ca5f
FAIL (2026-03-25 15:03:38 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 1.24387637s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
1.50940355s trace:
https://jaeger.dev.storj.io/trace/1d1f0bc4e4438495
FAIL (2026-03-25 15:03:34 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
upload=514680]: Uploading piece...
Mar 25 14:21:03.327216643 (459h21m0.923987655s) [noise=true , address=45.91.116.12:4254 , piece_id=UIQ3FWZSZ52RJYWFVWNH3OGNQPZSDYEZEDX35K3R7SDKRIW3WOAQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.327580267 (459h21m0.924351301s) [noise=true , address=193.149.164.70:4120 , piece_id=HMFOJCRQTG3PAQDIZERNEYWKFFBUIRKRXP56OQUQWH6R2YX6NYLQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.327966258 (459h21m0.924737281s) [noise=true , address=77.90.31.16:4739 , piece_id=LEV3TZ2XFAYLQHULRJOG3DDBFMP3XRWX4EU2CQ2YOFICYXLIR3SA, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.328304150 (459h21m0.925075204s) [noise=true , address=77.90.31.14:4688 , piece_id=XDS6GE2YZK3LWUTBOMHLWTHM4ASCQJI25WTENUU75TJGFULYPOCQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.328528343 (459h21m0.925299366s) [noise=true , address=64.49.13.138:4487 , piece_id=NMZRB7X7MX2KJRDEUBROG3F6ZPQOGYJLTJ5WHBBFOUQCKXIHIZKQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.328832656 (459h21m0.925603704s) [noise=true , address=145.79.227.44:4201 , piece_id=SHQIEKFZSOKO7SUBGDC7GSJQNL47IEWENP5KN5ZN5ZOS4K3R7Y7A, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.328990765 (459h21m0.925761788s) [noise=true , address=77.90.31.17:4791 , piece_id=L5H6QECJGEQPTPRB6INQWHZPMK7ZXXDYYFFG5ANE37HAE33OTVXQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.329286413 (459h21m0.926057443s) [noise=true , address=64.49.13.27:4964 , piece_id=UOBMO3UGR2SVYEXRSW2SQMHSDJV47OXOZRZJHCOHVWNXMIIDSA2A, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.329346877 (459h21m0.9261179s) [noise=true , address=193.149.164.66:4984 , piece_id=CGM44RIHZGETSCKRZQOFL3VYIJTSB57RQDXTG3OPX3F3HJNQDYZA, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.329706811 (459h21m0.926477834s) [noise=true , address=45.91.116.7:4023 , piece_id=GYWVVXKLOAZSDJIS64INTFQIPQZCLBH7YPLTWY7QJSB766A3QTGQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.329728635 (459h21m0.926499661s) [noise=true , address=64.49.13.140:4509 , piece_id=7HN3FXTKC4RRUJZ34AHFDJHMUBIOWIQVFST626CR7X2ZYQS3CFCQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.330142941 (459h21m0.926913971s) [noise=true , address=145.79.227.34:4916 , piece_id=645H2KOPV4YB2HTP3XZF2RF5ZICIR34QNYRNRKX5KHH2VANVVZOQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.330171985 (459h21m0.926943015s) [noise=true , address=193.149.164.71:4296 , piece_id=RXZXV3FCVDEFHBJQVGIWEKS57EJ3IKAJFTDGMOEZHRRBHV3NRDSA, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.330543280 (459h21m0.927314303s) [noise=true , address=193.149.164.67:4030 , piece_id=PBIDHXYQ22HHK3HBJLFPC5X4L74Q332LNNEDYPFDV6SHWCQKRCNQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.330675401 (459h21m0.927446433s) [noise=true , address=64.49.13.26:4955 , piece_id=VIN3RVGRSUUHYCHPXCDNVSPRLFSCH3JXIDERBV7NIM4K33ZUBUGQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.330936609 (459h21m0.927707633s) [noise=true , address=193.149.164.68:4050 , piece_id=F4AQMCLJL6ML2LQKWUIKJAN6C2Y3M7UVYXRCRVRON5BC6CLTEK6Q, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.331355022 (459h21m0.928126051s) [noise=true , address=77.90.31.11:4567 , piece_id=ZSITPEN2OCYCHLSV3PDXCAXBAHEAQMS3YFCCOOE2SISJMUFRPHOA, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.402821184 (459h21m0.999592291s) [upload=514680]: Next returned nil segment
Mar 25 14:21:03.402848744 (459h21m0.99961978s) [upload=514680]: Waiting for error group managing segments...
Mar 25 14:21:03.408851473 (459h21m1.005622504s) [noise=true , address=77.90.31.16:4739 , piece_id=LEV3TZ2XFAYLQHULRJOG3DDBFMP3XRWX4EU2CQ2YOFICYXLIR3SA, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.409008646 (459h21m1.005779679s) [noise=true , address=145.79.227.34:4916 , piece_id=645H2KOPV4YB2HTP3XZF2RF5ZICIR34QNYRNRKX5KHH2VANVVZOQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.418584053 (459h21m1.015355084s) [noise=true , address=77.90.31.17:4791 , piece_id=L5H6QECJGEQPTPRB6INQWHZPMK7ZXXDYYFFG5ANE37HAE33OTVXQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.446927214 (459h21m1.043698248s) [noise=true , address=45.91.116.7:4023 , piece_id=GYWVVXKLOAZSDJIS64INTFQIPQZCLBH7YPLTWY7QJSB766A3QTGQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.448923805 (459h21m1.045694815s) [noise=true , address=77.90.31.14:4688 , piece_id=XDS6GE2YZK3LWUTBOMHLWTHM4ASCQJI25WTENUU75TJGFULYPOCQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.458340727 (459h21m1.055111751s) [noise=true , address=45.91.116.12:4254 , piece_id=UIQ3FWZSZ52RJYWFVWNH3OGNQPZSDYEZEDX35K3R7SDKRIW3WOAQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.468833670 (459h21m1.065604702s) [noise=true , address=77.90.31.11:4567 , piece_id=ZSITPEN2OCYCHLSV3PDXCAXBAHEAQMS3YFCCOOE2SISJMUFRPHOA, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.925971986 (459h21m1.522743012s) [noise=true , address=193.149.164.66:4984 , piece_id=CGM44RIHZGETSCKRZQOFL3VYIJTSB57RQDXTG3OPX3F3HJNQDYZA, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.928271525 (459h21m1.525042552s) [noise=true , address=64.49.13.140:4509 , piece_id=7HN3FXTKC4RRUJZ34AHFDJHMUBIOWIQVFST626CR7X2ZYQS3CFCQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.929087375 (459h21m1.525858388s) [noise=true , address=193.149.164.70:4120 , piece_id=HMFOJCRQTG3PAQDIZERNEYWKFFBUIRKRXP56OQUQWH6R2YX6NYLQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.929851130 (459h21m1.526622145s) [noise=true , address=193.149.164.71:4296 , piece_id=RXZXV3FCVDEFHBJQVGIWEKS57EJ3IKAJFTDGMOEZHRRBHV3NRDSA, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.935607616 (459h21m1.532378643s) [noise=true , address=193.149.164.69:4085 , piece_id=KGCVBEK344OW573PN2Y55JA7AC3ZTQD36G3T6TUHF7ZMZ4WN7N5A, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.938874518 (459h21m1.535645546s) [noise=true , address=193.149.164.67:4030 , piece_id=PBIDHXYQ22HHK3HBJLFPC5X4L74Q332LNNEDYPFDV6SHWCQKRCNQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.950243460 (459h21m1.547014493s) [noise=true , address=64.49.13.26:4955 , piece_id=VIN3RVGRSUUHYCHPXCDNVSPRLFSCH3JXIDERBV7NIM4K33ZUBUGQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.950277580 (459h21m1.54704861s) [seg_pos={0 0}, upload=514680]: Satisfied cohort requirements after 14 pieces
Mar 25 14:21:03.950707500 (459h21m1.547478532s) [noise=true , address=193.149.164.68:4050 , piece_id=F4AQMCLJL6ML2LQKWUIKJAN6C2Y3M7UVYXRCRVRON5BC6CLTEK6Q, seg_pos={0 0}, upload=514680]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1WPfYrCJrhPTqZ2qgBvXFCUodKz6QeXCnkKgToFs5muNRSSLtK): protocol: expected piece hash; context canceled; EOF
Mar 25 14:21:03.950721648 (459h21m1.547492674s) [noise=true , address=193.149.164.68:4050 , piece_id=F4AQMCLJL6ML2LQKWUIKJAN6C2Y3M7UVYXRCRVRON5BC6CLTEK6Q, seg_pos={0 0}, upload=514680]: Long tail cancellation due to optimal threshold
Mar 25 14:21:03.950793002 (459h21m1.547564034s) [noise=true , address=64.49.13.27:4964 , piece_id=UOBMO3UGR2SVYEXRSW2SQMHSDJV47OXOZRZJHCOHVWNXMIIDSA2A, seg_pos={0 0}, upload=514680]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RcgfaeKP3XpZGuSB9GMURCCsjjc79WeZasUL2RRiQVDwxbnAk): protocol: expected piece hash; context canceled; EOF
Mar 25 14:21:03.950804838 (459h21m1.547575864s) [noise=true , address=64.49.13.27:4964 , piece_id=UOBMO3UGR2SVYEXRSW2SQMHSDJV47OXOZRZJHCOHVWNXMIIDSA2A, seg_pos={0 0}, upload=514680]: Long tail cancellation due to optimal threshold
Mar 25 14:21:03.950899119 (459h21m1.547670144s) [noise=true , address=145.79.227.44:4201 , piece_id=SHQIEKFZSOKO7SUBGDC7GSJQNL47IEWENP5KN5ZN5ZOS4K3R7Y7A, seg_pos={0 0}, upload=514680]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bPDvhDfu8Z58LxZcicg1vAiiu1sZYGpwQnvuhk4t5m6yR87Fi): protocol: expected piece hash; context canceled; EOF
Mar 25 14:21:03.950907089 (459h21m1.547678114s) [noise=true , address=145.79.227.44:4201 , piece_id=SHQIEKFZSOKO7SUBGDC7GSJQNL47IEWENP5KN5ZN5ZOS4K3R7Y7A, seg_pos={0 0}, upload=514680]: Long tail cancellation due to optimal threshold
Mar 25 14:21:03.951133312 (459h21m1.547904347s) [noise=true , address=64.49.13.138:4487 , piece_id=NMZRB7X7MX2KJRDEUBROG3F6ZPQOGYJLTJ5WHBBFOUQCKXIHIZKQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12taemzrausuc58d8Dyo8xaR7MqDf72f37rEheF32sUfy8Z8GZP): protocol: expected piece hash; context canceled; EOF
Mar 25 14:21:03.951146101 (459h21m1.547917132s) [noise=true , address=64.49.13.138:4487 , piece_id=NMZRB7X7MX2KJRDEUBROG3F6ZPQOGYJLTJ5WHBBFOUQCKXIHIZKQ, seg_pos={0 0}, upload=514680]: Long tail cancellation due to optimal threshold
Mar 25 14:21:03.951175027 (459h21m1.547946039s) [seg_pos={0 0}, upload=514680]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Mar 25 14:21:03.951282270 (459h21m1.548053293s) [upload=514680]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Mar 25 14:21:03.951290421 (459h21m1.548061457s) [upload=514680]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Mar 25 14:21:04.013599996 (459h21m1.610371022s) [upload=514680]: Done uploading segments...
Mar 25 14:21:04.013621043 (459h21m1.610392066s) [upload=514680]: Upload finished. err: <nil>
Duration: 1.509496512s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........X................................................] duration:
1.292699569s trace:
https://jaeger.dev.storj.io/trace/62357518aac8ca5f
FAIL (2026-03-25 15:02:53 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 1.292731993s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................] duration:
36.364313ms target:
https://us1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/21096334d2fd8925
FAIL (2026-03-25 15:02:36 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.4.32.59:443: i/o timeout
Duration: 36.396961ms
core
satellite
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................] duration:
1.40962902s trace:
https://jaeger.dev.storj.io/trace/1d1f0bc4e4438495
FAIL (2026-03-25 15:02:26 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
upload=514680]: Uploading piece...
Mar 25 14:21:03.327216643 (459h21m0.923987655s) [noise=true , address=45.91.116.12:4254 , piece_id=UIQ3FWZSZ52RJYWFVWNH3OGNQPZSDYEZEDX35K3R7SDKRIW3WOAQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.327580267 (459h21m0.924351301s) [noise=true , address=193.149.164.70:4120 , piece_id=HMFOJCRQTG3PAQDIZERNEYWKFFBUIRKRXP56OQUQWH6R2YX6NYLQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.327966258 (459h21m0.924737281s) [noise=true , address=77.90.31.16:4739 , piece_id=LEV3TZ2XFAYLQHULRJOG3DDBFMP3XRWX4EU2CQ2YOFICYXLIR3SA, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.328304150 (459h21m0.925075204s) [noise=true , address=77.90.31.14:4688 , piece_id=XDS6GE2YZK3LWUTBOMHLWTHM4ASCQJI25WTENUU75TJGFULYPOCQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.328528343 (459h21m0.925299366s) [noise=true , address=64.49.13.138:4487 , piece_id=NMZRB7X7MX2KJRDEUBROG3F6ZPQOGYJLTJ5WHBBFOUQCKXIHIZKQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.328832656 (459h21m0.925603704s) [noise=true , address=145.79.227.44:4201 , piece_id=SHQIEKFZSOKO7SUBGDC7GSJQNL47IEWENP5KN5ZN5ZOS4K3R7Y7A, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.328990765 (459h21m0.925761788s) [noise=true , address=77.90.31.17:4791 , piece_id=L5H6QECJGEQPTPRB6INQWHZPMK7ZXXDYYFFG5ANE37HAE33OTVXQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.329286413 (459h21m0.926057443s) [noise=true , address=64.49.13.27:4964 , piece_id=UOBMO3UGR2SVYEXRSW2SQMHSDJV47OXOZRZJHCOHVWNXMIIDSA2A, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.329346877 (459h21m0.9261179s) [noise=true , address=193.149.164.66:4984 , piece_id=CGM44RIHZGETSCKRZQOFL3VYIJTSB57RQDXTG3OPX3F3HJNQDYZA, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.329706811 (459h21m0.926477834s) [noise=true , address=45.91.116.7:4023 , piece_id=GYWVVXKLOAZSDJIS64INTFQIPQZCLBH7YPLTWY7QJSB766A3QTGQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.329728635 (459h21m0.926499661s) [noise=true , address=64.49.13.140:4509 , piece_id=7HN3FXTKC4RRUJZ34AHFDJHMUBIOWIQVFST626CR7X2ZYQS3CFCQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.330142941 (459h21m0.926913971s) [noise=true , address=145.79.227.34:4916 , piece_id=645H2KOPV4YB2HTP3XZF2RF5ZICIR34QNYRNRKX5KHH2VANVVZOQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.330171985 (459h21m0.926943015s) [noise=true , address=193.149.164.71:4296 , piece_id=RXZXV3FCVDEFHBJQVGIWEKS57EJ3IKAJFTDGMOEZHRRBHV3NRDSA, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.330543280 (459h21m0.927314303s) [noise=true , address=193.149.164.67:4030 , piece_id=PBIDHXYQ22HHK3HBJLFPC5X4L74Q332LNNEDYPFDV6SHWCQKRCNQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.330675401 (459h21m0.927446433s) [noise=true , address=64.49.13.26:4955 , piece_id=VIN3RVGRSUUHYCHPXCDNVSPRLFSCH3JXIDERBV7NIM4K33ZUBUGQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.330936609 (459h21m0.927707633s) [noise=true , address=193.149.164.68:4050 , piece_id=F4AQMCLJL6ML2LQKWUIKJAN6C2Y3M7UVYXRCRVRON5BC6CLTEK6Q, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.331355022 (459h21m0.928126051s) [noise=true , address=77.90.31.11:4567 , piece_id=ZSITPEN2OCYCHLSV3PDXCAXBAHEAQMS3YFCCOOE2SISJMUFRPHOA, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.402821184 (459h21m0.999592291s) [upload=514680]: Next returned nil segment
Mar 25 14:21:03.402848744 (459h21m0.99961978s) [upload=514680]: Waiting for error group managing segments...
Mar 25 14:21:03.408851473 (459h21m1.005622504s) [noise=true , address=77.90.31.16:4739 , piece_id=LEV3TZ2XFAYLQHULRJOG3DDBFMP3XRWX4EU2CQ2YOFICYXLIR3SA, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.409008646 (459h21m1.005779679s) [noise=true , address=145.79.227.34:4916 , piece_id=645H2KOPV4YB2HTP3XZF2RF5ZICIR34QNYRNRKX5KHH2VANVVZOQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.418584053 (459h21m1.015355084s) [noise=true , address=77.90.31.17:4791 , piece_id=L5H6QECJGEQPTPRB6INQWHZPMK7ZXXDYYFFG5ANE37HAE33OTVXQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.446927214 (459h21m1.043698248s) [noise=true , address=45.91.116.7:4023 , piece_id=GYWVVXKLOAZSDJIS64INTFQIPQZCLBH7YPLTWY7QJSB766A3QTGQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.448923805 (459h21m1.045694815s) [noise=true , address=77.90.31.14:4688 , piece_id=XDS6GE2YZK3LWUTBOMHLWTHM4ASCQJI25WTENUU75TJGFULYPOCQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.458340727 (459h21m1.055111751s) [noise=true , address=45.91.116.12:4254 , piece_id=UIQ3FWZSZ52RJYWFVWNH3OGNQPZSDYEZEDX35K3R7SDKRIW3WOAQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.468833670 (459h21m1.065604702s) [noise=true , address=77.90.31.11:4567 , piece_id=ZSITPEN2OCYCHLSV3PDXCAXBAHEAQMS3YFCCOOE2SISJMUFRPHOA, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.925971986 (459h21m1.522743012s) [noise=true , address=193.149.164.66:4984 , piece_id=CGM44RIHZGETSCKRZQOFL3VYIJTSB57RQDXTG3OPX3F3HJNQDYZA, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.928271525 (459h21m1.525042552s) [noise=true , address=64.49.13.140:4509 , piece_id=7HN3FXTKC4RRUJZ34AHFDJHMUBIOWIQVFST626CR7X2ZYQS3CFCQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.929087375 (459h21m1.525858388s) [noise=true , address=193.149.164.70:4120 , piece_id=HMFOJCRQTG3PAQDIZERNEYWKFFBUIRKRXP56OQUQWH6R2YX6NYLQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.929851130 (459h21m1.526622145s) [noise=true , address=193.149.164.71:4296 , piece_id=RXZXV3FCVDEFHBJQVGIWEKS57EJ3IKAJFTDGMOEZHRRBHV3NRDSA, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.935607616 (459h21m1.532378643s) [noise=true , address=193.149.164.69:4085 , piece_id=KGCVBEK344OW573PN2Y55JA7AC3ZTQD36G3T6TUHF7ZMZ4WN7N5A, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.938874518 (459h21m1.535645546s) [noise=true , address=193.149.164.67:4030 , piece_id=PBIDHXYQ22HHK3HBJLFPC5X4L74Q332LNNEDYPFDV6SHWCQKRCNQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.950243460 (459h21m1.547014493s) [noise=true , address=64.49.13.26:4955 , piece_id=VIN3RVGRSUUHYCHPXCDNVSPRLFSCH3JXIDERBV7NIM4K33ZUBUGQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.950277580 (459h21m1.54704861s) [seg_pos={0 0}, upload=514680]: Satisfied cohort requirements after 14 pieces
Mar 25 14:21:03.950707500 (459h21m1.547478532s) [noise=true , address=193.149.164.68:4050 , piece_id=F4AQMCLJL6ML2LQKWUIKJAN6C2Y3M7UVYXRCRVRON5BC6CLTEK6Q, seg_pos={0 0}, upload=514680]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1WPfYrCJrhPTqZ2qgBvXFCUodKz6QeXCnkKgToFs5muNRSSLtK): protocol: expected piece hash; context canceled; EOF
Mar 25 14:21:03.950721648 (459h21m1.547492674s) [noise=true , address=193.149.164.68:4050 , piece_id=F4AQMCLJL6ML2LQKWUIKJAN6C2Y3M7UVYXRCRVRON5BC6CLTEK6Q, seg_pos={0 0}, upload=514680]: Long tail cancellation due to optimal threshold
Mar 25 14:21:03.950793002 (459h21m1.547564034s) [noise=true , address=64.49.13.27:4964 , piece_id=UOBMO3UGR2SVYEXRSW2SQMHSDJV47OXOZRZJHCOHVWNXMIIDSA2A, seg_pos={0 0}, upload=514680]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RcgfaeKP3XpZGuSB9GMURCCsjjc79WeZasUL2RRiQVDwxbnAk): protocol: expected piece hash; context canceled; EOF
Mar 25 14:21:03.950804838 (459h21m1.547575864s) [noise=true , address=64.49.13.27:4964 , piece_id=UOBMO3UGR2SVYEXRSW2SQMHSDJV47OXOZRZJHCOHVWNXMIIDSA2A, seg_pos={0 0}, upload=514680]: Long tail cancellation due to optimal threshold
Mar 25 14:21:03.950899119 (459h21m1.547670144s) [noise=true , address=145.79.227.44:4201 , piece_id=SHQIEKFZSOKO7SUBGDC7GSJQNL47IEWENP5KN5ZN5ZOS4K3R7Y7A, seg_pos={0 0}, upload=514680]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bPDvhDfu8Z58LxZcicg1vAiiu1sZYGpwQnvuhk4t5m6yR87Fi): protocol: expected piece hash; context canceled; EOF
Mar 25 14:21:03.950907089 (459h21m1.547678114s) [noise=true , address=145.79.227.44:4201 , piece_id=SHQIEKFZSOKO7SUBGDC7GSJQNL47IEWENP5KN5ZN5ZOS4K3R7Y7A, seg_pos={0 0}, upload=514680]: Long tail cancellation due to optimal threshold
Mar 25 14:21:03.951133312 (459h21m1.547904347s) [noise=true , address=64.49.13.138:4487 , piece_id=NMZRB7X7MX2KJRDEUBROG3F6ZPQOGYJLTJ5WHBBFOUQCKXIHIZKQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12taemzrausuc58d8Dyo8xaR7MqDf72f37rEheF32sUfy8Z8GZP): protocol: expected piece hash; context canceled; EOF
Mar 25 14:21:03.951146101 (459h21m1.547917132s) [noise=true , address=64.49.13.138:4487 , piece_id=NMZRB7X7MX2KJRDEUBROG3F6ZPQOGYJLTJ5WHBBFOUQCKXIHIZKQ, seg_pos={0 0}, upload=514680]: Long tail cancellation due to optimal threshold
Mar 25 14:21:03.951175027 (459h21m1.547946039s) [seg_pos={0 0}, upload=514680]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Mar 25 14:21:03.951282270 (459h21m1.548053293s) [upload=514680]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Mar 25 14:21:03.951290421 (459h21m1.548061457s) [upload=514680]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Mar 25 14:21:04.013599996 (459h21m1.610371022s) [upload=514680]: Done uploading segments...
Mar 25 14:21:04.013621043 (459h21m1.610392066s) [upload=514680]: Upload finished. err: <nil>
Duration: 1.409703548s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [............X...............................................] duration:
1.396171632s trace:
https://jaeger.dev.storj.io/trace/62357518aac8ca5f
FAIL (2026-03-25 15:02:13 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 1.396359811s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
30.064429ms target:
https://us1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/21096334d2fd8925
FAIL (2026-03-25 15:01:59 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.4.32.59:443: i/o timeout
Duration: 30.089866ms
core
satellite
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [......X.....................................................] duration:
2.0292279s trace:
https://jaeger.dev.storj.io/trace/1d1f0bc4e4438495
FAIL (2026-03-25 15:01:29 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
upload=514680]: Uploading piece...
Mar 25 14:21:03.327216643 (459h21m0.923987655s) [noise=true , address=45.91.116.12:4254 , piece_id=UIQ3FWZSZ52RJYWFVWNH3OGNQPZSDYEZEDX35K3R7SDKRIW3WOAQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.327580267 (459h21m0.924351301s) [noise=true , address=193.149.164.70:4120 , piece_id=HMFOJCRQTG3PAQDIZERNEYWKFFBUIRKRXP56OQUQWH6R2YX6NYLQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.327966258 (459h21m0.924737281s) [noise=true , address=77.90.31.16:4739 , piece_id=LEV3TZ2XFAYLQHULRJOG3DDBFMP3XRWX4EU2CQ2YOFICYXLIR3SA, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.328304150 (459h21m0.925075204s) [noise=true , address=77.90.31.14:4688 , piece_id=XDS6GE2YZK3LWUTBOMHLWTHM4ASCQJI25WTENUU75TJGFULYPOCQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.328528343 (459h21m0.925299366s) [noise=true , address=64.49.13.138:4487 , piece_id=NMZRB7X7MX2KJRDEUBROG3F6ZPQOGYJLTJ5WHBBFOUQCKXIHIZKQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.328832656 (459h21m0.925603704s) [noise=true , address=145.79.227.44:4201 , piece_id=SHQIEKFZSOKO7SUBGDC7GSJQNL47IEWENP5KN5ZN5ZOS4K3R7Y7A, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.328990765 (459h21m0.925761788s) [noise=true , address=77.90.31.17:4791 , piece_id=L5H6QECJGEQPTPRB6INQWHZPMK7ZXXDYYFFG5ANE37HAE33OTVXQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.329286413 (459h21m0.926057443s) [noise=true , address=64.49.13.27:4964 , piece_id=UOBMO3UGR2SVYEXRSW2SQMHSDJV47OXOZRZJHCOHVWNXMIIDSA2A, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.329346877 (459h21m0.9261179s) [noise=true , address=193.149.164.66:4984 , piece_id=CGM44RIHZGETSCKRZQOFL3VYIJTSB57RQDXTG3OPX3F3HJNQDYZA, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.329706811 (459h21m0.926477834s) [noise=true , address=45.91.116.7:4023 , piece_id=GYWVVXKLOAZSDJIS64INTFQIPQZCLBH7YPLTWY7QJSB766A3QTGQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.329728635 (459h21m0.926499661s) [noise=true , address=64.49.13.140:4509 , piece_id=7HN3FXTKC4RRUJZ34AHFDJHMUBIOWIQVFST626CR7X2ZYQS3CFCQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.330142941 (459h21m0.926913971s) [noise=true , address=145.79.227.34:4916 , piece_id=645H2KOPV4YB2HTP3XZF2RF5ZICIR34QNYRNRKX5KHH2VANVVZOQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.330171985 (459h21m0.926943015s) [noise=true , address=193.149.164.71:4296 , piece_id=RXZXV3FCVDEFHBJQVGIWEKS57EJ3IKAJFTDGMOEZHRRBHV3NRDSA, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.330543280 (459h21m0.927314303s) [noise=true , address=193.149.164.67:4030 , piece_id=PBIDHXYQ22HHK3HBJLFPC5X4L74Q332LNNEDYPFDV6SHWCQKRCNQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.330675401 (459h21m0.927446433s) [noise=true , address=64.49.13.26:4955 , piece_id=VIN3RVGRSUUHYCHPXCDNVSPRLFSCH3JXIDERBV7NIM4K33ZUBUGQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.330936609 (459h21m0.927707633s) [noise=true , address=193.149.164.68:4050 , piece_id=F4AQMCLJL6ML2LQKWUIKJAN6C2Y3M7UVYXRCRVRON5BC6CLTEK6Q, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.331355022 (459h21m0.928126051s) [noise=true , address=77.90.31.11:4567 , piece_id=ZSITPEN2OCYCHLSV3PDXCAXBAHEAQMS3YFCCOOE2SISJMUFRPHOA, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.402821184 (459h21m0.999592291s) [upload=514680]: Next returned nil segment
Mar 25 14:21:03.402848744 (459h21m0.99961978s) [upload=514680]: Waiting for error group managing segments...
Mar 25 14:21:03.408851473 (459h21m1.005622504s) [noise=true , address=77.90.31.16:4739 , piece_id=LEV3TZ2XFAYLQHULRJOG3DDBFMP3XRWX4EU2CQ2YOFICYXLIR3SA, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.409008646 (459h21m1.005779679s) [noise=true , address=145.79.227.34:4916 , piece_id=645H2KOPV4YB2HTP3XZF2RF5ZICIR34QNYRNRKX5KHH2VANVVZOQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.418584053 (459h21m1.015355084s) [noise=true , address=77.90.31.17:4791 , piece_id=L5H6QECJGEQPTPRB6INQWHZPMK7ZXXDYYFFG5ANE37HAE33OTVXQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.446927214 (459h21m1.043698248s) [noise=true , address=45.91.116.7:4023 , piece_id=GYWVVXKLOAZSDJIS64INTFQIPQZCLBH7YPLTWY7QJSB766A3QTGQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.448923805 (459h21m1.045694815s) [noise=true , address=77.90.31.14:4688 , piece_id=XDS6GE2YZK3LWUTBOMHLWTHM4ASCQJI25WTENUU75TJGFULYPOCQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.458340727 (459h21m1.055111751s) [noise=true , address=45.91.116.12:4254 , piece_id=UIQ3FWZSZ52RJYWFVWNH3OGNQPZSDYEZEDX35K3R7SDKRIW3WOAQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.468833670 (459h21m1.065604702s) [noise=true , address=77.90.31.11:4567 , piece_id=ZSITPEN2OCYCHLSV3PDXCAXBAHEAQMS3YFCCOOE2SISJMUFRPHOA, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.925971986 (459h21m1.522743012s) [noise=true , address=193.149.164.66:4984 , piece_id=CGM44RIHZGETSCKRZQOFL3VYIJTSB57RQDXTG3OPX3F3HJNQDYZA, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.928271525 (459h21m1.525042552s) [noise=true , address=64.49.13.140:4509 , piece_id=7HN3FXTKC4RRUJZ34AHFDJHMUBIOWIQVFST626CR7X2ZYQS3CFCQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.929087375 (459h21m1.525858388s) [noise=true , address=193.149.164.70:4120 , piece_id=HMFOJCRQTG3PAQDIZERNEYWKFFBUIRKRXP56OQUQWH6R2YX6NYLQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.929851130 (459h21m1.526622145s) [noise=true , address=193.149.164.71:4296 , piece_id=RXZXV3FCVDEFHBJQVGIWEKS57EJ3IKAJFTDGMOEZHRRBHV3NRDSA, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.935607616 (459h21m1.532378643s) [noise=true , address=193.149.164.69:4085 , piece_id=KGCVBEK344OW573PN2Y55JA7AC3ZTQD36G3T6TUHF7ZMZ4WN7N5A, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.938874518 (459h21m1.535645546s) [noise=true , address=193.149.164.67:4030 , piece_id=PBIDHXYQ22HHK3HBJLFPC5X4L74Q332LNNEDYPFDV6SHWCQKRCNQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.950243460 (459h21m1.547014493s) [noise=true , address=64.49.13.26:4955 , piece_id=VIN3RVGRSUUHYCHPXCDNVSPRLFSCH3JXIDERBV7NIM4K33ZUBUGQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.950277580 (459h21m1.54704861s) [seg_pos={0 0}, upload=514680]: Satisfied cohort requirements after 14 pieces
Mar 25 14:21:03.950707500 (459h21m1.547478532s) [noise=true , address=193.149.164.68:4050 , piece_id=F4AQMCLJL6ML2LQKWUIKJAN6C2Y3M7UVYXRCRVRON5BC6CLTEK6Q, seg_pos={0 0}, upload=514680]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1WPfYrCJrhPTqZ2qgBvXFCUodKz6QeXCnkKgToFs5muNRSSLtK): protocol: expected piece hash; context canceled; EOF
Mar 25 14:21:03.950721648 (459h21m1.547492674s) [noise=true , address=193.149.164.68:4050 , piece_id=F4AQMCLJL6ML2LQKWUIKJAN6C2Y3M7UVYXRCRVRON5BC6CLTEK6Q, seg_pos={0 0}, upload=514680]: Long tail cancellation due to optimal threshold
Mar 25 14:21:03.950793002 (459h21m1.547564034s) [noise=true , address=64.49.13.27:4964 , piece_id=UOBMO3UGR2SVYEXRSW2SQMHSDJV47OXOZRZJHCOHVWNXMIIDSA2A, seg_pos={0 0}, upload=514680]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RcgfaeKP3XpZGuSB9GMURCCsjjc79WeZasUL2RRiQVDwxbnAk): protocol: expected piece hash; context canceled; EOF
Mar 25 14:21:03.950804838 (459h21m1.547575864s) [noise=true , address=64.49.13.27:4964 , piece_id=UOBMO3UGR2SVYEXRSW2SQMHSDJV47OXOZRZJHCOHVWNXMIIDSA2A, seg_pos={0 0}, upload=514680]: Long tail cancellation due to optimal threshold
Mar 25 14:21:03.950899119 (459h21m1.547670144s) [noise=true , address=145.79.227.44:4201 , piece_id=SHQIEKFZSOKO7SUBGDC7GSJQNL47IEWENP5KN5ZN5ZOS4K3R7Y7A, seg_pos={0 0}, upload=514680]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bPDvhDfu8Z58LxZcicg1vAiiu1sZYGpwQnvuhk4t5m6yR87Fi): protocol: expected piece hash; context canceled; EOF
Mar 25 14:21:03.950907089 (459h21m1.547678114s) [noise=true , address=145.79.227.44:4201 , piece_id=SHQIEKFZSOKO7SUBGDC7GSJQNL47IEWENP5KN5ZN5ZOS4K3R7Y7A, seg_pos={0 0}, upload=514680]: Long tail cancellation due to optimal threshold
Mar 25 14:21:03.951133312 (459h21m1.547904347s) [noise=true , address=64.49.13.138:4487 , piece_id=NMZRB7X7MX2KJRDEUBROG3F6ZPQOGYJLTJ5WHBBFOUQCKXIHIZKQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12taemzrausuc58d8Dyo8xaR7MqDf72f37rEheF32sUfy8Z8GZP): protocol: expected piece hash; context canceled; EOF
Mar 25 14:21:03.951146101 (459h21m1.547917132s) [noise=true , address=64.49.13.138:4487 , piece_id=NMZRB7X7MX2KJRDEUBROG3F6ZPQOGYJLTJ5WHBBFOUQCKXIHIZKQ, seg_pos={0 0}, upload=514680]: Long tail cancellation due to optimal threshold
Mar 25 14:21:03.951175027 (459h21m1.547946039s) [seg_pos={0 0}, upload=514680]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Mar 25 14:21:03.951282270 (459h21m1.548053293s) [upload=514680]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Mar 25 14:21:03.951290421 (459h21m1.548061457s) [upload=514680]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Mar 25 14:21:04.013599996 (459h21m1.610371022s) [upload=514680]: Done uploading segments...
Mar 25 14:21:04.013621043 (459h21m1.610392066s) [upload=514680]: Upload finished. err: <nil>
Duration: 2.029256701s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............X..............................................] duration:
1.350305171s trace:
https://jaeger.dev.storj.io/trace/62357518aac8ca5f
FAIL (2026-03-25 15:01:16 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 1.350336031s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......X....................................................] duration:
1.879808783s trace:
https://jaeger.dev.storj.io/trace/1d1f0bc4e4438495
FAIL (2026-03-25 15:00:58 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
upload=514680]: Uploading piece...
Mar 25 14:21:03.327216643 (459h21m0.923987655s) [noise=true , address=45.91.116.12:4254 , piece_id=UIQ3FWZSZ52RJYWFVWNH3OGNQPZSDYEZEDX35K3R7SDKRIW3WOAQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.327580267 (459h21m0.924351301s) [noise=true , address=193.149.164.70:4120 , piece_id=HMFOJCRQTG3PAQDIZERNEYWKFFBUIRKRXP56OQUQWH6R2YX6NYLQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.327966258 (459h21m0.924737281s) [noise=true , address=77.90.31.16:4739 , piece_id=LEV3TZ2XFAYLQHULRJOG3DDBFMP3XRWX4EU2CQ2YOFICYXLIR3SA, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.328304150 (459h21m0.925075204s) [noise=true , address=77.90.31.14:4688 , piece_id=XDS6GE2YZK3LWUTBOMHLWTHM4ASCQJI25WTENUU75TJGFULYPOCQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.328528343 (459h21m0.925299366s) [noise=true , address=64.49.13.138:4487 , piece_id=NMZRB7X7MX2KJRDEUBROG3F6ZPQOGYJLTJ5WHBBFOUQCKXIHIZKQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.328832656 (459h21m0.925603704s) [noise=true , address=145.79.227.44:4201 , piece_id=SHQIEKFZSOKO7SUBGDC7GSJQNL47IEWENP5KN5ZN5ZOS4K3R7Y7A, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.328990765 (459h21m0.925761788s) [noise=true , address=77.90.31.17:4791 , piece_id=L5H6QECJGEQPTPRB6INQWHZPMK7ZXXDYYFFG5ANE37HAE33OTVXQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.329286413 (459h21m0.926057443s) [noise=true , address=64.49.13.27:4964 , piece_id=UOBMO3UGR2SVYEXRSW2SQMHSDJV47OXOZRZJHCOHVWNXMIIDSA2A, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.329346877 (459h21m0.9261179s) [noise=true , address=193.149.164.66:4984 , piece_id=CGM44RIHZGETSCKRZQOFL3VYIJTSB57RQDXTG3OPX3F3HJNQDYZA, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.329706811 (459h21m0.926477834s) [noise=true , address=45.91.116.7:4023 , piece_id=GYWVVXKLOAZSDJIS64INTFQIPQZCLBH7YPLTWY7QJSB766A3QTGQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.329728635 (459h21m0.926499661s) [noise=true , address=64.49.13.140:4509 , piece_id=7HN3FXTKC4RRUJZ34AHFDJHMUBIOWIQVFST626CR7X2ZYQS3CFCQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.330142941 (459h21m0.926913971s) [noise=true , address=145.79.227.34:4916 , piece_id=645H2KOPV4YB2HTP3XZF2RF5ZICIR34QNYRNRKX5KHH2VANVVZOQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.330171985 (459h21m0.926943015s) [noise=true , address=193.149.164.71:4296 , piece_id=RXZXV3FCVDEFHBJQVGIWEKS57EJ3IKAJFTDGMOEZHRRBHV3NRDSA, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.330543280 (459h21m0.927314303s) [noise=true , address=193.149.164.67:4030 , piece_id=PBIDHXYQ22HHK3HBJLFPC5X4L74Q332LNNEDYPFDV6SHWCQKRCNQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.330675401 (459h21m0.927446433s) [noise=true , address=64.49.13.26:4955 , piece_id=VIN3RVGRSUUHYCHPXCDNVSPRLFSCH3JXIDERBV7NIM4K33ZUBUGQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.330936609 (459h21m0.927707633s) [noise=true , address=193.149.164.68:4050 , piece_id=F4AQMCLJL6ML2LQKWUIKJAN6C2Y3M7UVYXRCRVRON5BC6CLTEK6Q, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.331355022 (459h21m0.928126051s) [noise=true , address=77.90.31.11:4567 , piece_id=ZSITPEN2OCYCHLSV3PDXCAXBAHEAQMS3YFCCOOE2SISJMUFRPHOA, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.402821184 (459h21m0.999592291s) [upload=514680]: Next returned nil segment
Mar 25 14:21:03.402848744 (459h21m0.99961978s) [upload=514680]: Waiting for error group managing segments...
Mar 25 14:21:03.408851473 (459h21m1.005622504s) [noise=true , address=77.90.31.16:4739 , piece_id=LEV3TZ2XFAYLQHULRJOG3DDBFMP3XRWX4EU2CQ2YOFICYXLIR3SA, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.409008646 (459h21m1.005779679s) [noise=true , address=145.79.227.34:4916 , piece_id=645H2KOPV4YB2HTP3XZF2RF5ZICIR34QNYRNRKX5KHH2VANVVZOQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.418584053 (459h21m1.015355084s) [noise=true , address=77.90.31.17:4791 , piece_id=L5H6QECJGEQPTPRB6INQWHZPMK7ZXXDYYFFG5ANE37HAE33OTVXQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.446927214 (459h21m1.043698248s) [noise=true , address=45.91.116.7:4023 , piece_id=GYWVVXKLOAZSDJIS64INTFQIPQZCLBH7YPLTWY7QJSB766A3QTGQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.448923805 (459h21m1.045694815s) [noise=true , address=77.90.31.14:4688 , piece_id=XDS6GE2YZK3LWUTBOMHLWTHM4ASCQJI25WTENUU75TJGFULYPOCQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.458340727 (459h21m1.055111751s) [noise=true , address=45.91.116.12:4254 , piece_id=UIQ3FWZSZ52RJYWFVWNH3OGNQPZSDYEZEDX35K3R7SDKRIW3WOAQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.468833670 (459h21m1.065604702s) [noise=true , address=77.90.31.11:4567 , piece_id=ZSITPEN2OCYCHLSV3PDXCAXBAHEAQMS3YFCCOOE2SISJMUFRPHOA, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.925971986 (459h21m1.522743012s) [noise=true , address=193.149.164.66:4984 , piece_id=CGM44RIHZGETSCKRZQOFL3VYIJTSB57RQDXTG3OPX3F3HJNQDYZA, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.928271525 (459h21m1.525042552s) [noise=true , address=64.49.13.140:4509 , piece_id=7HN3FXTKC4RRUJZ34AHFDJHMUBIOWIQVFST626CR7X2ZYQS3CFCQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.929087375 (459h21m1.525858388s) [noise=true , address=193.149.164.70:4120 , piece_id=HMFOJCRQTG3PAQDIZERNEYWKFFBUIRKRXP56OQUQWH6R2YX6NYLQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.929851130 (459h21m1.526622145s) [noise=true , address=193.149.164.71:4296 , piece_id=RXZXV3FCVDEFHBJQVGIWEKS57EJ3IKAJFTDGMOEZHRRBHV3NRDSA, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.935607616 (459h21m1.532378643s) [noise=true , address=193.149.164.69:4085 , piece_id=KGCVBEK344OW573PN2Y55JA7AC3ZTQD36G3T6TUHF7ZMZ4WN7N5A, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.938874518 (459h21m1.535645546s) [noise=true , address=193.149.164.67:4030 , piece_id=PBIDHXYQ22HHK3HBJLFPC5X4L74Q332LNNEDYPFDV6SHWCQKRCNQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.950243460 (459h21m1.547014493s) [noise=true , address=64.49.13.26:4955 , piece_id=VIN3RVGRSUUHYCHPXCDNVSPRLFSCH3JXIDERBV7NIM4K33ZUBUGQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.950277580 (459h21m1.54704861s) [seg_pos={0 0}, upload=514680]: Satisfied cohort requirements after 14 pieces
Mar 25 14:21:03.950707500 (459h21m1.547478532s) [noise=true , address=193.149.164.68:4050 , piece_id=F4AQMCLJL6ML2LQKWUIKJAN6C2Y3M7UVYXRCRVRON5BC6CLTEK6Q, seg_pos={0 0}, upload=514680]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1WPfYrCJrhPTqZ2qgBvXFCUodKz6QeXCnkKgToFs5muNRSSLtK): protocol: expected piece hash; context canceled; EOF
Mar 25 14:21:03.950721648 (459h21m1.547492674s) [noise=true , address=193.149.164.68:4050 , piece_id=F4AQMCLJL6ML2LQKWUIKJAN6C2Y3M7UVYXRCRVRON5BC6CLTEK6Q, seg_pos={0 0}, upload=514680]: Long tail cancellation due to optimal threshold
Mar 25 14:21:03.950793002 (459h21m1.547564034s) [noise=true , address=64.49.13.27:4964 , piece_id=UOBMO3UGR2SVYEXRSW2SQMHSDJV47OXOZRZJHCOHVWNXMIIDSA2A, seg_pos={0 0}, upload=514680]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RcgfaeKP3XpZGuSB9GMURCCsjjc79WeZasUL2RRiQVDwxbnAk): protocol: expected piece hash; context canceled; EOF
Mar 25 14:21:03.950804838 (459h21m1.547575864s) [noise=true , address=64.49.13.27:4964 , piece_id=UOBMO3UGR2SVYEXRSW2SQMHSDJV47OXOZRZJHCOHVWNXMIIDSA2A, seg_pos={0 0}, upload=514680]: Long tail cancellation due to optimal threshold
Mar 25 14:21:03.950899119 (459h21m1.547670144s) [noise=true , address=145.79.227.44:4201 , piece_id=SHQIEKFZSOKO7SUBGDC7GSJQNL47IEWENP5KN5ZN5ZOS4K3R7Y7A, seg_pos={0 0}, upload=514680]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bPDvhDfu8Z58LxZcicg1vAiiu1sZYGpwQnvuhk4t5m6yR87Fi): protocol: expected piece hash; context canceled; EOF
Mar 25 14:21:03.950907089 (459h21m1.547678114s) [noise=true , address=145.79.227.44:4201 , piece_id=SHQIEKFZSOKO7SUBGDC7GSJQNL47IEWENP5KN5ZN5ZOS4K3R7Y7A, seg_pos={0 0}, upload=514680]: Long tail cancellation due to optimal threshold
Mar 25 14:21:03.951133312 (459h21m1.547904347s) [noise=true , address=64.49.13.138:4487 , piece_id=NMZRB7X7MX2KJRDEUBROG3F6ZPQOGYJLTJ5WHBBFOUQCKXIHIZKQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12taemzrausuc58d8Dyo8xaR7MqDf72f37rEheF32sUfy8Z8GZP): protocol: expected piece hash; context canceled; EOF
Mar 25 14:21:03.951146101 (459h21m1.547917132s) [noise=true , address=64.49.13.138:4487 , piece_id=NMZRB7X7MX2KJRDEUBROG3F6ZPQOGYJLTJ5WHBBFOUQCKXIHIZKQ, seg_pos={0 0}, upload=514680]: Long tail cancellation due to optimal threshold
Mar 25 14:21:03.951175027 (459h21m1.547946039s) [seg_pos={0 0}, upload=514680]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Mar 25 14:21:03.951282270 (459h21m1.548053293s) [upload=514680]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Mar 25 14:21:03.951290421 (459h21m1.548061457s) [upload=514680]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Mar 25 14:21:04.013599996 (459h21m1.610371022s) [upload=514680]: Done uploading segments...
Mar 25 14:21:04.013621043 (459h21m1.610392066s) [upload=514680]: Upload finished. err: <nil>
Duration: 1.879912779s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............X.............................................] duration:
1.893471635s trace:
https://jaeger.dev.storj.io/trace/62357518aac8ca5f
FAIL (2026-03-25 15:00:50 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 1.893517313s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................] duration:
32.097576ms target:
https://us1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/21096334d2fd8925
FAIL (2026-03-25 15:00:49 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.4.32.59:443: i/o timeout
Duration: 32.158804ms
core
satellite
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............X............................................] duration:
1.355455456s trace:
https://jaeger.dev.storj.io/trace/62357518aac8ca5f
FAIL (2026-03-25 15:00:32 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 1.355594311s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [........X...................................................] duration:
1.796675102s trace:
https://jaeger.dev.storj.io/trace/1d1f0bc4e4438495
FAIL (2026-03-25 15:00:31 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
upload=514680]: Uploading piece...
Mar 25 14:21:03.327216643 (459h21m0.923987655s) [noise=true , address=45.91.116.12:4254 , piece_id=UIQ3FWZSZ52RJYWFVWNH3OGNQPZSDYEZEDX35K3R7SDKRIW3WOAQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.327580267 (459h21m0.924351301s) [noise=true , address=193.149.164.70:4120 , piece_id=HMFOJCRQTG3PAQDIZERNEYWKFFBUIRKRXP56OQUQWH6R2YX6NYLQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.327966258 (459h21m0.924737281s) [noise=true , address=77.90.31.16:4739 , piece_id=LEV3TZ2XFAYLQHULRJOG3DDBFMP3XRWX4EU2CQ2YOFICYXLIR3SA, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.328304150 (459h21m0.925075204s) [noise=true , address=77.90.31.14:4688 , piece_id=XDS6GE2YZK3LWUTBOMHLWTHM4ASCQJI25WTENUU75TJGFULYPOCQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.328528343 (459h21m0.925299366s) [noise=true , address=64.49.13.138:4487 , piece_id=NMZRB7X7MX2KJRDEUBROG3F6ZPQOGYJLTJ5WHBBFOUQCKXIHIZKQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.328832656 (459h21m0.925603704s) [noise=true , address=145.79.227.44:4201 , piece_id=SHQIEKFZSOKO7SUBGDC7GSJQNL47IEWENP5KN5ZN5ZOS4K3R7Y7A, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.328990765 (459h21m0.925761788s) [noise=true , address=77.90.31.17:4791 , piece_id=L5H6QECJGEQPTPRB6INQWHZPMK7ZXXDYYFFG5ANE37HAE33OTVXQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.329286413 (459h21m0.926057443s) [noise=true , address=64.49.13.27:4964 , piece_id=UOBMO3UGR2SVYEXRSW2SQMHSDJV47OXOZRZJHCOHVWNXMIIDSA2A, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.329346877 (459h21m0.9261179s) [noise=true , address=193.149.164.66:4984 , piece_id=CGM44RIHZGETSCKRZQOFL3VYIJTSB57RQDXTG3OPX3F3HJNQDYZA, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.329706811 (459h21m0.926477834s) [noise=true , address=45.91.116.7:4023 , piece_id=GYWVVXKLOAZSDJIS64INTFQIPQZCLBH7YPLTWY7QJSB766A3QTGQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.329728635 (459h21m0.926499661s) [noise=true , address=64.49.13.140:4509 , piece_id=7HN3FXTKC4RRUJZ34AHFDJHMUBIOWIQVFST626CR7X2ZYQS3CFCQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.330142941 (459h21m0.926913971s) [noise=true , address=145.79.227.34:4916 , piece_id=645H2KOPV4YB2HTP3XZF2RF5ZICIR34QNYRNRKX5KHH2VANVVZOQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.330171985 (459h21m0.926943015s) [noise=true , address=193.149.164.71:4296 , piece_id=RXZXV3FCVDEFHBJQVGIWEKS57EJ3IKAJFTDGMOEZHRRBHV3NRDSA, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.330543280 (459h21m0.927314303s) [noise=true , address=193.149.164.67:4030 , piece_id=PBIDHXYQ22HHK3HBJLFPC5X4L74Q332LNNEDYPFDV6SHWCQKRCNQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.330675401 (459h21m0.927446433s) [noise=true , address=64.49.13.26:4955 , piece_id=VIN3RVGRSUUHYCHPXCDNVSPRLFSCH3JXIDERBV7NIM4K33ZUBUGQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.330936609 (459h21m0.927707633s) [noise=true , address=193.149.164.68:4050 , piece_id=F4AQMCLJL6ML2LQKWUIKJAN6C2Y3M7UVYXRCRVRON5BC6CLTEK6Q, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.331355022 (459h21m0.928126051s) [noise=true , address=77.90.31.11:4567 , piece_id=ZSITPEN2OCYCHLSV3PDXCAXBAHEAQMS3YFCCOOE2SISJMUFRPHOA, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.402821184 (459h21m0.999592291s) [upload=514680]: Next returned nil segment
Mar 25 14:21:03.402848744 (459h21m0.99961978s) [upload=514680]: Waiting for error group managing segments...
Mar 25 14:21:03.408851473 (459h21m1.005622504s) [noise=true , address=77.90.31.16:4739 , piece_id=LEV3TZ2XFAYLQHULRJOG3DDBFMP3XRWX4EU2CQ2YOFICYXLIR3SA, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.409008646 (459h21m1.005779679s) [noise=true , address=145.79.227.34:4916 , piece_id=645H2KOPV4YB2HTP3XZF2RF5ZICIR34QNYRNRKX5KHH2VANVVZOQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.418584053 (459h21m1.015355084s) [noise=true , address=77.90.31.17:4791 , piece_id=L5H6QECJGEQPTPRB6INQWHZPMK7ZXXDYYFFG5ANE37HAE33OTVXQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.446927214 (459h21m1.043698248s) [noise=true , address=45.91.116.7:4023 , piece_id=GYWVVXKLOAZSDJIS64INTFQIPQZCLBH7YPLTWY7QJSB766A3QTGQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.448923805 (459h21m1.045694815s) [noise=true , address=77.90.31.14:4688 , piece_id=XDS6GE2YZK3LWUTBOMHLWTHM4ASCQJI25WTENUU75TJGFULYPOCQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.458340727 (459h21m1.055111751s) [noise=true , address=45.91.116.12:4254 , piece_id=UIQ3FWZSZ52RJYWFVWNH3OGNQPZSDYEZEDX35K3R7SDKRIW3WOAQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.468833670 (459h21m1.065604702s) [noise=true , address=77.90.31.11:4567 , piece_id=ZSITPEN2OCYCHLSV3PDXCAXBAHEAQMS3YFCCOOE2SISJMUFRPHOA, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.925971986 (459h21m1.522743012s) [noise=true , address=193.149.164.66:4984 , piece_id=CGM44RIHZGETSCKRZQOFL3VYIJTSB57RQDXTG3OPX3F3HJNQDYZA, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.928271525 (459h21m1.525042552s) [noise=true , address=64.49.13.140:4509 , piece_id=7HN3FXTKC4RRUJZ34AHFDJHMUBIOWIQVFST626CR7X2ZYQS3CFCQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.929087375 (459h21m1.525858388s) [noise=true , address=193.149.164.70:4120 , piece_id=HMFOJCRQTG3PAQDIZERNEYWKFFBUIRKRXP56OQUQWH6R2YX6NYLQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.929851130 (459h21m1.526622145s) [noise=true , address=193.149.164.71:4296 , piece_id=RXZXV3FCVDEFHBJQVGIWEKS57EJ3IKAJFTDGMOEZHRRBHV3NRDSA, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.935607616 (459h21m1.532378643s) [noise=true , address=193.149.164.69:4085 , piece_id=KGCVBEK344OW573PN2Y55JA7AC3ZTQD36G3T6TUHF7ZMZ4WN7N5A, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.938874518 (459h21m1.535645546s) [noise=true , address=193.149.164.67:4030 , piece_id=PBIDHXYQ22HHK3HBJLFPC5X4L74Q332LNNEDYPFDV6SHWCQKRCNQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.950243460 (459h21m1.547014493s) [noise=true , address=64.49.13.26:4955 , piece_id=VIN3RVGRSUUHYCHPXCDNVSPRLFSCH3JXIDERBV7NIM4K33ZUBUGQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.950277580 (459h21m1.54704861s) [seg_pos={0 0}, upload=514680]: Satisfied cohort requirements after 14 pieces
Mar 25 14:21:03.950707500 (459h21m1.547478532s) [noise=true , address=193.149.164.68:4050 , piece_id=F4AQMCLJL6ML2LQKWUIKJAN6C2Y3M7UVYXRCRVRON5BC6CLTEK6Q, seg_pos={0 0}, upload=514680]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1WPfYrCJrhPTqZ2qgBvXFCUodKz6QeXCnkKgToFs5muNRSSLtK): protocol: expected piece hash; context canceled; EOF
Mar 25 14:21:03.950721648 (459h21m1.547492674s) [noise=true , address=193.149.164.68:4050 , piece_id=F4AQMCLJL6ML2LQKWUIKJAN6C2Y3M7UVYXRCRVRON5BC6CLTEK6Q, seg_pos={0 0}, upload=514680]: Long tail cancellation due to optimal threshold
Mar 25 14:21:03.950793002 (459h21m1.547564034s) [noise=true , address=64.49.13.27:4964 , piece_id=UOBMO3UGR2SVYEXRSW2SQMHSDJV47OXOZRZJHCOHVWNXMIIDSA2A, seg_pos={0 0}, upload=514680]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RcgfaeKP3XpZGuSB9GMURCCsjjc79WeZasUL2RRiQVDwxbnAk): protocol: expected piece hash; context canceled; EOF
Mar 25 14:21:03.950804838 (459h21m1.547575864s) [noise=true , address=64.49.13.27:4964 , piece_id=UOBMO3UGR2SVYEXRSW2SQMHSDJV47OXOZRZJHCOHVWNXMIIDSA2A, seg_pos={0 0}, upload=514680]: Long tail cancellation due to optimal threshold
Mar 25 14:21:03.950899119 (459h21m1.547670144s) [noise=true , address=145.79.227.44:4201 , piece_id=SHQIEKFZSOKO7SUBGDC7GSJQNL47IEWENP5KN5ZN5ZOS4K3R7Y7A, seg_pos={0 0}, upload=514680]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bPDvhDfu8Z58LxZcicg1vAiiu1sZYGpwQnvuhk4t5m6yR87Fi): protocol: expected piece hash; context canceled; EOF
Mar 25 14:21:03.950907089 (459h21m1.547678114s) [noise=true , address=145.79.227.44:4201 , piece_id=SHQIEKFZSOKO7SUBGDC7GSJQNL47IEWENP5KN5ZN5ZOS4K3R7Y7A, seg_pos={0 0}, upload=514680]: Long tail cancellation due to optimal threshold
Mar 25 14:21:03.951133312 (459h21m1.547904347s) [noise=true , address=64.49.13.138:4487 , piece_id=NMZRB7X7MX2KJRDEUBROG3F6ZPQOGYJLTJ5WHBBFOUQCKXIHIZKQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12taemzrausuc58d8Dyo8xaR7MqDf72f37rEheF32sUfy8Z8GZP): protocol: expected piece hash; context canceled; EOF
Mar 25 14:21:03.951146101 (459h21m1.547917132s) [noise=true , address=64.49.13.138:4487 , piece_id=NMZRB7X7MX2KJRDEUBROG3F6ZPQOGYJLTJ5WHBBFOUQCKXIHIZKQ, seg_pos={0 0}, upload=514680]: Long tail cancellation due to optimal threshold
Mar 25 14:21:03.951175027 (459h21m1.547946039s) [seg_pos={0 0}, upload=514680]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Mar 25 14:21:03.951282270 (459h21m1.548053293s) [upload=514680]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Mar 25 14:21:03.951290421 (459h21m1.548061457s) [upload=514680]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Mar 25 14:21:04.013599996 (459h21m1.610371022s) [upload=514680]: Done uploading segments...
Mar 25 14:21:04.013621043 (459h21m1.610392066s) [upload=514680]: Upload finished. err: <nil>
Duration: 1.796719141s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [......X.....................................................] duration:
42.230675ms target:
https://us1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/21096334d2fd8925
FAIL (2026-03-25 15:00:26 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.4.32.59:443: i/o timeout
Duration: 42.259825ms
core
satellite
us1
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [.......X....................................................] duration:
46.578655ms target:
https://us1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/21096334d2fd8925
FAIL (2026-03-25 15:00:11 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.4.32.59:443: i/o timeout
Duration: 46.628757ms
core
satellite
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [................X...........................................] duration:
1.314533468s trace:
https://jaeger.dev.storj.io/trace/62357518aac8ca5f
FAIL (2026-03-25 15:00:07 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 1.314631414s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........X..................................................] duration:
1.431637978s trace:
https://jaeger.dev.storj.io/trace/1d1f0bc4e4438495
FAIL (2026-03-25 14:59:19 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
upload=514680]: Uploading piece...
Mar 25 14:21:03.327216643 (459h21m0.923987655s) [noise=true , address=45.91.116.12:4254 , piece_id=UIQ3FWZSZ52RJYWFVWNH3OGNQPZSDYEZEDX35K3R7SDKRIW3WOAQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.327580267 (459h21m0.924351301s) [noise=true , address=193.149.164.70:4120 , piece_id=HMFOJCRQTG3PAQDIZERNEYWKFFBUIRKRXP56OQUQWH6R2YX6NYLQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.327966258 (459h21m0.924737281s) [noise=true , address=77.90.31.16:4739 , piece_id=LEV3TZ2XFAYLQHULRJOG3DDBFMP3XRWX4EU2CQ2YOFICYXLIR3SA, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.328304150 (459h21m0.925075204s) [noise=true , address=77.90.31.14:4688 , piece_id=XDS6GE2YZK3LWUTBOMHLWTHM4ASCQJI25WTENUU75TJGFULYPOCQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.328528343 (459h21m0.925299366s) [noise=true , address=64.49.13.138:4487 , piece_id=NMZRB7X7MX2KJRDEUBROG3F6ZPQOGYJLTJ5WHBBFOUQCKXIHIZKQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.328832656 (459h21m0.925603704s) [noise=true , address=145.79.227.44:4201 , piece_id=SHQIEKFZSOKO7SUBGDC7GSJQNL47IEWENP5KN5ZN5ZOS4K3R7Y7A, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.328990765 (459h21m0.925761788s) [noise=true , address=77.90.31.17:4791 , piece_id=L5H6QECJGEQPTPRB6INQWHZPMK7ZXXDYYFFG5ANE37HAE33OTVXQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.329286413 (459h21m0.926057443s) [noise=true , address=64.49.13.27:4964 , piece_id=UOBMO3UGR2SVYEXRSW2SQMHSDJV47OXOZRZJHCOHVWNXMIIDSA2A, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.329346877 (459h21m0.9261179s) [noise=true , address=193.149.164.66:4984 , piece_id=CGM44RIHZGETSCKRZQOFL3VYIJTSB57RQDXTG3OPX3F3HJNQDYZA, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.329706811 (459h21m0.926477834s) [noise=true , address=45.91.116.7:4023 , piece_id=GYWVVXKLOAZSDJIS64INTFQIPQZCLBH7YPLTWY7QJSB766A3QTGQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.329728635 (459h21m0.926499661s) [noise=true , address=64.49.13.140:4509 , piece_id=7HN3FXTKC4RRUJZ34AHFDJHMUBIOWIQVFST626CR7X2ZYQS3CFCQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.330142941 (459h21m0.926913971s) [noise=true , address=145.79.227.34:4916 , piece_id=645H2KOPV4YB2HTP3XZF2RF5ZICIR34QNYRNRKX5KHH2VANVVZOQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.330171985 (459h21m0.926943015s) [noise=true , address=193.149.164.71:4296 , piece_id=RXZXV3FCVDEFHBJQVGIWEKS57EJ3IKAJFTDGMOEZHRRBHV3NRDSA, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.330543280 (459h21m0.927314303s) [noise=true , address=193.149.164.67:4030 , piece_id=PBIDHXYQ22HHK3HBJLFPC5X4L74Q332LNNEDYPFDV6SHWCQKRCNQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.330675401 (459h21m0.927446433s) [noise=true , address=64.49.13.26:4955 , piece_id=VIN3RVGRSUUHYCHPXCDNVSPRLFSCH3JXIDERBV7NIM4K33ZUBUGQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.330936609 (459h21m0.927707633s) [noise=true , address=193.149.164.68:4050 , piece_id=F4AQMCLJL6ML2LQKWUIKJAN6C2Y3M7UVYXRCRVRON5BC6CLTEK6Q, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.331355022 (459h21m0.928126051s) [noise=true , address=77.90.31.11:4567 , piece_id=ZSITPEN2OCYCHLSV3PDXCAXBAHEAQMS3YFCCOOE2SISJMUFRPHOA, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.402821184 (459h21m0.999592291s) [upload=514680]: Next returned nil segment
Mar 25 14:21:03.402848744 (459h21m0.99961978s) [upload=514680]: Waiting for error group managing segments...
Mar 25 14:21:03.408851473 (459h21m1.005622504s) [noise=true , address=77.90.31.16:4739 , piece_id=LEV3TZ2XFAYLQHULRJOG3DDBFMP3XRWX4EU2CQ2YOFICYXLIR3SA, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.409008646 (459h21m1.005779679s) [noise=true , address=145.79.227.34:4916 , piece_id=645H2KOPV4YB2HTP3XZF2RF5ZICIR34QNYRNRKX5KHH2VANVVZOQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.418584053 (459h21m1.015355084s) [noise=true , address=77.90.31.17:4791 , piece_id=L5H6QECJGEQPTPRB6INQWHZPMK7ZXXDYYFFG5ANE37HAE33OTVXQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.446927214 (459h21m1.043698248s) [noise=true , address=45.91.116.7:4023 , piece_id=GYWVVXKLOAZSDJIS64INTFQIPQZCLBH7YPLTWY7QJSB766A3QTGQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.448923805 (459h21m1.045694815s) [noise=true , address=77.90.31.14:4688 , piece_id=XDS6GE2YZK3LWUTBOMHLWTHM4ASCQJI25WTENUU75TJGFULYPOCQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.458340727 (459h21m1.055111751s) [noise=true , address=45.91.116.12:4254 , piece_id=UIQ3FWZSZ52RJYWFVWNH3OGNQPZSDYEZEDX35K3R7SDKRIW3WOAQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.468833670 (459h21m1.065604702s) [noise=true , address=77.90.31.11:4567 , piece_id=ZSITPEN2OCYCHLSV3PDXCAXBAHEAQMS3YFCCOOE2SISJMUFRPHOA, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.925971986 (459h21m1.522743012s) [noise=true , address=193.149.164.66:4984 , piece_id=CGM44RIHZGETSCKRZQOFL3VYIJTSB57RQDXTG3OPX3F3HJNQDYZA, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.928271525 (459h21m1.525042552s) [noise=true , address=64.49.13.140:4509 , piece_id=7HN3FXTKC4RRUJZ34AHFDJHMUBIOWIQVFST626CR7X2ZYQS3CFCQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.929087375 (459h21m1.525858388s) [noise=true , address=193.149.164.70:4120 , piece_id=HMFOJCRQTG3PAQDIZERNEYWKFFBUIRKRXP56OQUQWH6R2YX6NYLQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.929851130 (459h21m1.526622145s) [noise=true , address=193.149.164.71:4296 , piece_id=RXZXV3FCVDEFHBJQVGIWEKS57EJ3IKAJFTDGMOEZHRRBHV3NRDSA, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.935607616 (459h21m1.532378643s) [noise=true , address=193.149.164.69:4085 , piece_id=KGCVBEK344OW573PN2Y55JA7AC3ZTQD36G3T6TUHF7ZMZ4WN7N5A, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.938874518 (459h21m1.535645546s) [noise=true , address=193.149.164.67:4030 , piece_id=PBIDHXYQ22HHK3HBJLFPC5X4L74Q332LNNEDYPFDV6SHWCQKRCNQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.950243460 (459h21m1.547014493s) [noise=true , address=64.49.13.26:4955 , piece_id=VIN3RVGRSUUHYCHPXCDNVSPRLFSCH3JXIDERBV7NIM4K33ZUBUGQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.950277580 (459h21m1.54704861s) [seg_pos={0 0}, upload=514680]: Satisfied cohort requirements after 14 pieces
Mar 25 14:21:03.950707500 (459h21m1.547478532s) [noise=true , address=193.149.164.68:4050 , piece_id=F4AQMCLJL6ML2LQKWUIKJAN6C2Y3M7UVYXRCRVRON5BC6CLTEK6Q, seg_pos={0 0}, upload=514680]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1WPfYrCJrhPTqZ2qgBvXFCUodKz6QeXCnkKgToFs5muNRSSLtK): protocol: expected piece hash; context canceled; EOF
Mar 25 14:21:03.950721648 (459h21m1.547492674s) [noise=true , address=193.149.164.68:4050 , piece_id=F4AQMCLJL6ML2LQKWUIKJAN6C2Y3M7UVYXRCRVRON5BC6CLTEK6Q, seg_pos={0 0}, upload=514680]: Long tail cancellation due to optimal threshold
Mar 25 14:21:03.950793002 (459h21m1.547564034s) [noise=true , address=64.49.13.27:4964 , piece_id=UOBMO3UGR2SVYEXRSW2SQMHSDJV47OXOZRZJHCOHVWNXMIIDSA2A, seg_pos={0 0}, upload=514680]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RcgfaeKP3XpZGuSB9GMURCCsjjc79WeZasUL2RRiQVDwxbnAk): protocol: expected piece hash; context canceled; EOF
Mar 25 14:21:03.950804838 (459h21m1.547575864s) [noise=true , address=64.49.13.27:4964 , piece_id=UOBMO3UGR2SVYEXRSW2SQMHSDJV47OXOZRZJHCOHVWNXMIIDSA2A, seg_pos={0 0}, upload=514680]: Long tail cancellation due to optimal threshold
Mar 25 14:21:03.950899119 (459h21m1.547670144s) [noise=true , address=145.79.227.44:4201 , piece_id=SHQIEKFZSOKO7SUBGDC7GSJQNL47IEWENP5KN5ZN5ZOS4K3R7Y7A, seg_pos={0 0}, upload=514680]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bPDvhDfu8Z58LxZcicg1vAiiu1sZYGpwQnvuhk4t5m6yR87Fi): protocol: expected piece hash; context canceled; EOF
Mar 25 14:21:03.950907089 (459h21m1.547678114s) [noise=true , address=145.79.227.44:4201 , piece_id=SHQIEKFZSOKO7SUBGDC7GSJQNL47IEWENP5KN5ZN5ZOS4K3R7Y7A, seg_pos={0 0}, upload=514680]: Long tail cancellation due to optimal threshold
Mar 25 14:21:03.951133312 (459h21m1.547904347s) [noise=true , address=64.49.13.138:4487 , piece_id=NMZRB7X7MX2KJRDEUBROG3F6ZPQOGYJLTJ5WHBBFOUQCKXIHIZKQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12taemzrausuc58d8Dyo8xaR7MqDf72f37rEheF32sUfy8Z8GZP): protocol: expected piece hash; context canceled; EOF
Mar 25 14:21:03.951146101 (459h21m1.547917132s) [noise=true , address=64.49.13.138:4487 , piece_id=NMZRB7X7MX2KJRDEUBROG3F6ZPQOGYJLTJ5WHBBFOUQCKXIHIZKQ, seg_pos={0 0}, upload=514680]: Long tail cancellation due to optimal threshold
Mar 25 14:21:03.951175027 (459h21m1.547946039s) [seg_pos={0 0}, upload=514680]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Mar 25 14:21:03.951282270 (459h21m1.548053293s) [upload=514680]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Mar 25 14:21:03.951290421 (459h21m1.548061457s) [upload=514680]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Mar 25 14:21:04.013599996 (459h21m1.610371022s) [upload=514680]: Done uploading segments...
Mar 25 14:21:04.013621043 (459h21m1.610392066s) [upload=514680]: Upload finished. err: <nil>
Duration: 1.4316695s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [........X...................................................] duration:
36.042831ms target:
https://us1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/21096334d2fd8925
FAIL (2026-03-25 14:59:19 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.4.32.59:443: i/o timeout
Duration: 36.069052ms
core
satellite
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................X..........................................] duration:
1.226209465s trace:
https://jaeger.dev.storj.io/trace/62357518aac8ca5f
FAIL (2026-03-25 14:59:11 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 1.226241379s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [.........X..................................................] duration:
31.448883ms target:
https://us1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/21096334d2fd8925
FAIL (2026-03-25 14:58:57 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.4.32.59:443: i/o timeout
Duration: 31.538005ms
core
satellite
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................X.........................................] duration:
1.314679067s trace:
https://jaeger.dev.storj.io/trace/62357518aac8ca5f
FAIL (2026-03-25 14:58:33 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 1.314714921s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........X.................................................] duration:
1.77960956s trace:
https://jaeger.dev.storj.io/trace/1d1f0bc4e4438495
FAIL (2026-03-25 14:58:20 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
upload=514680]: Uploading piece...
Mar 25 14:21:03.327216643 (459h21m0.923987655s) [noise=true , address=45.91.116.12:4254 , piece_id=UIQ3FWZSZ52RJYWFVWNH3OGNQPZSDYEZEDX35K3R7SDKRIW3WOAQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.327580267 (459h21m0.924351301s) [noise=true , address=193.149.164.70:4120 , piece_id=HMFOJCRQTG3PAQDIZERNEYWKFFBUIRKRXP56OQUQWH6R2YX6NYLQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.327966258 (459h21m0.924737281s) [noise=true , address=77.90.31.16:4739 , piece_id=LEV3TZ2XFAYLQHULRJOG3DDBFMP3XRWX4EU2CQ2YOFICYXLIR3SA, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.328304150 (459h21m0.925075204s) [noise=true , address=77.90.31.14:4688 , piece_id=XDS6GE2YZK3LWUTBOMHLWTHM4ASCQJI25WTENUU75TJGFULYPOCQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.328528343 (459h21m0.925299366s) [noise=true , address=64.49.13.138:4487 , piece_id=NMZRB7X7MX2KJRDEUBROG3F6ZPQOGYJLTJ5WHBBFOUQCKXIHIZKQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.328832656 (459h21m0.925603704s) [noise=true , address=145.79.227.44:4201 , piece_id=SHQIEKFZSOKO7SUBGDC7GSJQNL47IEWENP5KN5ZN5ZOS4K3R7Y7A, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.328990765 (459h21m0.925761788s) [noise=true , address=77.90.31.17:4791 , piece_id=L5H6QECJGEQPTPRB6INQWHZPMK7ZXXDYYFFG5ANE37HAE33OTVXQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.329286413 (459h21m0.926057443s) [noise=true , address=64.49.13.27:4964 , piece_id=UOBMO3UGR2SVYEXRSW2SQMHSDJV47OXOZRZJHCOHVWNXMIIDSA2A, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.329346877 (459h21m0.9261179s) [noise=true , address=193.149.164.66:4984 , piece_id=CGM44RIHZGETSCKRZQOFL3VYIJTSB57RQDXTG3OPX3F3HJNQDYZA, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.329706811 (459h21m0.926477834s) [noise=true , address=45.91.116.7:4023 , piece_id=GYWVVXKLOAZSDJIS64INTFQIPQZCLBH7YPLTWY7QJSB766A3QTGQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.329728635 (459h21m0.926499661s) [noise=true , address=64.49.13.140:4509 , piece_id=7HN3FXTKC4RRUJZ34AHFDJHMUBIOWIQVFST626CR7X2ZYQS3CFCQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.330142941 (459h21m0.926913971s) [noise=true , address=145.79.227.34:4916 , piece_id=645H2KOPV4YB2HTP3XZF2RF5ZICIR34QNYRNRKX5KHH2VANVVZOQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.330171985 (459h21m0.926943015s) [noise=true , address=193.149.164.71:4296 , piece_id=RXZXV3FCVDEFHBJQVGIWEKS57EJ3IKAJFTDGMOEZHRRBHV3NRDSA, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.330543280 (459h21m0.927314303s) [noise=true , address=193.149.164.67:4030 , piece_id=PBIDHXYQ22HHK3HBJLFPC5X4L74Q332LNNEDYPFDV6SHWCQKRCNQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.330675401 (459h21m0.927446433s) [noise=true , address=64.49.13.26:4955 , piece_id=VIN3RVGRSUUHYCHPXCDNVSPRLFSCH3JXIDERBV7NIM4K33ZUBUGQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.330936609 (459h21m0.927707633s) [noise=true , address=193.149.164.68:4050 , piece_id=F4AQMCLJL6ML2LQKWUIKJAN6C2Y3M7UVYXRCRVRON5BC6CLTEK6Q, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.331355022 (459h21m0.928126051s) [noise=true , address=77.90.31.11:4567 , piece_id=ZSITPEN2OCYCHLSV3PDXCAXBAHEAQMS3YFCCOOE2SISJMUFRPHOA, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.402821184 (459h21m0.999592291s) [upload=514680]: Next returned nil segment
Mar 25 14:21:03.402848744 (459h21m0.99961978s) [upload=514680]: Waiting for error group managing segments...
Mar 25 14:21:03.408851473 (459h21m1.005622504s) [noise=true , address=77.90.31.16:4739 , piece_id=LEV3TZ2XFAYLQHULRJOG3DDBFMP3XRWX4EU2CQ2YOFICYXLIR3SA, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.409008646 (459h21m1.005779679s) [noise=true , address=145.79.227.34:4916 , piece_id=645H2KOPV4YB2HTP3XZF2RF5ZICIR34QNYRNRKX5KHH2VANVVZOQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.418584053 (459h21m1.015355084s) [noise=true , address=77.90.31.17:4791 , piece_id=L5H6QECJGEQPTPRB6INQWHZPMK7ZXXDYYFFG5ANE37HAE33OTVXQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.446927214 (459h21m1.043698248s) [noise=true , address=45.91.116.7:4023 , piece_id=GYWVVXKLOAZSDJIS64INTFQIPQZCLBH7YPLTWY7QJSB766A3QTGQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.448923805 (459h21m1.045694815s) [noise=true , address=77.90.31.14:4688 , piece_id=XDS6GE2YZK3LWUTBOMHLWTHM4ASCQJI25WTENUU75TJGFULYPOCQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.458340727 (459h21m1.055111751s) [noise=true , address=45.91.116.12:4254 , piece_id=UIQ3FWZSZ52RJYWFVWNH3OGNQPZSDYEZEDX35K3R7SDKRIW3WOAQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.468833670 (459h21m1.065604702s) [noise=true , address=77.90.31.11:4567 , piece_id=ZSITPEN2OCYCHLSV3PDXCAXBAHEAQMS3YFCCOOE2SISJMUFRPHOA, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.925971986 (459h21m1.522743012s) [noise=true , address=193.149.164.66:4984 , piece_id=CGM44RIHZGETSCKRZQOFL3VYIJTSB57RQDXTG3OPX3F3HJNQDYZA, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.928271525 (459h21m1.525042552s) [noise=true , address=64.49.13.140:4509 , piece_id=7HN3FXTKC4RRUJZ34AHFDJHMUBIOWIQVFST626CR7X2ZYQS3CFCQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.929087375 (459h21m1.525858388s) [noise=true , address=193.149.164.70:4120 , piece_id=HMFOJCRQTG3PAQDIZERNEYWKFFBUIRKRXP56OQUQWH6R2YX6NYLQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.929851130 (459h21m1.526622145s) [noise=true , address=193.149.164.71:4296 , piece_id=RXZXV3FCVDEFHBJQVGIWEKS57EJ3IKAJFTDGMOEZHRRBHV3NRDSA, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.935607616 (459h21m1.532378643s) [noise=true , address=193.149.164.69:4085 , piece_id=KGCVBEK344OW573PN2Y55JA7AC3ZTQD36G3T6TUHF7ZMZ4WN7N5A, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.938874518 (459h21m1.535645546s) [noise=true , address=193.149.164.67:4030 , piece_id=PBIDHXYQ22HHK3HBJLFPC5X4L74Q332LNNEDYPFDV6SHWCQKRCNQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.950243460 (459h21m1.547014493s) [noise=true , address=64.49.13.26:4955 , piece_id=VIN3RVGRSUUHYCHPXCDNVSPRLFSCH3JXIDERBV7NIM4K33ZUBUGQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.950277580 (459h21m1.54704861s) [seg_pos={0 0}, upload=514680]: Satisfied cohort requirements after 14 pieces
Mar 25 14:21:03.950707500 (459h21m1.547478532s) [noise=true , address=193.149.164.68:4050 , piece_id=F4AQMCLJL6ML2LQKWUIKJAN6C2Y3M7UVYXRCRVRON5BC6CLTEK6Q, seg_pos={0 0}, upload=514680]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1WPfYrCJrhPTqZ2qgBvXFCUodKz6QeXCnkKgToFs5muNRSSLtK): protocol: expected piece hash; context canceled; EOF
Mar 25 14:21:03.950721648 (459h21m1.547492674s) [noise=true , address=193.149.164.68:4050 , piece_id=F4AQMCLJL6ML2LQKWUIKJAN6C2Y3M7UVYXRCRVRON5BC6CLTEK6Q, seg_pos={0 0}, upload=514680]: Long tail cancellation due to optimal threshold
Mar 25 14:21:03.950793002 (459h21m1.547564034s) [noise=true , address=64.49.13.27:4964 , piece_id=UOBMO3UGR2SVYEXRSW2SQMHSDJV47OXOZRZJHCOHVWNXMIIDSA2A, seg_pos={0 0}, upload=514680]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RcgfaeKP3XpZGuSB9GMURCCsjjc79WeZasUL2RRiQVDwxbnAk): protocol: expected piece hash; context canceled; EOF
Mar 25 14:21:03.950804838 (459h21m1.547575864s) [noise=true , address=64.49.13.27:4964 , piece_id=UOBMO3UGR2SVYEXRSW2SQMHSDJV47OXOZRZJHCOHVWNXMIIDSA2A, seg_pos={0 0}, upload=514680]: Long tail cancellation due to optimal threshold
Mar 25 14:21:03.950899119 (459h21m1.547670144s) [noise=true , address=145.79.227.44:4201 , piece_id=SHQIEKFZSOKO7SUBGDC7GSJQNL47IEWENP5KN5ZN5ZOS4K3R7Y7A, seg_pos={0 0}, upload=514680]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bPDvhDfu8Z58LxZcicg1vAiiu1sZYGpwQnvuhk4t5m6yR87Fi): protocol: expected piece hash; context canceled; EOF
Mar 25 14:21:03.950907089 (459h21m1.547678114s) [noise=true , address=145.79.227.44:4201 , piece_id=SHQIEKFZSOKO7SUBGDC7GSJQNL47IEWENP5KN5ZN5ZOS4K3R7Y7A, seg_pos={0 0}, upload=514680]: Long tail cancellation due to optimal threshold
Mar 25 14:21:03.951133312 (459h21m1.547904347s) [noise=true , address=64.49.13.138:4487 , piece_id=NMZRB7X7MX2KJRDEUBROG3F6ZPQOGYJLTJ5WHBBFOUQCKXIHIZKQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12taemzrausuc58d8Dyo8xaR7MqDf72f37rEheF32sUfy8Z8GZP): protocol: expected piece hash; context canceled; EOF
Mar 25 14:21:03.951146101 (459h21m1.547917132s) [noise=true , address=64.49.13.138:4487 , piece_id=NMZRB7X7MX2KJRDEUBROG3F6ZPQOGYJLTJ5WHBBFOUQCKXIHIZKQ, seg_pos={0 0}, upload=514680]: Long tail cancellation due to optimal threshold
Mar 25 14:21:03.951175027 (459h21m1.547946039s) [seg_pos={0 0}, upload=514680]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Mar 25 14:21:03.951282270 (459h21m1.548053293s) [upload=514680]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Mar 25 14:21:03.951290421 (459h21m1.548061457s) [upload=514680]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Mar 25 14:21:04.013599996 (459h21m1.610371022s) [upload=514680]: Done uploading segments...
Mar 25 14:21:04.013621043 (459h21m1.610392066s) [upload=514680]: Upload finished. err: <nil>
Duration: 1.779696253s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [..........X.................................................] duration:
45.901953ms target:
https://us1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/21096334d2fd8925
FAIL (2026-03-25 14:58:09 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.4.32.59:443: i/o timeout
Duration: 45.931241ms
core
satellite
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................X........................................] duration:
1.287092219s trace:
https://jaeger.dev.storj.io/trace/62357518aac8ca5f
FAIL (2026-03-25 14:57:51 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 1.287125211s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [...........X................................................] duration:
32.467045ms target:
https://us1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/21096334d2fd8925
FAIL (2026-03-25 14:57:45 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.4.32.59:443: i/o timeout
Duration: 32.495195ms
core
satellite
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........X................................................] duration:
1.748474843s trace:
https://jaeger.dev.storj.io/trace/1d1f0bc4e4438495
FAIL (2026-03-25 14:57:30 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
upload=514680]: Uploading piece...
Mar 25 14:21:03.327216643 (459h21m0.923987655s) [noise=true , address=45.91.116.12:4254 , piece_id=UIQ3FWZSZ52RJYWFVWNH3OGNQPZSDYEZEDX35K3R7SDKRIW3WOAQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.327580267 (459h21m0.924351301s) [noise=true , address=193.149.164.70:4120 , piece_id=HMFOJCRQTG3PAQDIZERNEYWKFFBUIRKRXP56OQUQWH6R2YX6NYLQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.327966258 (459h21m0.924737281s) [noise=true , address=77.90.31.16:4739 , piece_id=LEV3TZ2XFAYLQHULRJOG3DDBFMP3XRWX4EU2CQ2YOFICYXLIR3SA, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.328304150 (459h21m0.925075204s) [noise=true , address=77.90.31.14:4688 , piece_id=XDS6GE2YZK3LWUTBOMHLWTHM4ASCQJI25WTENUU75TJGFULYPOCQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.328528343 (459h21m0.925299366s) [noise=true , address=64.49.13.138:4487 , piece_id=NMZRB7X7MX2KJRDEUBROG3F6ZPQOGYJLTJ5WHBBFOUQCKXIHIZKQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.328832656 (459h21m0.925603704s) [noise=true , address=145.79.227.44:4201 , piece_id=SHQIEKFZSOKO7SUBGDC7GSJQNL47IEWENP5KN5ZN5ZOS4K3R7Y7A, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.328990765 (459h21m0.925761788s) [noise=true , address=77.90.31.17:4791 , piece_id=L5H6QECJGEQPTPRB6INQWHZPMK7ZXXDYYFFG5ANE37HAE33OTVXQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.329286413 (459h21m0.926057443s) [noise=true , address=64.49.13.27:4964 , piece_id=UOBMO3UGR2SVYEXRSW2SQMHSDJV47OXOZRZJHCOHVWNXMIIDSA2A, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.329346877 (459h21m0.9261179s) [noise=true , address=193.149.164.66:4984 , piece_id=CGM44RIHZGETSCKRZQOFL3VYIJTSB57RQDXTG3OPX3F3HJNQDYZA, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.329706811 (459h21m0.926477834s) [noise=true , address=45.91.116.7:4023 , piece_id=GYWVVXKLOAZSDJIS64INTFQIPQZCLBH7YPLTWY7QJSB766A3QTGQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.329728635 (459h21m0.926499661s) [noise=true , address=64.49.13.140:4509 , piece_id=7HN3FXTKC4RRUJZ34AHFDJHMUBIOWIQVFST626CR7X2ZYQS3CFCQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.330142941 (459h21m0.926913971s) [noise=true , address=145.79.227.34:4916 , piece_id=645H2KOPV4YB2HTP3XZF2RF5ZICIR34QNYRNRKX5KHH2VANVVZOQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.330171985 (459h21m0.926943015s) [noise=true , address=193.149.164.71:4296 , piece_id=RXZXV3FCVDEFHBJQVGIWEKS57EJ3IKAJFTDGMOEZHRRBHV3NRDSA, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.330543280 (459h21m0.927314303s) [noise=true , address=193.149.164.67:4030 , piece_id=PBIDHXYQ22HHK3HBJLFPC5X4L74Q332LNNEDYPFDV6SHWCQKRCNQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.330675401 (459h21m0.927446433s) [noise=true , address=64.49.13.26:4955 , piece_id=VIN3RVGRSUUHYCHPXCDNVSPRLFSCH3JXIDERBV7NIM4K33ZUBUGQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.330936609 (459h21m0.927707633s) [noise=true , address=193.149.164.68:4050 , piece_id=F4AQMCLJL6ML2LQKWUIKJAN6C2Y3M7UVYXRCRVRON5BC6CLTEK6Q, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.331355022 (459h21m0.928126051s) [noise=true , address=77.90.31.11:4567 , piece_id=ZSITPEN2OCYCHLSV3PDXCAXBAHEAQMS3YFCCOOE2SISJMUFRPHOA, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.402821184 (459h21m0.999592291s) [upload=514680]: Next returned nil segment
Mar 25 14:21:03.402848744 (459h21m0.99961978s) [upload=514680]: Waiting for error group managing segments...
Mar 25 14:21:03.408851473 (459h21m1.005622504s) [noise=true , address=77.90.31.16:4739 , piece_id=LEV3TZ2XFAYLQHULRJOG3DDBFMP3XRWX4EU2CQ2YOFICYXLIR3SA, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.409008646 (459h21m1.005779679s) [noise=true , address=145.79.227.34:4916 , piece_id=645H2KOPV4YB2HTP3XZF2RF5ZICIR34QNYRNRKX5KHH2VANVVZOQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.418584053 (459h21m1.015355084s) [noise=true , address=77.90.31.17:4791 , piece_id=L5H6QECJGEQPTPRB6INQWHZPMK7ZXXDYYFFG5ANE37HAE33OTVXQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.446927214 (459h21m1.043698248s) [noise=true , address=45.91.116.7:4023 , piece_id=GYWVVXKLOAZSDJIS64INTFQIPQZCLBH7YPLTWY7QJSB766A3QTGQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.448923805 (459h21m1.045694815s) [noise=true , address=77.90.31.14:4688 , piece_id=XDS6GE2YZK3LWUTBOMHLWTHM4ASCQJI25WTENUU75TJGFULYPOCQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.458340727 (459h21m1.055111751s) [noise=true , address=45.91.116.12:4254 , piece_id=UIQ3FWZSZ52RJYWFVWNH3OGNQPZSDYEZEDX35K3R7SDKRIW3WOAQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.468833670 (459h21m1.065604702s) [noise=true , address=77.90.31.11:4567 , piece_id=ZSITPEN2OCYCHLSV3PDXCAXBAHEAQMS3YFCCOOE2SISJMUFRPHOA, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.925971986 (459h21m1.522743012s) [noise=true , address=193.149.164.66:4984 , piece_id=CGM44RIHZGETSCKRZQOFL3VYIJTSB57RQDXTG3OPX3F3HJNQDYZA, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.928271525 (459h21m1.525042552s) [noise=true , address=64.49.13.140:4509 , piece_id=7HN3FXTKC4RRUJZ34AHFDJHMUBIOWIQVFST626CR7X2ZYQS3CFCQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.929087375 (459h21m1.525858388s) [noise=true , address=193.149.164.70:4120 , piece_id=HMFOJCRQTG3PAQDIZERNEYWKFFBUIRKRXP56OQUQWH6R2YX6NYLQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.929851130 (459h21m1.526622145s) [noise=true , address=193.149.164.71:4296 , piece_id=RXZXV3FCVDEFHBJQVGIWEKS57EJ3IKAJFTDGMOEZHRRBHV3NRDSA, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.935607616 (459h21m1.532378643s) [noise=true , address=193.149.164.69:4085 , piece_id=KGCVBEK344OW573PN2Y55JA7AC3ZTQD36G3T6TUHF7ZMZ4WN7N5A, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.938874518 (459h21m1.535645546s) [noise=true , address=193.149.164.67:4030 , piece_id=PBIDHXYQ22HHK3HBJLFPC5X4L74Q332LNNEDYPFDV6SHWCQKRCNQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.950243460 (459h21m1.547014493s) [noise=true , address=64.49.13.26:4955 , piece_id=VIN3RVGRSUUHYCHPXCDNVSPRLFSCH3JXIDERBV7NIM4K33ZUBUGQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.950277580 (459h21m1.54704861s) [seg_pos={0 0}, upload=514680]: Satisfied cohort requirements after 14 pieces
Mar 25 14:21:03.950707500 (459h21m1.547478532s) [noise=true , address=193.149.164.68:4050 , piece_id=F4AQMCLJL6ML2LQKWUIKJAN6C2Y3M7UVYXRCRVRON5BC6CLTEK6Q, seg_pos={0 0}, upload=514680]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1WPfYrCJrhPTqZ2qgBvXFCUodKz6QeXCnkKgToFs5muNRSSLtK): protocol: expected piece hash; context canceled; EOF
Mar 25 14:21:03.950721648 (459h21m1.547492674s) [noise=true , address=193.149.164.68:4050 , piece_id=F4AQMCLJL6ML2LQKWUIKJAN6C2Y3M7UVYXRCRVRON5BC6CLTEK6Q, seg_pos={0 0}, upload=514680]: Long tail cancellation due to optimal threshold
Mar 25 14:21:03.950793002 (459h21m1.547564034s) [noise=true , address=64.49.13.27:4964 , piece_id=UOBMO3UGR2SVYEXRSW2SQMHSDJV47OXOZRZJHCOHVWNXMIIDSA2A, seg_pos={0 0}, upload=514680]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RcgfaeKP3XpZGuSB9GMURCCsjjc79WeZasUL2RRiQVDwxbnAk): protocol: expected piece hash; context canceled; EOF
Mar 25 14:21:03.950804838 (459h21m1.547575864s) [noise=true , address=64.49.13.27:4964 , piece_id=UOBMO3UGR2SVYEXRSW2SQMHSDJV47OXOZRZJHCOHVWNXMIIDSA2A, seg_pos={0 0}, upload=514680]: Long tail cancellation due to optimal threshold
Mar 25 14:21:03.950899119 (459h21m1.547670144s) [noise=true , address=145.79.227.44:4201 , piece_id=SHQIEKFZSOKO7SUBGDC7GSJQNL47IEWENP5KN5ZN5ZOS4K3R7Y7A, seg_pos={0 0}, upload=514680]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bPDvhDfu8Z58LxZcicg1vAiiu1sZYGpwQnvuhk4t5m6yR87Fi): protocol: expected piece hash; context canceled; EOF
Mar 25 14:21:03.950907089 (459h21m1.547678114s) [noise=true , address=145.79.227.44:4201 , piece_id=SHQIEKFZSOKO7SUBGDC7GSJQNL47IEWENP5KN5ZN5ZOS4K3R7Y7A, seg_pos={0 0}, upload=514680]: Long tail cancellation due to optimal threshold
Mar 25 14:21:03.951133312 (459h21m1.547904347s) [noise=true , address=64.49.13.138:4487 , piece_id=NMZRB7X7MX2KJRDEUBROG3F6ZPQOGYJLTJ5WHBBFOUQCKXIHIZKQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12taemzrausuc58d8Dyo8xaR7MqDf72f37rEheF32sUfy8Z8GZP): protocol: expected piece hash; context canceled; EOF
Mar 25 14:21:03.951146101 (459h21m1.547917132s) [noise=true , address=64.49.13.138:4487 , piece_id=NMZRB7X7MX2KJRDEUBROG3F6ZPQOGYJLTJ5WHBBFOUQCKXIHIZKQ, seg_pos={0 0}, upload=514680]: Long tail cancellation due to optimal threshold
Mar 25 14:21:03.951175027 (459h21m1.547946039s) [seg_pos={0 0}, upload=514680]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Mar 25 14:21:03.951282270 (459h21m1.548053293s) [upload=514680]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Mar 25 14:21:03.951290421 (459h21m1.548061457s) [upload=514680]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Mar 25 14:21:04.013599996 (459h21m1.610371022s) [upload=514680]: Done uploading segments...
Mar 25 14:21:04.013621043 (459h21m1.610392066s) [upload=514680]: Upload finished. err: <nil>
Duration: 1.748509478s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [............X...............................................] duration:
1.385683812s trace:
https://jaeger.dev.storj.io/trace/1d1f0bc4e4438495
FAIL (2026-03-25 14:57:06 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
upload=514680]: Uploading piece...
Mar 25 14:21:03.327216643 (459h21m0.923987655s) [noise=true , address=45.91.116.12:4254 , piece_id=UIQ3FWZSZ52RJYWFVWNH3OGNQPZSDYEZEDX35K3R7SDKRIW3WOAQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.327580267 (459h21m0.924351301s) [noise=true , address=193.149.164.70:4120 , piece_id=HMFOJCRQTG3PAQDIZERNEYWKFFBUIRKRXP56OQUQWH6R2YX6NYLQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.327966258 (459h21m0.924737281s) [noise=true , address=77.90.31.16:4739 , piece_id=LEV3TZ2XFAYLQHULRJOG3DDBFMP3XRWX4EU2CQ2YOFICYXLIR3SA, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.328304150 (459h21m0.925075204s) [noise=true , address=77.90.31.14:4688 , piece_id=XDS6GE2YZK3LWUTBOMHLWTHM4ASCQJI25WTENUU75TJGFULYPOCQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.328528343 (459h21m0.925299366s) [noise=true , address=64.49.13.138:4487 , piece_id=NMZRB7X7MX2KJRDEUBROG3F6ZPQOGYJLTJ5WHBBFOUQCKXIHIZKQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.328832656 (459h21m0.925603704s) [noise=true , address=145.79.227.44:4201 , piece_id=SHQIEKFZSOKO7SUBGDC7GSJQNL47IEWENP5KN5ZN5ZOS4K3R7Y7A, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.328990765 (459h21m0.925761788s) [noise=true , address=77.90.31.17:4791 , piece_id=L5H6QECJGEQPTPRB6INQWHZPMK7ZXXDYYFFG5ANE37HAE33OTVXQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.329286413 (459h21m0.926057443s) [noise=true , address=64.49.13.27:4964 , piece_id=UOBMO3UGR2SVYEXRSW2SQMHSDJV47OXOZRZJHCOHVWNXMIIDSA2A, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.329346877 (459h21m0.9261179s) [noise=true , address=193.149.164.66:4984 , piece_id=CGM44RIHZGETSCKRZQOFL3VYIJTSB57RQDXTG3OPX3F3HJNQDYZA, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.329706811 (459h21m0.926477834s) [noise=true , address=45.91.116.7:4023 , piece_id=GYWVVXKLOAZSDJIS64INTFQIPQZCLBH7YPLTWY7QJSB766A3QTGQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.329728635 (459h21m0.926499661s) [noise=true , address=64.49.13.140:4509 , piece_id=7HN3FXTKC4RRUJZ34AHFDJHMUBIOWIQVFST626CR7X2ZYQS3CFCQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.330142941 (459h21m0.926913971s) [noise=true , address=145.79.227.34:4916 , piece_id=645H2KOPV4YB2HTP3XZF2RF5ZICIR34QNYRNRKX5KHH2VANVVZOQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.330171985 (459h21m0.926943015s) [noise=true , address=193.149.164.71:4296 , piece_id=RXZXV3FCVDEFHBJQVGIWEKS57EJ3IKAJFTDGMOEZHRRBHV3NRDSA, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.330543280 (459h21m0.927314303s) [noise=true , address=193.149.164.67:4030 , piece_id=PBIDHXYQ22HHK3HBJLFPC5X4L74Q332LNNEDYPFDV6SHWCQKRCNQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.330675401 (459h21m0.927446433s) [noise=true , address=64.49.13.26:4955 , piece_id=VIN3RVGRSUUHYCHPXCDNVSPRLFSCH3JXIDERBV7NIM4K33ZUBUGQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.330936609 (459h21m0.927707633s) [noise=true , address=193.149.164.68:4050 , piece_id=F4AQMCLJL6ML2LQKWUIKJAN6C2Y3M7UVYXRCRVRON5BC6CLTEK6Q, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.331355022 (459h21m0.928126051s) [noise=true , address=77.90.31.11:4567 , piece_id=ZSITPEN2OCYCHLSV3PDXCAXBAHEAQMS3YFCCOOE2SISJMUFRPHOA, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.402821184 (459h21m0.999592291s) [upload=514680]: Next returned nil segment
Mar 25 14:21:03.402848744 (459h21m0.99961978s) [upload=514680]: Waiting for error group managing segments...
Mar 25 14:21:03.408851473 (459h21m1.005622504s) [noise=true , address=77.90.31.16:4739 , piece_id=LEV3TZ2XFAYLQHULRJOG3DDBFMP3XRWX4EU2CQ2YOFICYXLIR3SA, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.409008646 (459h21m1.005779679s) [noise=true , address=145.79.227.34:4916 , piece_id=645H2KOPV4YB2HTP3XZF2RF5ZICIR34QNYRNRKX5KHH2VANVVZOQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.418584053 (459h21m1.015355084s) [noise=true , address=77.90.31.17:4791 , piece_id=L5H6QECJGEQPTPRB6INQWHZPMK7ZXXDYYFFG5ANE37HAE33OTVXQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.446927214 (459h21m1.043698248s) [noise=true , address=45.91.116.7:4023 , piece_id=GYWVVXKLOAZSDJIS64INTFQIPQZCLBH7YPLTWY7QJSB766A3QTGQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.448923805 (459h21m1.045694815s) [noise=true , address=77.90.31.14:4688 , piece_id=XDS6GE2YZK3LWUTBOMHLWTHM4ASCQJI25WTENUU75TJGFULYPOCQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.458340727 (459h21m1.055111751s) [noise=true , address=45.91.116.12:4254 , piece_id=UIQ3FWZSZ52RJYWFVWNH3OGNQPZSDYEZEDX35K3R7SDKRIW3WOAQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.468833670 (459h21m1.065604702s) [noise=true , address=77.90.31.11:4567 , piece_id=ZSITPEN2OCYCHLSV3PDXCAXBAHEAQMS3YFCCOOE2SISJMUFRPHOA, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.925971986 (459h21m1.522743012s) [noise=true , address=193.149.164.66:4984 , piece_id=CGM44RIHZGETSCKRZQOFL3VYIJTSB57RQDXTG3OPX3F3HJNQDYZA, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.928271525 (459h21m1.525042552s) [noise=true , address=64.49.13.140:4509 , piece_id=7HN3FXTKC4RRUJZ34AHFDJHMUBIOWIQVFST626CR7X2ZYQS3CFCQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.929087375 (459h21m1.525858388s) [noise=true , address=193.149.164.70:4120 , piece_id=HMFOJCRQTG3PAQDIZERNEYWKFFBUIRKRXP56OQUQWH6R2YX6NYLQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.929851130 (459h21m1.526622145s) [noise=true , address=193.149.164.71:4296 , piece_id=RXZXV3FCVDEFHBJQVGIWEKS57EJ3IKAJFTDGMOEZHRRBHV3NRDSA, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.935607616 (459h21m1.532378643s) [noise=true , address=193.149.164.69:4085 , piece_id=KGCVBEK344OW573PN2Y55JA7AC3ZTQD36G3T6TUHF7ZMZ4WN7N5A, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.938874518 (459h21m1.535645546s) [noise=true , address=193.149.164.67:4030 , piece_id=PBIDHXYQ22HHK3HBJLFPC5X4L74Q332LNNEDYPFDV6SHWCQKRCNQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.950243460 (459h21m1.547014493s) [noise=true , address=64.49.13.26:4955 , piece_id=VIN3RVGRSUUHYCHPXCDNVSPRLFSCH3JXIDERBV7NIM4K33ZUBUGQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.950277580 (459h21m1.54704861s) [seg_pos={0 0}, upload=514680]: Satisfied cohort requirements after 14 pieces
Mar 25 14:21:03.950707500 (459h21m1.547478532s) [noise=true , address=193.149.164.68:4050 , piece_id=F4AQMCLJL6ML2LQKWUIKJAN6C2Y3M7UVYXRCRVRON5BC6CLTEK6Q, seg_pos={0 0}, upload=514680]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1WPfYrCJrhPTqZ2qgBvXFCUodKz6QeXCnkKgToFs5muNRSSLtK): protocol: expected piece hash; context canceled; EOF
Mar 25 14:21:03.950721648 (459h21m1.547492674s) [noise=true , address=193.149.164.68:4050 , piece_id=F4AQMCLJL6ML2LQKWUIKJAN6C2Y3M7UVYXRCRVRON5BC6CLTEK6Q, seg_pos={0 0}, upload=514680]: Long tail cancellation due to optimal threshold
Mar 25 14:21:03.950793002 (459h21m1.547564034s) [noise=true , address=64.49.13.27:4964 , piece_id=UOBMO3UGR2SVYEXRSW2SQMHSDJV47OXOZRZJHCOHVWNXMIIDSA2A, seg_pos={0 0}, upload=514680]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RcgfaeKP3XpZGuSB9GMURCCsjjc79WeZasUL2RRiQVDwxbnAk): protocol: expected piece hash; context canceled; EOF
Mar 25 14:21:03.950804838 (459h21m1.547575864s) [noise=true , address=64.49.13.27:4964 , piece_id=UOBMO3UGR2SVYEXRSW2SQMHSDJV47OXOZRZJHCOHVWNXMIIDSA2A, seg_pos={0 0}, upload=514680]: Long tail cancellation due to optimal threshold
Mar 25 14:21:03.950899119 (459h21m1.547670144s) [noise=true , address=145.79.227.44:4201 , piece_id=SHQIEKFZSOKO7SUBGDC7GSJQNL47IEWENP5KN5ZN5ZOS4K3R7Y7A, seg_pos={0 0}, upload=514680]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bPDvhDfu8Z58LxZcicg1vAiiu1sZYGpwQnvuhk4t5m6yR87Fi): protocol: expected piece hash; context canceled; EOF
Mar 25 14:21:03.950907089 (459h21m1.547678114s) [noise=true , address=145.79.227.44:4201 , piece_id=SHQIEKFZSOKO7SUBGDC7GSJQNL47IEWENP5KN5ZN5ZOS4K3R7Y7A, seg_pos={0 0}, upload=514680]: Long tail cancellation due to optimal threshold
Mar 25 14:21:03.951133312 (459h21m1.547904347s) [noise=true , address=64.49.13.138:4487 , piece_id=NMZRB7X7MX2KJRDEUBROG3F6ZPQOGYJLTJ5WHBBFOUQCKXIHIZKQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12taemzrausuc58d8Dyo8xaR7MqDf72f37rEheF32sUfy8Z8GZP): protocol: expected piece hash; context canceled; EOF
Mar 25 14:21:03.951146101 (459h21m1.547917132s) [noise=true , address=64.49.13.138:4487 , piece_id=NMZRB7X7MX2KJRDEUBROG3F6ZPQOGYJLTJ5WHBBFOUQCKXIHIZKQ, seg_pos={0 0}, upload=514680]: Long tail cancellation due to optimal threshold
Mar 25 14:21:03.951175027 (459h21m1.547946039s) [seg_pos={0 0}, upload=514680]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Mar 25 14:21:03.951282270 (459h21m1.548053293s) [upload=514680]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Mar 25 14:21:03.951290421 (459h21m1.548061457s) [upload=514680]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Mar 25 14:21:04.013599996 (459h21m1.610371022s) [upload=514680]: Done uploading segments...
Mar 25 14:21:04.013621043 (459h21m1.610392066s) [upload=514680]: Upload finished. err: <nil>
Duration: 1.385871378s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................X.......................................] duration:
1.399779681s trace:
https://jaeger.dev.storj.io/trace/62357518aac8ca5f
FAIL (2026-03-25 14:56:44 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 1.399815469s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [............X...............................................] duration:
33.388899ms target:
https://us1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/21096334d2fd8925
FAIL (2026-03-25 14:56:39 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.4.32.59:443: i/o timeout
Duration: 33.416031ms
core
satellite
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............X..............................................] duration:
1.36390228s trace:
https://jaeger.dev.storj.io/trace/1d1f0bc4e4438495
FAIL (2026-03-25 14:56:38 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
upload=514680]: Uploading piece...
Mar 25 14:21:03.327216643 (459h21m0.923987655s) [noise=true , address=45.91.116.12:4254 , piece_id=UIQ3FWZSZ52RJYWFVWNH3OGNQPZSDYEZEDX35K3R7SDKRIW3WOAQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.327580267 (459h21m0.924351301s) [noise=true , address=193.149.164.70:4120 , piece_id=HMFOJCRQTG3PAQDIZERNEYWKFFBUIRKRXP56OQUQWH6R2YX6NYLQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.327966258 (459h21m0.924737281s) [noise=true , address=77.90.31.16:4739 , piece_id=LEV3TZ2XFAYLQHULRJOG3DDBFMP3XRWX4EU2CQ2YOFICYXLIR3SA, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.328304150 (459h21m0.925075204s) [noise=true , address=77.90.31.14:4688 , piece_id=XDS6GE2YZK3LWUTBOMHLWTHM4ASCQJI25WTENUU75TJGFULYPOCQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.328528343 (459h21m0.925299366s) [noise=true , address=64.49.13.138:4487 , piece_id=NMZRB7X7MX2KJRDEUBROG3F6ZPQOGYJLTJ5WHBBFOUQCKXIHIZKQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.328832656 (459h21m0.925603704s) [noise=true , address=145.79.227.44:4201 , piece_id=SHQIEKFZSOKO7SUBGDC7GSJQNL47IEWENP5KN5ZN5ZOS4K3R7Y7A, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.328990765 (459h21m0.925761788s) [noise=true , address=77.90.31.17:4791 , piece_id=L5H6QECJGEQPTPRB6INQWHZPMK7ZXXDYYFFG5ANE37HAE33OTVXQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.329286413 (459h21m0.926057443s) [noise=true , address=64.49.13.27:4964 , piece_id=UOBMO3UGR2SVYEXRSW2SQMHSDJV47OXOZRZJHCOHVWNXMIIDSA2A, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.329346877 (459h21m0.9261179s) [noise=true , address=193.149.164.66:4984 , piece_id=CGM44RIHZGETSCKRZQOFL3VYIJTSB57RQDXTG3OPX3F3HJNQDYZA, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.329706811 (459h21m0.926477834s) [noise=true , address=45.91.116.7:4023 , piece_id=GYWVVXKLOAZSDJIS64INTFQIPQZCLBH7YPLTWY7QJSB766A3QTGQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.329728635 (459h21m0.926499661s) [noise=true , address=64.49.13.140:4509 , piece_id=7HN3FXTKC4RRUJZ34AHFDJHMUBIOWIQVFST626CR7X2ZYQS3CFCQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.330142941 (459h21m0.926913971s) [noise=true , address=145.79.227.34:4916 , piece_id=645H2KOPV4YB2HTP3XZF2RF5ZICIR34QNYRNRKX5KHH2VANVVZOQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.330171985 (459h21m0.926943015s) [noise=true , address=193.149.164.71:4296 , piece_id=RXZXV3FCVDEFHBJQVGIWEKS57EJ3IKAJFTDGMOEZHRRBHV3NRDSA, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.330543280 (459h21m0.927314303s) [noise=true , address=193.149.164.67:4030 , piece_id=PBIDHXYQ22HHK3HBJLFPC5X4L74Q332LNNEDYPFDV6SHWCQKRCNQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.330675401 (459h21m0.927446433s) [noise=true , address=64.49.13.26:4955 , piece_id=VIN3RVGRSUUHYCHPXCDNVSPRLFSCH3JXIDERBV7NIM4K33ZUBUGQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.330936609 (459h21m0.927707633s) [noise=true , address=193.149.164.68:4050 , piece_id=F4AQMCLJL6ML2LQKWUIKJAN6C2Y3M7UVYXRCRVRON5BC6CLTEK6Q, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.331355022 (459h21m0.928126051s) [noise=true , address=77.90.31.11:4567 , piece_id=ZSITPEN2OCYCHLSV3PDXCAXBAHEAQMS3YFCCOOE2SISJMUFRPHOA, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.402821184 (459h21m0.999592291s) [upload=514680]: Next returned nil segment
Mar 25 14:21:03.402848744 (459h21m0.99961978s) [upload=514680]: Waiting for error group managing segments...
Mar 25 14:21:03.408851473 (459h21m1.005622504s) [noise=true , address=77.90.31.16:4739 , piece_id=LEV3TZ2XFAYLQHULRJOG3DDBFMP3XRWX4EU2CQ2YOFICYXLIR3SA, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.409008646 (459h21m1.005779679s) [noise=true , address=145.79.227.34:4916 , piece_id=645H2KOPV4YB2HTP3XZF2RF5ZICIR34QNYRNRKX5KHH2VANVVZOQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.418584053 (459h21m1.015355084s) [noise=true , address=77.90.31.17:4791 , piece_id=L5H6QECJGEQPTPRB6INQWHZPMK7ZXXDYYFFG5ANE37HAE33OTVXQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.446927214 (459h21m1.043698248s) [noise=true , address=45.91.116.7:4023 , piece_id=GYWVVXKLOAZSDJIS64INTFQIPQZCLBH7YPLTWY7QJSB766A3QTGQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.448923805 (459h21m1.045694815s) [noise=true , address=77.90.31.14:4688 , piece_id=XDS6GE2YZK3LWUTBOMHLWTHM4ASCQJI25WTENUU75TJGFULYPOCQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.458340727 (459h21m1.055111751s) [noise=true , address=45.91.116.12:4254 , piece_id=UIQ3FWZSZ52RJYWFVWNH3OGNQPZSDYEZEDX35K3R7SDKRIW3WOAQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.468833670 (459h21m1.065604702s) [noise=true , address=77.90.31.11:4567 , piece_id=ZSITPEN2OCYCHLSV3PDXCAXBAHEAQMS3YFCCOOE2SISJMUFRPHOA, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.925971986 (459h21m1.522743012s) [noise=true , address=193.149.164.66:4984 , piece_id=CGM44RIHZGETSCKRZQOFL3VYIJTSB57RQDXTG3OPX3F3HJNQDYZA, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.928271525 (459h21m1.525042552s) [noise=true , address=64.49.13.140:4509 , piece_id=7HN3FXTKC4RRUJZ34AHFDJHMUBIOWIQVFST626CR7X2ZYQS3CFCQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.929087375 (459h21m1.525858388s) [noise=true , address=193.149.164.70:4120 , piece_id=HMFOJCRQTG3PAQDIZERNEYWKFFBUIRKRXP56OQUQWH6R2YX6NYLQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.929851130 (459h21m1.526622145s) [noise=true , address=193.149.164.71:4296 , piece_id=RXZXV3FCVDEFHBJQVGIWEKS57EJ3IKAJFTDGMOEZHRRBHV3NRDSA, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.935607616 (459h21m1.532378643s) [noise=true , address=193.149.164.69:4085 , piece_id=KGCVBEK344OW573PN2Y55JA7AC3ZTQD36G3T6TUHF7ZMZ4WN7N5A, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.938874518 (459h21m1.535645546s) [noise=true , address=193.149.164.67:4030 , piece_id=PBIDHXYQ22HHK3HBJLFPC5X4L74Q332LNNEDYPFDV6SHWCQKRCNQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.950243460 (459h21m1.547014493s) [noise=true , address=64.49.13.26:4955 , piece_id=VIN3RVGRSUUHYCHPXCDNVSPRLFSCH3JXIDERBV7NIM4K33ZUBUGQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.950277580 (459h21m1.54704861s) [seg_pos={0 0}, upload=514680]: Satisfied cohort requirements after 14 pieces
Mar 25 14:21:03.950707500 (459h21m1.547478532s) [noise=true , address=193.149.164.68:4050 , piece_id=F4AQMCLJL6ML2LQKWUIKJAN6C2Y3M7UVYXRCRVRON5BC6CLTEK6Q, seg_pos={0 0}, upload=514680]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1WPfYrCJrhPTqZ2qgBvXFCUodKz6QeXCnkKgToFs5muNRSSLtK): protocol: expected piece hash; context canceled; EOF
Mar 25 14:21:03.950721648 (459h21m1.547492674s) [noise=true , address=193.149.164.68:4050 , piece_id=F4AQMCLJL6ML2LQKWUIKJAN6C2Y3M7UVYXRCRVRON5BC6CLTEK6Q, seg_pos={0 0}, upload=514680]: Long tail cancellation due to optimal threshold
Mar 25 14:21:03.950793002 (459h21m1.547564034s) [noise=true , address=64.49.13.27:4964 , piece_id=UOBMO3UGR2SVYEXRSW2SQMHSDJV47OXOZRZJHCOHVWNXMIIDSA2A, seg_pos={0 0}, upload=514680]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RcgfaeKP3XpZGuSB9GMURCCsjjc79WeZasUL2RRiQVDwxbnAk): protocol: expected piece hash; context canceled; EOF
Mar 25 14:21:03.950804838 (459h21m1.547575864s) [noise=true , address=64.49.13.27:4964 , piece_id=UOBMO3UGR2SVYEXRSW2SQMHSDJV47OXOZRZJHCOHVWNXMIIDSA2A, seg_pos={0 0}, upload=514680]: Long tail cancellation due to optimal threshold
Mar 25 14:21:03.950899119 (459h21m1.547670144s) [noise=true , address=145.79.227.44:4201 , piece_id=SHQIEKFZSOKO7SUBGDC7GSJQNL47IEWENP5KN5ZN5ZOS4K3R7Y7A, seg_pos={0 0}, upload=514680]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bPDvhDfu8Z58LxZcicg1vAiiu1sZYGpwQnvuhk4t5m6yR87Fi): protocol: expected piece hash; context canceled; EOF
Mar 25 14:21:03.950907089 (459h21m1.547678114s) [noise=true , address=145.79.227.44:4201 , piece_id=SHQIEKFZSOKO7SUBGDC7GSJQNL47IEWENP5KN5ZN5ZOS4K3R7Y7A, seg_pos={0 0}, upload=514680]: Long tail cancellation due to optimal threshold
Mar 25 14:21:03.951133312 (459h21m1.547904347s) [noise=true , address=64.49.13.138:4487 , piece_id=NMZRB7X7MX2KJRDEUBROG3F6ZPQOGYJLTJ5WHBBFOUQCKXIHIZKQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12taemzrausuc58d8Dyo8xaR7MqDf72f37rEheF32sUfy8Z8GZP): protocol: expected piece hash; context canceled; EOF
Mar 25 14:21:03.951146101 (459h21m1.547917132s) [noise=true , address=64.49.13.138:4487 , piece_id=NMZRB7X7MX2KJRDEUBROG3F6ZPQOGYJLTJ5WHBBFOUQCKXIHIZKQ, seg_pos={0 0}, upload=514680]: Long tail cancellation due to optimal threshold
Mar 25 14:21:03.951175027 (459h21m1.547946039s) [seg_pos={0 0}, upload=514680]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Mar 25 14:21:03.951282270 (459h21m1.548053293s) [upload=514680]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Mar 25 14:21:03.951290421 (459h21m1.548061457s) [upload=514680]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Mar 25 14:21:04.013599996 (459h21m1.610371022s) [upload=514680]: Done uploading segments...
Mar 25 14:21:04.013621043 (459h21m1.610392066s) [upload=514680]: Upload finished. err: <nil>
Duration: 1.363931502s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............X.............................................] duration:
1.289178883s trace:
https://jaeger.dev.storj.io/trace/1d1f0bc4e4438495
FAIL (2026-03-25 14:56:15 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
upload=514680]: Uploading piece...
Mar 25 14:21:03.327216643 (459h21m0.923987655s) [noise=true , address=45.91.116.12:4254 , piece_id=UIQ3FWZSZ52RJYWFVWNH3OGNQPZSDYEZEDX35K3R7SDKRIW3WOAQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.327580267 (459h21m0.924351301s) [noise=true , address=193.149.164.70:4120 , piece_id=HMFOJCRQTG3PAQDIZERNEYWKFFBUIRKRXP56OQUQWH6R2YX6NYLQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.327966258 (459h21m0.924737281s) [noise=true , address=77.90.31.16:4739 , piece_id=LEV3TZ2XFAYLQHULRJOG3DDBFMP3XRWX4EU2CQ2YOFICYXLIR3SA, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.328304150 (459h21m0.925075204s) [noise=true , address=77.90.31.14:4688 , piece_id=XDS6GE2YZK3LWUTBOMHLWTHM4ASCQJI25WTENUU75TJGFULYPOCQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.328528343 (459h21m0.925299366s) [noise=true , address=64.49.13.138:4487 , piece_id=NMZRB7X7MX2KJRDEUBROG3F6ZPQOGYJLTJ5WHBBFOUQCKXIHIZKQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.328832656 (459h21m0.925603704s) [noise=true , address=145.79.227.44:4201 , piece_id=SHQIEKFZSOKO7SUBGDC7GSJQNL47IEWENP5KN5ZN5ZOS4K3R7Y7A, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.328990765 (459h21m0.925761788s) [noise=true , address=77.90.31.17:4791 , piece_id=L5H6QECJGEQPTPRB6INQWHZPMK7ZXXDYYFFG5ANE37HAE33OTVXQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.329286413 (459h21m0.926057443s) [noise=true , address=64.49.13.27:4964 , piece_id=UOBMO3UGR2SVYEXRSW2SQMHSDJV47OXOZRZJHCOHVWNXMIIDSA2A, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.329346877 (459h21m0.9261179s) [noise=true , address=193.149.164.66:4984 , piece_id=CGM44RIHZGETSCKRZQOFL3VYIJTSB57RQDXTG3OPX3F3HJNQDYZA, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.329706811 (459h21m0.926477834s) [noise=true , address=45.91.116.7:4023 , piece_id=GYWVVXKLOAZSDJIS64INTFQIPQZCLBH7YPLTWY7QJSB766A3QTGQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.329728635 (459h21m0.926499661s) [noise=true , address=64.49.13.140:4509 , piece_id=7HN3FXTKC4RRUJZ34AHFDJHMUBIOWIQVFST626CR7X2ZYQS3CFCQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.330142941 (459h21m0.926913971s) [noise=true , address=145.79.227.34:4916 , piece_id=645H2KOPV4YB2HTP3XZF2RF5ZICIR34QNYRNRKX5KHH2VANVVZOQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.330171985 (459h21m0.926943015s) [noise=true , address=193.149.164.71:4296 , piece_id=RXZXV3FCVDEFHBJQVGIWEKS57EJ3IKAJFTDGMOEZHRRBHV3NRDSA, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.330543280 (459h21m0.927314303s) [noise=true , address=193.149.164.67:4030 , piece_id=PBIDHXYQ22HHK3HBJLFPC5X4L74Q332LNNEDYPFDV6SHWCQKRCNQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.330675401 (459h21m0.927446433s) [noise=true , address=64.49.13.26:4955 , piece_id=VIN3RVGRSUUHYCHPXCDNVSPRLFSCH3JXIDERBV7NIM4K33ZUBUGQ, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.330936609 (459h21m0.927707633s) [noise=true , address=193.149.164.68:4050 , piece_id=F4AQMCLJL6ML2LQKWUIKJAN6C2Y3M7UVYXRCRVRON5BC6CLTEK6Q, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.331355022 (459h21m0.928126051s) [noise=true , address=77.90.31.11:4567 , piece_id=ZSITPEN2OCYCHLSV3PDXCAXBAHEAQMS3YFCCOOE2SISJMUFRPHOA, seg_pos={0 0}, upload=514680]: Uploading piece...
Mar 25 14:21:03.402821184 (459h21m0.999592291s) [upload=514680]: Next returned nil segment
Mar 25 14:21:03.402848744 (459h21m0.99961978s) [upload=514680]: Waiting for error group managing segments...
Mar 25 14:21:03.408851473 (459h21m1.005622504s) [noise=true , address=77.90.31.16:4739 , piece_id=LEV3TZ2XFAYLQHULRJOG3DDBFMP3XRWX4EU2CQ2YOFICYXLIR3SA, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.409008646 (459h21m1.005779679s) [noise=true , address=145.79.227.34:4916 , piece_id=645H2KOPV4YB2HTP3XZF2RF5ZICIR34QNYRNRKX5KHH2VANVVZOQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.418584053 (459h21m1.015355084s) [noise=true , address=77.90.31.17:4791 , piece_id=L5H6QECJGEQPTPRB6INQWHZPMK7ZXXDYYFFG5ANE37HAE33OTVXQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.446927214 (459h21m1.043698248s) [noise=true , address=45.91.116.7:4023 , piece_id=GYWVVXKLOAZSDJIS64INTFQIPQZCLBH7YPLTWY7QJSB766A3QTGQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.448923805 (459h21m1.045694815s) [noise=true , address=77.90.31.14:4688 , piece_id=XDS6GE2YZK3LWUTBOMHLWTHM4ASCQJI25WTENUU75TJGFULYPOCQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.458340727 (459h21m1.055111751s) [noise=true , address=45.91.116.12:4254 , piece_id=UIQ3FWZSZ52RJYWFVWNH3OGNQPZSDYEZEDX35K3R7SDKRIW3WOAQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.468833670 (459h21m1.065604702s) [noise=true , address=77.90.31.11:4567 , piece_id=ZSITPEN2OCYCHLSV3PDXCAXBAHEAQMS3YFCCOOE2SISJMUFRPHOA, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.925971986 (459h21m1.522743012s) [noise=true , address=193.149.164.66:4984 , piece_id=CGM44RIHZGETSCKRZQOFL3VYIJTSB57RQDXTG3OPX3F3HJNQDYZA, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.928271525 (459h21m1.525042552s) [noise=true , address=64.49.13.140:4509 , piece_id=7HN3FXTKC4RRUJZ34AHFDJHMUBIOWIQVFST626CR7X2ZYQS3CFCQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.929087375 (459h21m1.525858388s) [noise=true , address=193.149.164.70:4120 , piece_id=HMFOJCRQTG3PAQDIZERNEYWKFFBUIRKRXP56OQUQWH6R2YX6NYLQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.929851130 (459h21m1.526622145s) [noise=true , address=193.149.164.71:4296 , piece_id=RXZXV3FCVDEFHBJQVGIWEKS57EJ3IKAJFTDGMOEZHRRBHV3NRDSA, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.935607616 (459h21m1.532378643s) [noise=true , address=193.149.164.69:4085 , piece_id=KGCVBEK344OW573PN2Y55JA7AC3ZTQD36G3T6TUHF7ZMZ4WN7N5A, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.938874518 (459h21m1.535645546s) [noise=true , address=193.149.164.67:4030 , piece_id=PBIDHXYQ22HHK3HBJLFPC5X4L74Q332LNNEDYPFDV6SHWCQKRCNQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.950243460 (459h21m1.547014493s) [noise=true , address=64.49.13.26:4955 , piece_id=VIN3RVGRSUUHYCHPXCDNVSPRLFSCH3JXIDERBV7NIM4K33ZUBUGQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: <nil>
Mar 25 14:21:03.950277580 (459h21m1.54704861s) [seg_pos={0 0}, upload=514680]: Satisfied cohort requirements after 14 pieces
Mar 25 14:21:03.950707500 (459h21m1.547478532s) [noise=true , address=193.149.164.68:4050 , piece_id=F4AQMCLJL6ML2LQKWUIKJAN6C2Y3M7UVYXRCRVRON5BC6CLTEK6Q, seg_pos={0 0}, upload=514680]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1WPfYrCJrhPTqZ2qgBvXFCUodKz6QeXCnkKgToFs5muNRSSLtK): protocol: expected piece hash; context canceled; EOF
Mar 25 14:21:03.950721648 (459h21m1.547492674s) [noise=true , address=193.149.164.68:4050 , piece_id=F4AQMCLJL6ML2LQKWUIKJAN6C2Y3M7UVYXRCRVRON5BC6CLTEK6Q, seg_pos={0 0}, upload=514680]: Long tail cancellation due to optimal threshold
Mar 25 14:21:03.950793002 (459h21m1.547564034s) [noise=true , address=64.49.13.27:4964 , piece_id=UOBMO3UGR2SVYEXRSW2SQMHSDJV47OXOZRZJHCOHVWNXMIIDSA2A, seg_pos={0 0}, upload=514680]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RcgfaeKP3XpZGuSB9GMURCCsjjc79WeZasUL2RRiQVDwxbnAk): protocol: expected piece hash; context canceled; EOF
Mar 25 14:21:03.950804838 (459h21m1.547575864s) [noise=true , address=64.49.13.27:4964 , piece_id=UOBMO3UGR2SVYEXRSW2SQMHSDJV47OXOZRZJHCOHVWNXMIIDSA2A, seg_pos={0 0}, upload=514680]: Long tail cancellation due to optimal threshold
Mar 25 14:21:03.950899119 (459h21m1.547670144s) [noise=true , address=145.79.227.44:4201 , piece_id=SHQIEKFZSOKO7SUBGDC7GSJQNL47IEWENP5KN5ZN5ZOS4K3R7Y7A, seg_pos={0 0}, upload=514680]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bPDvhDfu8Z58LxZcicg1vAiiu1sZYGpwQnvuhk4t5m6yR87Fi): protocol: expected piece hash; context canceled; EOF
Mar 25 14:21:03.950907089 (459h21m1.547678114s) [noise=true , address=145.79.227.44:4201 , piece_id=SHQIEKFZSOKO7SUBGDC7GSJQNL47IEWENP5KN5ZN5ZOS4K3R7Y7A, seg_pos={0 0}, upload=514680]: Long tail cancellation due to optimal threshold
Mar 25 14:21:03.951133312 (459h21m1.547904347s) [noise=true , address=64.49.13.138:4487 , piece_id=NMZRB7X7MX2KJRDEUBROG3F6ZPQOGYJLTJ5WHBBFOUQCKXIHIZKQ, seg_pos={0 0}, upload=514680]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12taemzrausuc58d8Dyo8xaR7MqDf72f37rEheF32sUfy8Z8GZP): protocol: expected piece hash; context canceled; EOF
Mar 25 14:21:03.951146101 (459h21m1.547917132s) [noise=true , address=64.49.13.138:4487 , piece_id=NMZRB7X7MX2KJRDEUBROG3F6ZPQOGYJLTJ5WHBBFOUQCKXIHIZKQ, seg_pos={0 0}, upload=514680]: Long tail cancellation due to optimal threshold
Mar 25 14:21:03.951175027 (459h21m1.547946039s) [seg_pos={0 0}, upload=514680]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Mar 25 14:21:03.951282270 (459h21m1.548053293s) [upload=514680]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Mar 25 14:21:03.951290421 (459h21m1.548061457s) [upload=514680]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Mar 25 14:21:04.013599996 (459h21m1.610371022s) [upload=514680]: Done uploading segments...
Mar 25 14:21:04.013621043 (459h21m1.610392066s) [upload=514680]: Upload finished. err: <nil>
Duration: 1.289210041s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [.............X..............................................] duration:
36.18654ms target:
https://us1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/21096334d2fd8925
FAIL (2026-03-25 14:56:15 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.4.32.59:443: i/o timeout
Duration: 36.216864ms
core
satellite
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................X......................................] duration:
1.667391193s trace:
https://jaeger.dev.storj.io/trace/62357518aac8ca5f
FAIL (2026-03-25 14:56:01 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 1.66752529s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................X.....................................] duration:
1.308928456s trace:
https://jaeger.dev.storj.io/trace/62357518aac8ca5f
FAIL (2026-03-25 14:55:29 UTC)
orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS======
Duration: 1.308961558s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|