Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
2 out of 60 is failed (threshold=3) [.....................X......X...............................] duration:
172.499243ms target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318
FAIL (2025-07-14 22:55:33 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318": dial tcp: lookup uptime-checks.gateway.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:40390->127.0.0.1:53: i/o timeout
Duration: 172.526138ms
edge
gateway
vhost
bgp
eu1
|
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777"} failure (antiblip window 3/60): (authservice.eu1)antiblip:
2 out of 60 is failed (threshold=3) [..................X..X......................................] duration:
398.797187ms
FAIL (2025-07-14 22:55:32 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp: lookup auth.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:50971->127.0.0.1:53: i/o timeout
Duration: 398.824746ms
edge
authservice
http
bgp
eu1
|
checkers.HTTP{"GET" "http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt", ExpectedStatus: 308, ExpectedRedirect: "https://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt"} failure (antiblip window 3/60): (http.us_select_1.linksharing)antiblip:
5 out of 60 is failed (threshold=3) [..............X............X........X...XX..................] duration:
15.356714ms target:
http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt
FAIL (2025-07-14 22:55:28 UTC)
orbiter warning: Get "http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt": dial tcp: lookup link.us-select-1.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:59356->127.0.0.1:53: i/o timeout
Duration: 15.396447ms
edge
linksharing
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [..............................XX............................] duration:
461.741962ms
FAIL (2025-07-14 22:55:23 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
22:32:36.255563160 (517h21m3.181857848s) [noise=true , address=45.91.116.5:4297 , piece_id=4DXCPSF4YTD77FXCVOUGH4GWVJBS3JM3WYNERF6DS4UNEBOU2QWQ, seg_pos={0 0}, upload=414047]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF
Jul 14 22:32:36.255577193 (517h21m3.181871868s) [seg_pos={0 0}, upload=414047]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 14 22:32:36.255645270 (517h21m3.181939946s) [upload=414047]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 14 22:32:36.255650241 (517h21m3.181944931s) [upload=414047]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 14 22:32:36.314863469 (517h21m3.241158149s) [upload=414047]: Done uploading segments...
Jul 14 22:32:36.314882474 (517h21m3.241177151s) [upload=414047]: Upload finished. err: <nil>
Jul 14 22:33:11.378492041 (517h21m38.304786743s) [upload=414053]: Starting upload
Jul 14 22:33:11.378577342 (517h21m38.304872032s) [upload=414053]: Done starting upload
Jul 14 22:33:11.378744513 (517h21m38.305039193s) [upload=414053]: Uploading segments...
Jul 14 22:33:11.378953197 (517h21m38.305247877s) [upload=414053]: Got next segment. Inline: false
Jul 14 22:33:11.378994874 (517h21m38.30528956s) [upload=414053]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 14 22:33:11.379000311 (517h21m38.305294992s) [upload=414053]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 14 22:33:11.409521121 (517h21m38.335815814s) [seg_pos={0 0}, upload=414053]: Begin upload segment...
Jul 14 22:33:11.409580940 (517h21m38.335875637s) [seg_pos={0 0}, upload=414053]: Done begin upload segment.
Jul 14 22:33:11.409628148 (517h21m38.335922844s) [noise=true , address=104.36.82.16:4294 , piece_id=QLMMGAWGTTQZXFKV4XVD3YDR2T355TB556XQKWGSLHYTIAZIVDDQ, seg_pos={0 0}, upload=414053]: Uploading piece...
Jul 14 22:33:11.409651610 (517h21m38.335946308s) [noise=true , address=45.91.116.5:4299 , piece_id=5F57Y5UYILBHBF3DV7M5POSK6RFAL3Z7CFA24WMKBEUQAZAL7ODQ, seg_pos={0 0}, upload=414053]: Uploading piece...
Jul 14 22:33:11.410094293 (517h21m38.336388984s) [noise=true , address=45.91.116.9:4263 , piece_id=IO6KT3CIHLSEYQEMZ4DAX4JT4MP4OYARWASXXGV35Q7UHMLIIBOA, seg_pos={0 0}, upload=414053]: Uploading piece...
Jul 14 22:33:11.410493120 (517h21m38.33678781s) [noise=true , address=77.90.31.18:4806 , piece_id=WH2NYTMISPD6II4BSWZDNNPM3H4G2JXZMUCCYKG4GM5LGFAQ2WPQ, seg_pos={0 0}, upload=414053]: Uploading piece...
Jul 14 22:33:11.410954124 (517h21m38.337248825s) [noise=true , address=104.36.82.15:4257 , piece_id=WJRQPXCD4ZS73SBUUPSKGG3VSDLYTDQ7ZQE6B4YW2OIPLAHGIDJA, seg_pos={0 0}, upload=414053]: Uploading piece...
Jul 14 22:33:11.411449799 (517h21m38.337744496s) [noise=true , address=45.91.116.4:4939 , piece_id=ISKKXMSPSMOSP4FWOUKSZ6IMIJFPO33IJDDZ73VNPNSDZ7KQ7SEA, seg_pos={0 0}, upload=414053]: Uploading piece...
Jul 14 22:33:11.411485810 (517h21m38.337780506s) [noise=true , address=104.36.82.11:4113 , piece_id=AK7SBE2BGLIXBKRJGGFRVTRXZGKILEP364KUPGDUS5D7Y74VHJLA, seg_pos={0 0}, upload=414053]: Uploading piece...
Jul 14 22:33:11.411850807 (517h21m38.338145504s) [noise=true , address=45.91.116.12:4225 , piece_id=MDOSVRR4VTFU6A4NIU3HZFTVMJLYTQDKPI2OTVR5SBQ6WGMWBLWA, seg_pos={0 0}, upload=414053]: Uploading piece...
Jul 14 22:33:11.411890633 (517h21m38.338185327s) [noise=true , address=77.90.31.10:4518 , piece_id=KRK47EJXTN4I7TWP7KXWPNYVDUBPOUQGNOG5VN6HDBJ36WE26RHA, seg_pos={0 0}, upload=414053]: Uploading piece...
Jul 14 22:33:11.412220793 (517h21m38.338515486s) [noise=true , address=104.36.82.8:4972 , piece_id=E46WB6ZEOOK23SMTLGJQPGGKDUN3FKBXHSKOY7QLLY24YPPIQVLQ, seg_pos={0 0}, upload=414053]: Uploading piece...
Jul 14 22:33:11.412279103 (517h21m38.338573797s) [noise=true , address=104.36.82.16:4295 , piece_id=NA6OQP5GZJKGH7OMU7OJLTME4OC5LLJUPE3AWBRISBEP7TFSIZIQ, seg_pos={0 0}, upload=414053]: Uploading piece...
Jul 14 22:33:11.412647456 (517h21m38.338942154s) [noise=true , address=104.36.82.4:4804 , piece_id=XO52GRQZ3KWIAQYBDMJTT7D6ADQK35EP232Z4JERAMI6HBEVIXNQ, seg_pos={0 0}, upload=414053]: Uploading piece...
Jul 14 22:33:11.413168375 (517h21m38.33946307s) [noise=true , address=45.91.116.11:4118 , piece_id=VYLKVTNKUCCC6QYXJKGQVRWAJS7X4PEMUDJKJIIGYKHUQBADXQIA, seg_pos={0 0}, upload=414053]: Uploading piece...
Jul 14 22:33:11.413438644 (517h21m38.33973334s) [noise=true , address=45.91.116.14:4191 , piece_id=Q7A6WPWJ3F6D7ZBT27AARER3R3MDQLTUXU3GVBKMUJCUTIE2P3KA, seg_pos={0 0}, upload=414053]: Uploading piece...
Jul 14 22:33:11.413572133 (517h21m38.339866829s) [noise=true , address=77.90.31.17:4772 , piece_id=2JZOVZDHXZ4J5DZYX6EWQ4WSJ6NTMSVQN3WF3VD6AFFHC3J7AFJA, seg_pos={0 0}, upload=414053]: Uploading piece...
Jul 14 22:33:11.413947381 (517h21m38.340242075s) [noise=true , address=104.36.82.13:4186 , piece_id=P7HJGYDOOI4GVIB6AQ3AHCLLXKHSWSR3VDWBIMHTLEVOYRTEV7FQ, seg_pos={0 0}, upload=414053]: Uploading piece...
Jul 14 22:33:11.524370644 (517h21m38.450665324s) [upload=414053]: Next returned nil segment
Jul 14 22:33:11.524388534 (517h21m38.450683224s) [upload=414053]: Waiting for error group managing segments...
Jul 14 22:33:11.529388413 (517h21m38.455683093s) [noise=true , address=77.90.31.10:4518 , piece_id=KRK47EJXTN4I7TWP7KXWPNYVDUBPOUQGNOG5VN6HDBJ36WE26RHA, seg_pos={0 0}, upload=414053]: Done uploading piece. err: <nil>
Jul 14 22:33:11.529842618 (517h21m38.456137297s) [noise=true , address=77.90.31.18:4806 , piece_id=WH2NYTMISPD6II4BSWZDNNPM3H4G2JXZMUCCYKG4GM5LGFAQ2WPQ, seg_pos={0 0}, upload=414053]: Done uploading piece. err: <nil>
Jul 14 22:33:11.530343771 (517h21m38.456638451s) [noise=true , address=45.91.116.5:4299 , piece_id=5F57Y5UYILBHBF3DV7M5POSK6RFAL3Z7CFA24WMKBEUQAZAL7ODQ, seg_pos={0 0}, upload=414053]: Done uploading piece. err: <nil>
Jul 14 22:33:11.530866496 (517h21m38.457161176s) [noise=true , address=45.91.116.12:4225 , piece_id=MDOSVRR4VTFU6A4NIU3HZFTVMJLYTQDKPI2OTVR5SBQ6WGMWBLWA, seg_pos={0 0}, upload=414053]: Done uploading piece. err: <nil>
Jul 14 22:33:11.530939893 (517h21m38.45723458s) [noise=true , address=45.91.116.4:4939 , piece_id=ISKKXMSPSMOSP4FWOUKSZ6IMIJFPO33IJDDZ73VNPNSDZ7KQ7SEA, seg_pos={0 0}, upload=414053]: Done uploading piece. err: <nil>
Jul 14 22:33:11.531326057 (517h21m38.457620733s) [noise=true , address=45.91.116.14:4191 , piece_id=Q7A6WPWJ3F6D7ZBT27AARER3R3MDQLTUXU3GVBKMUJCUTIE2P3KA, seg_pos={0 0}, upload=414053]: Done uploading piece. err: <nil>
Jul 14 22:33:11.531369084 (517h21m38.457663773s) [noise=true , address=45.91.116.11:4118 , piece_id=VYLKVTNKUCCC6QYXJKGQVRWAJS7X4PEMUDJKJIIGYKHUQBADXQIA, seg_pos={0 0}, upload=414053]: Done uploading piece. err: <nil>
Jul 14 22:33:11.531856839 (517h21m38.458151512s) [noise=true , address=45.91.116.9:4263 , piece_id=IO6KT3CIHLSEYQEMZ4DAX4JT4MP4OYARWASXXGV35Q7UHMLIIBOA, seg_pos={0 0}, upload=414053]: Done uploading piece. err: <nil>
Jul 14 22:33:11.534154262 (517h21m38.460448936s) [noise=true , address=104.36.82.13:4186 , piece_id=P7HJGYDOOI4GVIB6AQ3AHCLLXKHSWSR3VDWBIMHTLEVOYRTEV7FQ, seg_pos={0 0}, upload=414053]: Done uploading piece. err: <nil>
Jul 14 22:33:11.534476285 (517h21m38.460770973s) [noise=true , address=104.36.82.16:4295 , piece_id=NA6OQP5GZJKGH7OMU7OJLTME4OC5LLJUPE3AWBRISBEP7TFSIZIQ, seg_pos={0 0}, upload=414053]: Done uploading piece. err: <nil>
Jul 14 22:33:11.536382666 (517h21m38.462677357s) [noise=true , address=104.36.82.8:4972 , piece_id=E46WB6ZEOOK23SMTLGJQPGGKDUN3FKBXHSKOY7QLLY24YPPIQVLQ, seg_pos={0 0}, upload=414053]: Done uploading piece. err: <nil>
Jul 14 22:33:11.536961196 (517h21m38.463255883s) [noise=true , address=104.36.82.15:4257 , piece_id=WJRQPXCD4ZS73SBUUPSKGG3VSDLYTDQ7ZQE6B4YW2OIPLAHGIDJA, seg_pos={0 0}, upload=414053]: Done uploading piece. err: <nil>
Jul 14 22:33:11.537568434 (517h21m38.463863128s) [noise=true , address=104.36.82.16:4294 , piece_id=QLMMGAWGTTQZXFKV4XVD3YDR2T355TB556XQKWGSLHYTIAZIVDDQ, seg_pos={0 0}, upload=414053]: Done uploading piece. err: <nil>
Jul 14 22:33:11.537582802 (517h21m38.463877487s) [seg_pos={0 0}, upload=414053]: Segment reached optimal threshold of 13 pieces.
Jul 14 22:33:11.537702706 (517h21m38.463997382s) [noise=true , address=104.36.82.4:4804 , piece_id=XO52GRQZ3KWIAQYBDMJTT7D6ADQK35EP232Z4JERAMI6HBEVIXNQ, seg_pos={0 0}, upload=414053]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g): protocol: expected piece hash; context canceled; EOF
Jul 14 22:33:11.537835025 (517h21m38.464129712s) [noise=true , address=104.36.82.11:4113 , piece_id=AK7SBE2BGLIXBKRJGGFRVTRXZGKILEP364KUPGDUS5D7Y74VHJLA, seg_pos={0 0}, upload=414053]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV): protocol: expected piece hash; context canceled; EOF
Jul 14 22:33:11.539014185 (517h21m38.465308871s) [noise=true , address=77.90.31.17:4772 , piece_id=2JZOVZDHXZ4J5DZYX6EWQ4WSJ6NTMSVQN3WF3VD6AFFHC3J7AFJA, seg_pos={0 0}, upload=414053]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE): protocol: expected piece hash; context canceled; EOF
Jul 14 22:33:11.539064812 (517h21m38.465359489s) [seg_pos={0 0}, upload=414053]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 14 22:33:11.539134433 (517h21m38.465429119s) [upload=414053]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 14 22:33:11.539139366 (517h21m38.465434043s) [upload=414053]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 14 22:33:11.600764137 (517h21m38.527058814s) [upload=414053]: Done uploading segments...
Jul 14 22:33:11.600783410 (517h21m38.527078087s) [upload=414053]: Upload finished. err: <nil>
Duration: 461.769727ms
core
uplink
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
2 out of 60 is failed (threshold=3) [....................X........X..............................] duration:
117.734035ms target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222
FAIL (2025-07-14 22:55:23 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": dial tcp: lookup uptime-checks.gateway.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:40390->127.0.0.1:53: i/o timeout
Duration: 117.767915ms
edge
gateway
vhost
bgp
us1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................................X.........................] duration:
504.860087ms
FAIL (2025-07-14 22:55:23 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 504.910393ms
edge
gateway
us-select-1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)antiblip:
4 out of 60 is failed (threshold=6) [...................X..............X..X........X.............] duration:
261.514302ms target:
https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt
FAIL (2025-07-14 22:55:22 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": dial tcp: lookup link.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:51321->127.0.0.1:53: i/o timeout
Duration: 261.554147ms
edge
linksharing
bgp
ap1
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
4 out of 60 is failed (threshold=3) [............................XXX.............X...............] duration:
87.473748ms target:
https://us1.storj.io/login
FAIL (2025-07-14 22:55:20 UTC)
Get "https://us1.storj.io/login": dial tcp: lookup us1.storj.io on 127.0.0.1:53: read udp 127.0.0.1:44115->127.0.0.1:53: i/o timeout
Duration: 87.522814ms
core
satellite
us1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [...................X........................................] duration:
61.535873ms target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410
FAIL (2025-07-14 22:55:17 UTC)
orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": dial tcp: lookup uptime-checks-us-select.s3.us-select-1.storjapi.io on 127.0.0.1:53: read udp 127.0.0.1:58761->127.0.0.1:53: i/o timeout
Duration: 61.55663ms
edge
gateway
vhost
us-select-1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)antiblip:
3 out of 60 is failed (threshold=6) [...................X............X.X.........................] duration:
192.9536ms target:
https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt
FAIL (2025-07-14 22:55:17 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": dial tcp: lookup link.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:46744->127.0.0.1:53: i/o timeout
Duration: 193.051368ms
edge
linksharing
bgp
eu1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
2 out of 60 is failed (threshold=3) [..............................XX............................] duration:
254.592237ms
FAIL (2025-07-14 22:55:16 UTC)
orbiter warning: uplink: bucket: metaclient: context deadline exceeded
Duration: 254.619523ms
core
uplink
us1
|
checkers.HTTP{"GET" "https://ap1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.ap1.satellite_login)antiblip:
4 out of 60 is failed (threshold=3) [.......................XX.............XX....................] duration:
704.52311ms target:
https://ap1.storj.io/login
FAIL (2025-07-14 22:55:16 UTC)
orbiter warning: Get "https://ap1.storj.io/login": dial tcp: lookup ap1.storj.io on 127.0.0.1:53: read udp 127.0.0.1:45980->127.0.0.1:53: i/o timeout
Duration: 704.546464ms
core
satellite
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
5 out of 60 is failed (threshold=3) [...................X............XXX.........X...............] duration:
1.759814785s
FAIL (2025-07-14 22:55:12 UTC)
uplink: bucket: metaclient: context deadline exceeded
Duration: 1.759843703s
core
uplink
us1
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
2 out of 60 is failed (threshold=3) [....................X...............X.......................] duration:
4.810600466s
FAIL (2025-07-14 22:55:09 UTC)
orbiter warning: uplink: bucket: metaclient: context deadline exceeded
Duration: 4.810630058s
core
uplink
ap1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
2 out of 60 is failed (threshold=3) [.............................X...X..........................] duration:
433.598267ms
FAIL (2025-07-14 22:55:08 UTC)
orbiter warning: operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/gateway-tests/from_do_prod_ops_orbiter_us_0/1K?x-id=PutObject": dial tcp: lookup s3.us-select-1.storjapi.io on 127.0.0.1:53: read udp 127.0.0.1:59998->127.0.0.1:53: i/o timeout
Duration: 433.629603ms
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)antiblip:
2 out of 60 is failed (threshold=3) [.................................X..................X.......] duration:
902.59809ms
FAIL (2025-07-14 22:55:07 UTC)
orbiter warning: uplink: bucket: metaclient: context deadline exceeded
Duration: 902.626127ms
core
uplink
eu1
|
checkers.HTTP{"GET" "http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt", ExpectedStatus: 308, ExpectedRedirect: "https://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt"} failure (antiblip window 3/60): (http.us_select_1.linksharing)antiblip:
5 out of 60 is failed (threshold=3) [...............X............X........X...XX.................] duration:
27.866126ms target:
http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt
FAIL (2025-07-14 22:55:07 UTC)
orbiter warning: Get "http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt": dial tcp: lookup link.us-select-1.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:59356->127.0.0.1:53: i/o timeout
Duration: 27.891794ms
edge
linksharing
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [.................................XX.........................] duration:
651.640268ms
FAIL (2025-07-14 22:55:04 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: rpc: tcp connector failed: rpc: context deadline exceeded
======LOGS======
RequestItem_ObjectBegin
Jul 14 22:33:20.920268234 (517h21m47.84656293s) [upload=414055]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 14 22:33:20.956049835 (517h21m47.882344525s) [seg_pos={0 0}, upload=414055]: Begin upload segment...
Jul 14 22:33:20.956138063 (517h21m47.882432739s) [seg_pos={0 0}, upload=414055]: Done begin upload segment.
Jul 14 22:33:20.956148696 (517h21m47.88244339s) [noise=true , address=104.36.82.19:4432 , piece_id=ZNUQB26NCFHHETSPPLIX52BSR23AI2VO3J2BJA5O4FPUDRD6ILEA, seg_pos={0 0}, upload=414055]: Uploading piece...
Jul 14 22:33:20.956179183 (517h21m47.882473875s) [noise=true , address=45.91.95.4:4001 , piece_id=C4KRMZGJX4PWVYSTRKXHCRK34AABH2ZLHVXXRKADJIOFOXX45LTA, seg_pos={0 0}, upload=414055]: Uploading piece...
Jul 14 22:33:20.956739778 (517h21m47.883034473s) [noise=true , address=104.36.82.17:4334 , piece_id=6QFH4V2XP5HENEXYSPBLCGWESKAO2M7S243GVWQCSGYYANMDC5BQ, seg_pos={0 0}, upload=414055]: Uploading piece...
Jul 14 22:33:20.957112661 (517h21m47.883407356s) [noise=true , address=45.91.116.14:4215 , piece_id=JB6J36RX7ZLVP4SBF3ZOJSXEZWILQXZOFUQON5WIWMRIAFDZYARA, seg_pos={0 0}, upload=414055]: Uploading piece...
Jul 14 22:33:20.957321762 (517h21m47.883616454s) [noise=true , address=77.90.31.19:4873 , piece_id=ALRBAJG2MS2FX2JQQORO5ZLLXGFQGMBIXLKRTQB3OKIICVAAQTKA, seg_pos={0 0}, upload=414055]: Uploading piece...
Jul 14 22:33:20.957534791 (517h21m47.883829483s) [noise=true , address=104.36.82.51:4074 , piece_id=HOC7FQM2ROFQMQOOQMXPD5HVLON2C6U2UVSS5HMLGRIWJIIO3MUQ, seg_pos={0 0}, upload=414055]: Uploading piece...
Jul 14 22:33:20.957685297 (517h21m47.883979986s) [noise=true , address=45.91.116.10:4098 , piece_id=QAD3V2YB62ZWM7NR5YTZD5YBHIZONG64A27OM6C7WS6R6XE5IAKA, seg_pos={0 0}, upload=414055]: Uploading piece...
Jul 14 22:33:20.957863790 (517h21m47.884158483s) [noise=true , address=77.90.31.18:4810 , piece_id=L2G6QQTX3FVYCW7Z7FMLM6SAA7XZI64BRADRPCOUBLBRVL223KWQ, seg_pos={0 0}, upload=414055]: Uploading piece...
Jul 14 22:33:20.958078206 (517h21m47.884372898s) [noise=true , address=77.90.31.11:4556 , piece_id=4WXTBJSV6F4WCYX3ETHYA7LKJEHEAFNNOD7E5X6GY65ZYUYEAC3A, seg_pos={0 0}, upload=414055]: Uploading piece...
Jul 14 22:33:20.958206699 (517h21m47.884501391s) [noise=true , address=45.91.116.6:4005 , piece_id=KZTESMI5CMWF5U7S6PXTUK5Z4HSVSXXA6CJ2VAVL5SLK2IVJG3DQ, seg_pos={0 0}, upload=414055]: Uploading piece...
Jul 14 22:33:20.958449262 (517h21m47.884743949s) [noise=true , address=104.36.82.10:4101 , piece_id=6I325W4T6GYA7B6TXAEA33PGTNXQSWH5AB6MRRWPJM6O66LXTDAA, seg_pos={0 0}, upload=414055]: Uploading piece...
Jul 14 22:33:20.958598885 (517h21m47.884893577s) [noise=true , address=45.91.95.3:4028 , piece_id=WPAZ53X2MORZF3ZAL42BEZW2HQ2FC7YEBX7YGGFNLOBALDZB7JUQ, seg_pos={0 0}, upload=414055]: Uploading piece...
Jul 14 22:33:20.958933707 (517h21m47.885228398s) [noise=true , address=77.90.31.15:4704 , piece_id=K7MIP54PNDQCEH6FKFEHFDYAYQ2ZWOHHARJ26JJYLD6YKJCRX64A, seg_pos={0 0}, upload=414055]: Uploading piece...
Jul 14 22:33:20.959332655 (517h21m47.885627352s) [noise=true , address=77.90.31.12:4604 , piece_id=ZCE56GJ3B52UXNUIYQ6CFVR2CGCXDKZYXGY5CWY5235HAQHXRX6A, seg_pos={0 0}, upload=414055]: Uploading piece...
Jul 14 22:33:20.959715836 (517h21m47.886010533s) [noise=true , address=45.91.116.10:4111 , piece_id=LCZSVS34DIVD4LVRF73L6ATATHWMR4DVJQ4GN3HADJSZTPLCFBTA, seg_pos={0 0}, upload=414055]: Uploading piece...
Jul 14 22:33:20.960115490 (517h21m47.886410182s) [noise=true , address=45.91.95.2:4005 , piece_id=XTY4LNMWW23XE7HEZ6GB2WOS37K3ZAOMY2EVCHIQLDB6IQDMWZ6A, seg_pos={0 0}, upload=414055]: Uploading piece...
Jul 14 22:33:20.960349946 (517h21m47.886644642s) [noise=true , address=77.90.31.14:4694 , piece_id=GXYYM54EJCQQHN7UNLTOP52UVHPQ2E7I36ETL64JMQWH6FT5KEGA, seg_pos={0 0}, upload=414055]: Uploading piece...
Jul 14 22:33:20.960857778 (517h21m47.88715247s) [noise=true , address=104.36.82.34:4562 , piece_id=HZ7BIXYA7TOMLS3ANW4SSS5WTPOYTRF6JOKO26KJR6TMBSJJPA4Q, seg_pos={0 0}, upload=414055]: Uploading piece...
Jul 14 22:33:21.054689329 (517h21m47.980984015s) [upload=414055]: Next returned nil segment
Jul 14 22:33:21.054703519 (517h21m47.980998192s) [upload=414055]: Waiting for error group managing segments...
Jul 14 22:33:21.067731227 (517h21m47.994025919s) [noise=true , address=104.36.82.34:4562 , piece_id=HZ7BIXYA7TOMLS3ANW4SSS5WTPOYTRF6JOKO26KJR6TMBSJJPA4Q, seg_pos={0 0}, upload=414055]: Done uploading piece. err: <nil>
Jul 14 22:33:21.068562910 (517h21m47.994857586s) [noise=true , address=77.90.31.18:4810 , piece_id=L2G6QQTX3FVYCW7Z7FMLM6SAA7XZI64BRADRPCOUBLBRVL223KWQ, seg_pos={0 0}, upload=414055]: Done uploading piece. err: <nil>
Jul 14 22:33:21.070198720 (517h21m47.996493396s) [noise=true , address=45.91.116.10:4111 , piece_id=LCZSVS34DIVD4LVRF73L6ATATHWMR4DVJQ4GN3HADJSZTPLCFBTA, seg_pos={0 0}, upload=414055]: Done uploading piece. err: <nil>
Jul 14 22:33:21.071365233 (517h21m47.997659906s) [noise=true , address=45.91.116.14:4215 , piece_id=JB6J36RX7ZLVP4SBF3ZOJSXEZWILQXZOFUQON5WIWMRIAFDZYARA, seg_pos={0 0}, upload=414055]: Done uploading piece. err: <nil>
Jul 14 22:33:21.071792066 (517h21m47.998086738s) [noise=true , address=45.91.116.6:4005 , piece_id=KZTESMI5CMWF5U7S6PXTUK5Z4HSVSXXA6CJ2VAVL5SLK2IVJG3DQ, seg_pos={0 0}, upload=414055]: Done uploading piece. err: <nil>
Jul 14 22:33:21.074850939 (517h21m48.001145615s) [noise=true , address=77.90.31.11:4556 , piece_id=4WXTBJSV6F4WCYX3ETHYA7LKJEHEAFNNOD7E5X6GY65ZYUYEAC3A, seg_pos={0 0}, upload=414055]: Done uploading piece. err: <nil>
Jul 14 22:33:21.075297712 (517h21m48.001592384s) [noise=true , address=45.91.116.10:4098 , piece_id=QAD3V2YB62ZWM7NR5YTZD5YBHIZONG64A27OM6C7WS6R6XE5IAKA, seg_pos={0 0}, upload=414055]: Done uploading piece. err: <nil>
Jul 14 22:33:21.076301394 (517h21m48.002596084s) [noise=true , address=104.36.82.17:4334 , piece_id=6QFH4V2XP5HENEXYSPBLCGWESKAO2M7S243GVWQCSGYYANMDC5BQ, seg_pos={0 0}, upload=414055]: Done uploading piece. err: <nil>
Jul 14 22:33:21.078308059 (517h21m48.00460275s) [noise=true , address=104.36.82.19:4432 , piece_id=ZNUQB26NCFHHETSPPLIX52BSR23AI2VO3J2BJA5O4FPUDRD6ILEA, seg_pos={0 0}, upload=414055]: Done uploading piece. err: <nil>
Jul 14 22:33:21.079380196 (517h21m48.005674875s) [noise=true , address=77.90.31.14:4694 , piece_id=GXYYM54EJCQQHN7UNLTOP52UVHPQ2E7I36ETL64JMQWH6FT5KEGA, seg_pos={0 0}, upload=414055]: Done uploading piece. err: <nil>
Jul 14 22:33:21.086892708 (517h21m48.013187388s) [noise=true , address=104.36.82.10:4101 , piece_id=6I325W4T6GYA7B6TXAEA33PGTNXQSWH5AB6MRRWPJM6O66LXTDAA, seg_pos={0 0}, upload=414055]: Done uploading piece. err: <nil>
Jul 14 22:33:21.096904024 (517h21m48.023198705s) [noise=true , address=77.90.31.19:4873 , piece_id=ALRBAJG2MS2FX2JQQORO5ZLLXGFQGMBIXLKRTQB3OKIICVAAQTKA, seg_pos={0 0}, upload=414055]: Done uploading piece. err: <nil>
Jul 14 22:33:21.097936100 (517h21m48.024230776s) [noise=true , address=77.90.31.12:4604 , piece_id=ZCE56GJ3B52UXNUIYQ6CFVR2CGCXDKZYXGY5CWY5235HAQHXRX6A, seg_pos={0 0}, upload=414055]: Done uploading piece. err: <nil>
Jul 14 22:33:21.107109151 (517h21m48.03340383s) [noise=true , address=77.90.31.15:4704 , piece_id=K7MIP54PNDQCEH6FKFEHFDYAYQ2ZWOHHARJ26JJYLD6YKJCRX64A, seg_pos={0 0}, upload=414055]: Done uploading piece. err: <nil>
Jul 14 22:33:21.107140287 (517h21m48.033434963s) [seg_pos={0 0}, upload=414055]: Segment reached optimal threshold of 14 pieces.
Jul 14 22:33:21.107798713 (517h21m48.034093407s) [noise=true , address=45.91.95.2:4005 , piece_id=XTY4LNMWW23XE7HEZ6GB2WOS37K3ZAOMY2EVCHIQLDB6IQDMWZ6A, seg_pos={0 0}, upload=414055]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bxFpBmz99gV4rL6bMtnHuthK2Xg4EusbfMAWF37RTRtPkqhQS): protocol: context canceled
Jul 14 22:33:21.107904677 (517h21m48.034199363s) [noise=true , address=104.36.82.51:4074 , piece_id=HOC7FQM2ROFQMQOOQMXPD5HVLON2C6U2UVSS5HMLGRIWJIIO3MUQ, seg_pos={0 0}, upload=414055]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1DBQpuE1EC1ENCy67We5QhMT6zsa2AbeMDSv8hmQLjokGRvoxc): protocol: write tcp 159.223.143.39:44794->104.36.82.51:4074: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 14 22:33:21.107928716 (517h21m48.034223405s) [noise=true , address=45.91.95.3:4028 , piece_id=WPAZ53X2MORZF3ZAL42BEZW2HQ2FC7YEBX7YGGFNLOBALDZB7JUQ, seg_pos={0 0}, upload=414055]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Cn2vydXYjCSFYkLuBck57avzeCstHowf3cDQwtYn2RV2Wskh2): protocol: write tcp 159.223.143.39:33072->45.91.95.3:4028: use of closed network connection; write tcp 159.223.143.39:33072->45.91.95.3:4028: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:33072->45.91.95.3:4028: use of closed network connection
Jul 14 22:33:21.108008084 (517h21m48.034302763s) [noise=true , address=45.91.95.4:4001 , piece_id=C4KRMZGJX4PWVYSTRKXHCRK34AABH2ZLHVXXRKADJIOFOXX45LTA, seg_pos={0 0}, upload=414055]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PdimTdB4ysD3SQ8hKMAGAXkzzC71irLv6rW9FArf5qmrvZ5MY): protocol: read tcp 159.223.143.39:55404->45.91.95.4:4001: use of closed network connection; context canceled
Jul 14 22:33:21.108022527 (517h21m48.034317213s) [seg_pos={0 0}, upload=414055]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jul 14 22:33:21.108148869 (517h21m48.034443556s) [upload=414055]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 14 22:33:21.108154867 (517h21m48.034449552s) [upload=414055]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 14 22:33:21.175128505 (517h21m48.101423192s) [upload=414055]: Done uploading segments...
Jul 14 22:33:21.175148328 (517h21m48.101443004s) [upload=414055]: Upload finished. err: <nil>
Duration: 651.6716ms
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)antiblip:
2 out of 60 is failed (threshold=3) [....................X...............................X.......] duration:
365.160451ms target:
https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi
FAIL (2025-07-14 22:54:59 UTC)
orbiter warning: Get "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi": dial tcp: lookup www.storj-ipfs.com on 127.0.0.1:53: read udp 127.0.0.1:40713->127.0.0.1:53: i/o timeout
Duration: 365.188553ms
edge
ipfs
us1
|
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)antiblip:
2 out of 60 is failed (threshold=3) [....................XX......................................] duration:
144.815208ms
FAIL (2025-07-14 22:54:59 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp: lookup auth.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:53370->127.0.0.1:53: i/o timeout
Duration: 144.854227ms
edge
authservice
http
bgp
us1
|
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777"} failure (antiblip window 3/60): (authservice.ap1)antiblip:
2 out of 60 is failed (threshold=3) [................X.X.........................................] duration:
833.391039ms
FAIL (2025-07-14 22:54:57 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp: lookup auth.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:50971->127.0.0.1:53: i/o timeout
Duration: 833.43346ms
edge
authservice
http
bgp
ap1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing)antiblip:
5 out of 60 is failed (threshold=6) [............X................X.X...........XX...............] duration:
26.480154ms target:
https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt
FAIL (2025-07-14 22:54:52 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt": dial tcp: lookup link.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:51321->127.0.0.1:53: i/o timeout
Duration: 26.504471ms
edge
linksharing
bgp
us1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
2 out of 60 is failed (threshold=3) [...................................X.................X......] duration:
2.700412774s
FAIL (2025-07-14 22:54:50 UTC)
orbiter warning: uplink: bucket: metaclient: context deadline exceeded
Duration: 2.700454998s
core
uplink
eu1
|
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)antiblip:
3 out of 60 is failed (threshold=3) [...............X..................................XX........] duration:
225.13429ms target:
https://www.storj-ipfs.com/api/v0/add
FAIL (2025-07-14 22:54:50 UTC)
orbiter warning: Post "https://www.storj-ipfs.com/api/v0/add": dial tcp: lookup www.storj-ipfs.com on 127.0.0.1:53: read udp 127.0.0.1:54967->127.0.0.1:53: i/o timeout
Duration: 225.155839ms
edge
ipfs
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
2 out of 60 is failed (threshold=3) [...............................XX...........................] duration:
274.470029ms
FAIL (2025-07-14 22:54:46 UTC)
orbiter warning: uplink: bucket: metaclient: context deadline exceeded
Duration: 274.500748ms
core
uplink
us1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)antiblip:
2 out of 60 is failed (threshold=3) [...................X........X...............................] duration:
265.511996ms target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011
FAIL (2025-07-14 22:54:46 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011": dial tcp: lookup uptime-checks.gateway.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:40445->127.0.0.1:53: i/o timeout
Duration: 265.62418ms
edge
gateway
vhost
bgp
ap1
|
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)antiblip:
2 out of 60 is failed (threshold=3) [..................................X...................X.....] duration:
251.377902ms target:
https://eu1.storj.io/login
FAIL (2025-07-14 22:54:43 UTC)
orbiter warning: Get "https://eu1.storj.io/login": dial tcp: lookup eu1.storj.io on 127.0.0.1:53: read udp 127.0.0.1:37393->127.0.0.1:53: i/o timeout
Duration: 251.425827ms
core
satellite
eu1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [................X...X.......................................] duration:
1.121196359s
FAIL (2025-07-14 22:54:43 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp: lookup auth.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:50870->127.0.0.1:53: i/o timeout
Duration: 1.121216085s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [..................................XX........................] duration:
507.424816ms
FAIL (2025-07-14 22:54:42 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: rpc: tcp connector failed: rpc: context deadline exceeded
======LOGS======
RequestItem_ObjectBegin
Jul 14 22:33:20.920268234 (517h21m47.84656293s) [upload=414055]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 14 22:33:20.956049835 (517h21m47.882344525s) [seg_pos={0 0}, upload=414055]: Begin upload segment...
Jul 14 22:33:20.956138063 (517h21m47.882432739s) [seg_pos={0 0}, upload=414055]: Done begin upload segment.
Jul 14 22:33:20.956148696 (517h21m47.88244339s) [noise=true , address=104.36.82.19:4432 , piece_id=ZNUQB26NCFHHETSPPLIX52BSR23AI2VO3J2BJA5O4FPUDRD6ILEA, seg_pos={0 0}, upload=414055]: Uploading piece...
Jul 14 22:33:20.956179183 (517h21m47.882473875s) [noise=true , address=45.91.95.4:4001 , piece_id=C4KRMZGJX4PWVYSTRKXHCRK34AABH2ZLHVXXRKADJIOFOXX45LTA, seg_pos={0 0}, upload=414055]: Uploading piece...
Jul 14 22:33:20.956739778 (517h21m47.883034473s) [noise=true , address=104.36.82.17:4334 , piece_id=6QFH4V2XP5HENEXYSPBLCGWESKAO2M7S243GVWQCSGYYANMDC5BQ, seg_pos={0 0}, upload=414055]: Uploading piece...
Jul 14 22:33:20.957112661 (517h21m47.883407356s) [noise=true , address=45.91.116.14:4215 , piece_id=JB6J36RX7ZLVP4SBF3ZOJSXEZWILQXZOFUQON5WIWMRIAFDZYARA, seg_pos={0 0}, upload=414055]: Uploading piece...
Jul 14 22:33:20.957321762 (517h21m47.883616454s) [noise=true , address=77.90.31.19:4873 , piece_id=ALRBAJG2MS2FX2JQQORO5ZLLXGFQGMBIXLKRTQB3OKIICVAAQTKA, seg_pos={0 0}, upload=414055]: Uploading piece...
Jul 14 22:33:20.957534791 (517h21m47.883829483s) [noise=true , address=104.36.82.51:4074 , piece_id=HOC7FQM2ROFQMQOOQMXPD5HVLON2C6U2UVSS5HMLGRIWJIIO3MUQ, seg_pos={0 0}, upload=414055]: Uploading piece...
Jul 14 22:33:20.957685297 (517h21m47.883979986s) [noise=true , address=45.91.116.10:4098 , piece_id=QAD3V2YB62ZWM7NR5YTZD5YBHIZONG64A27OM6C7WS6R6XE5IAKA, seg_pos={0 0}, upload=414055]: Uploading piece...
Jul 14 22:33:20.957863790 (517h21m47.884158483s) [noise=true , address=77.90.31.18:4810 , piece_id=L2G6QQTX3FVYCW7Z7FMLM6SAA7XZI64BRADRPCOUBLBRVL223KWQ, seg_pos={0 0}, upload=414055]: Uploading piece...
Jul 14 22:33:20.958078206 (517h21m47.884372898s) [noise=true , address=77.90.31.11:4556 , piece_id=4WXTBJSV6F4WCYX3ETHYA7LKJEHEAFNNOD7E5X6GY65ZYUYEAC3A, seg_pos={0 0}, upload=414055]: Uploading piece...
Jul 14 22:33:20.958206699 (517h21m47.884501391s) [noise=true , address=45.91.116.6:4005 , piece_id=KZTESMI5CMWF5U7S6PXTUK5Z4HSVSXXA6CJ2VAVL5SLK2IVJG3DQ, seg_pos={0 0}, upload=414055]: Uploading piece...
Jul 14 22:33:20.958449262 (517h21m47.884743949s) [noise=true , address=104.36.82.10:4101 , piece_id=6I325W4T6GYA7B6TXAEA33PGTNXQSWH5AB6MRRWPJM6O66LXTDAA, seg_pos={0 0}, upload=414055]: Uploading piece...
Jul 14 22:33:20.958598885 (517h21m47.884893577s) [noise=true , address=45.91.95.3:4028 , piece_id=WPAZ53X2MORZF3ZAL42BEZW2HQ2FC7YEBX7YGGFNLOBALDZB7JUQ, seg_pos={0 0}, upload=414055]: Uploading piece...
Jul 14 22:33:20.958933707 (517h21m47.885228398s) [noise=true , address=77.90.31.15:4704 , piece_id=K7MIP54PNDQCEH6FKFEHFDYAYQ2ZWOHHARJ26JJYLD6YKJCRX64A, seg_pos={0 0}, upload=414055]: Uploading piece...
Jul 14 22:33:20.959332655 (517h21m47.885627352s) [noise=true , address=77.90.31.12:4604 , piece_id=ZCE56GJ3B52UXNUIYQ6CFVR2CGCXDKZYXGY5CWY5235HAQHXRX6A, seg_pos={0 0}, upload=414055]: Uploading piece...
Jul 14 22:33:20.959715836 (517h21m47.886010533s) [noise=true , address=45.91.116.10:4111 , piece_id=LCZSVS34DIVD4LVRF73L6ATATHWMR4DVJQ4GN3HADJSZTPLCFBTA, seg_pos={0 0}, upload=414055]: Uploading piece...
Jul 14 22:33:20.960115490 (517h21m47.886410182s) [noise=true , address=45.91.95.2:4005 , piece_id=XTY4LNMWW23XE7HEZ6GB2WOS37K3ZAOMY2EVCHIQLDB6IQDMWZ6A, seg_pos={0 0}, upload=414055]: Uploading piece...
Jul 14 22:33:20.960349946 (517h21m47.886644642s) [noise=true , address=77.90.31.14:4694 , piece_id=GXYYM54EJCQQHN7UNLTOP52UVHPQ2E7I36ETL64JMQWH6FT5KEGA, seg_pos={0 0}, upload=414055]: Uploading piece...
Jul 14 22:33:20.960857778 (517h21m47.88715247s) [noise=true , address=104.36.82.34:4562 , piece_id=HZ7BIXYA7TOMLS3ANW4SSS5WTPOYTRF6JOKO26KJR6TMBSJJPA4Q, seg_pos={0 0}, upload=414055]: Uploading piece...
Jul 14 22:33:21.054689329 (517h21m47.980984015s) [upload=414055]: Next returned nil segment
Jul 14 22:33:21.054703519 (517h21m47.980998192s) [upload=414055]: Waiting for error group managing segments...
Jul 14 22:33:21.067731227 (517h21m47.994025919s) [noise=true , address=104.36.82.34:4562 , piece_id=HZ7BIXYA7TOMLS3ANW4SSS5WTPOYTRF6JOKO26KJR6TMBSJJPA4Q, seg_pos={0 0}, upload=414055]: Done uploading piece. err: <nil>
Jul 14 22:33:21.068562910 (517h21m47.994857586s) [noise=true , address=77.90.31.18:4810 , piece_id=L2G6QQTX3FVYCW7Z7FMLM6SAA7XZI64BRADRPCOUBLBRVL223KWQ, seg_pos={0 0}, upload=414055]: Done uploading piece. err: <nil>
Jul 14 22:33:21.070198720 (517h21m47.996493396s) [noise=true , address=45.91.116.10:4111 , piece_id=LCZSVS34DIVD4LVRF73L6ATATHWMR4DVJQ4GN3HADJSZTPLCFBTA, seg_pos={0 0}, upload=414055]: Done uploading piece. err: <nil>
Jul 14 22:33:21.071365233 (517h21m47.997659906s) [noise=true , address=45.91.116.14:4215 , piece_id=JB6J36RX7ZLVP4SBF3ZOJSXEZWILQXZOFUQON5WIWMRIAFDZYARA, seg_pos={0 0}, upload=414055]: Done uploading piece. err: <nil>
Jul 14 22:33:21.071792066 (517h21m47.998086738s) [noise=true , address=45.91.116.6:4005 , piece_id=KZTESMI5CMWF5U7S6PXTUK5Z4HSVSXXA6CJ2VAVL5SLK2IVJG3DQ, seg_pos={0 0}, upload=414055]: Done uploading piece. err: <nil>
Jul 14 22:33:21.074850939 (517h21m48.001145615s) [noise=true , address=77.90.31.11:4556 , piece_id=4WXTBJSV6F4WCYX3ETHYA7LKJEHEAFNNOD7E5X6GY65ZYUYEAC3A, seg_pos={0 0}, upload=414055]: Done uploading piece. err: <nil>
Jul 14 22:33:21.075297712 (517h21m48.001592384s) [noise=true , address=45.91.116.10:4098 , piece_id=QAD3V2YB62ZWM7NR5YTZD5YBHIZONG64A27OM6C7WS6R6XE5IAKA, seg_pos={0 0}, upload=414055]: Done uploading piece. err: <nil>
Jul 14 22:33:21.076301394 (517h21m48.002596084s) [noise=true , address=104.36.82.17:4334 , piece_id=6QFH4V2XP5HENEXYSPBLCGWESKAO2M7S243GVWQCSGYYANMDC5BQ, seg_pos={0 0}, upload=414055]: Done uploading piece. err: <nil>
Jul 14 22:33:21.078308059 (517h21m48.00460275s) [noise=true , address=104.36.82.19:4432 , piece_id=ZNUQB26NCFHHETSPPLIX52BSR23AI2VO3J2BJA5O4FPUDRD6ILEA, seg_pos={0 0}, upload=414055]: Done uploading piece. err: <nil>
Jul 14 22:33:21.079380196 (517h21m48.005674875s) [noise=true , address=77.90.31.14:4694 , piece_id=GXYYM54EJCQQHN7UNLTOP52UVHPQ2E7I36ETL64JMQWH6FT5KEGA, seg_pos={0 0}, upload=414055]: Done uploading piece. err: <nil>
Jul 14 22:33:21.086892708 (517h21m48.013187388s) [noise=true , address=104.36.82.10:4101 , piece_id=6I325W4T6GYA7B6TXAEA33PGTNXQSWH5AB6MRRWPJM6O66LXTDAA, seg_pos={0 0}, upload=414055]: Done uploading piece. err: <nil>
Jul 14 22:33:21.096904024 (517h21m48.023198705s) [noise=true , address=77.90.31.19:4873 , piece_id=ALRBAJG2MS2FX2JQQORO5ZLLXGFQGMBIXLKRTQB3OKIICVAAQTKA, seg_pos={0 0}, upload=414055]: Done uploading piece. err: <nil>
Jul 14 22:33:21.097936100 (517h21m48.024230776s) [noise=true , address=77.90.31.12:4604 , piece_id=ZCE56GJ3B52UXNUIYQ6CFVR2CGCXDKZYXGY5CWY5235HAQHXRX6A, seg_pos={0 0}, upload=414055]: Done uploading piece. err: <nil>
Jul 14 22:33:21.107109151 (517h21m48.03340383s) [noise=true , address=77.90.31.15:4704 , piece_id=K7MIP54PNDQCEH6FKFEHFDYAYQ2ZWOHHARJ26JJYLD6YKJCRX64A, seg_pos={0 0}, upload=414055]: Done uploading piece. err: <nil>
Jul 14 22:33:21.107140287 (517h21m48.033434963s) [seg_pos={0 0}, upload=414055]: Segment reached optimal threshold of 14 pieces.
Jul 14 22:33:21.107798713 (517h21m48.034093407s) [noise=true , address=45.91.95.2:4005 , piece_id=XTY4LNMWW23XE7HEZ6GB2WOS37K3ZAOMY2EVCHIQLDB6IQDMWZ6A, seg_pos={0 0}, upload=414055]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bxFpBmz99gV4rL6bMtnHuthK2Xg4EusbfMAWF37RTRtPkqhQS): protocol: context canceled
Jul 14 22:33:21.107904677 (517h21m48.034199363s) [noise=true , address=104.36.82.51:4074 , piece_id=HOC7FQM2ROFQMQOOQMXPD5HVLON2C6U2UVSS5HMLGRIWJIIO3MUQ, seg_pos={0 0}, upload=414055]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1DBQpuE1EC1ENCy67We5QhMT6zsa2AbeMDSv8hmQLjokGRvoxc): protocol: write tcp 159.223.143.39:44794->104.36.82.51:4074: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 14 22:33:21.107928716 (517h21m48.034223405s) [noise=true , address=45.91.95.3:4028 , piece_id=WPAZ53X2MORZF3ZAL42BEZW2HQ2FC7YEBX7YGGFNLOBALDZB7JUQ, seg_pos={0 0}, upload=414055]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Cn2vydXYjCSFYkLuBck57avzeCstHowf3cDQwtYn2RV2Wskh2): protocol: write tcp 159.223.143.39:33072->45.91.95.3:4028: use of closed network connection; write tcp 159.223.143.39:33072->45.91.95.3:4028: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:33072->45.91.95.3:4028: use of closed network connection
Jul 14 22:33:21.108008084 (517h21m48.034302763s) [noise=true , address=45.91.95.4:4001 , piece_id=C4KRMZGJX4PWVYSTRKXHCRK34AABH2ZLHVXXRKADJIOFOXX45LTA, seg_pos={0 0}, upload=414055]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PdimTdB4ysD3SQ8hKMAGAXkzzC71irLv6rW9FArf5qmrvZ5MY): protocol: read tcp 159.223.143.39:55404->45.91.95.4:4001: use of closed network connection; context canceled
Jul 14 22:33:21.108022527 (517h21m48.034317213s) [seg_pos={0 0}, upload=414055]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jul 14 22:33:21.108148869 (517h21m48.034443556s) [upload=414055]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 14 22:33:21.108154867 (517h21m48.034449552s) [upload=414055]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 14 22:33:21.175128505 (517h21m48.101423192s) [upload=414055]: Done uploading segments...
Jul 14 22:33:21.175148328 (517h21m48.101443004s) [upload=414055]: Upload finished. err: <nil>
Duration: 507.456821ms
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
4 out of 60 is failed (threshold=3) [.............................XXX.............X..............] duration:
41.290882ms target:
https://us1.storj.io/login
FAIL (2025-07-14 22:54:42 UTC)
Get "https://us1.storj.io/login": dial tcp: lookup us1.storj.io on 127.0.0.1:53: read udp 127.0.0.1:44115->127.0.0.1:53: i/o timeout
Duration: 41.3397ms
core
satellite
us1
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
2 out of 60 is failed (threshold=3) [.................X....................X.....................] duration:
2.164618602s
FAIL (2025-07-14 22:54:41 UTC)
orbiter warning: uplink: bucket: metaclient: context deadline exceeded
Duration: 2.164645901s
core
uplink
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
3 out of 60 is failed (threshold=3) [............................XXX.............................] duration:
291.129496ms
FAIL (2025-07-14 22:54:40 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ad Jul 14 22:26:22.101876911 (517h14m49.028171602s) [upload=413961]: Done starting upload Jul 14 22:26:22.101939016 (517h14m49.028233702s) [upload=413961]: Uploading segments... Jul 14 22:26:22.102013964 (517h14m49.028308659s) [upload=413961]: Got next segment. Inline: true Jul 14 22:26:22.102020563 (517h14m49.02831525s) [upload=413961]: Waiting for error group managing segments... Jul 14 22:26:22.102066345 (517h14m49.028361031s) [upload=413961]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 22:26:22.102074758 (517h14m49.028369446s) [upload=413961]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 22:26:22.102078292 (517h14m49.028372977s) [upload=413961]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 22:26:22.191721055 (517h14m49.118015741s) [upload=413961]: Done uploading segments... Jul 14 22:26:22.191739109 (517h14m49.118033784s) [upload=413961]: Upload finished. err: <nil> Jul 14 22:27:05.187541018 (517h15m32.113835711s) [upload=413970]: Starting upload Jul 14 22:27:05.187627985 (517h15m32.113922676s) [upload=413970]: Done starting upload Jul 14 22:27:05.187699511 (517h15m32.113994192s) [upload=413970]: Uploading segments... Jul 14 22:27:05.187769044 (517h15m32.114063739s) [upload=413970]: Got next segment. Inline: true Jul 14 22:27:05.187775252 (517h15m32.11406995s) [upload=413970]: Waiting for error group managing segments... Jul 14 22:27:05.187827226 (517h15m32.114121924s) [upload=413970]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 22:27:05.187833426 (517h15m32.114128119s) [upload=413970]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 22:27:05.187837312 (517h15m32.114132002s) [upload=413970]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 22:27:05.268547355 (517h15m32.194842047s) [upload=413970]: Done uploading segments... Jul 14 22:27:05.268567502 (517h15m32.194862178s) [upload=413970]: Upload finished. err: <nil> Jul 14 22:28:14.458618300 (517h16m41.384912975s) [upload=413987]: Starting upload Jul 14 22:28:14.458745109 (517h16m41.385039799s) [upload=413987]: Done starting upload Jul 14 22:28:14.458859196 (517h16m41.385153874s) [upload=413987]: Uploading segments... Jul 14 22:28:14.458961146 (517h16m41.385255841s) [upload=413987]: Got next segment. Inline: true Jul 14 22:28:14.458967913 (517h16m41.38526261s) [upload=413987]: Waiting for error group managing segments... Jul 14 22:28:14.459061851 (517h16m41.385356542s) [upload=413987]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 22:28:14.459069295 (517h16m41.385363981s) [upload=413987]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 22:28:14.459073440 (517h16m41.38536813s) [upload=413987]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 22:28:14.527793388 (517h16m41.454088085s) [upload=413987]: Done uploading segments... Jul 14 22:28:14.527816718 (517h16m41.454111407s) [upload=413987]: Upload finished. err: <nil> Jul 14 22:28:45.584768006 (517h17m12.511062681s) [upload=413995]: Starting upload Jul 14 22:28:45.584835645 (517h17m12.511130321s) [upload=413995]: Done starting upload Jul 14 22:28:45.584904793 (517h17m12.511199473s) [upload=413995]: Uploading segments... Jul 14 22:28:45.584979419 (517h17m12.511274114s) [upload=413995]: Got next segment. Inline: true Jul 14 22:28:45.584989608 (517h17m12.511284295s) [upload=413995]: Waiting for error group managing segments... Jul 14 22:28:45.585050237 (517h17m12.511344923s) [upload=413995]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 22:28:45.585055965 (517h17m12.511350652s) [upload=413995]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 22:28:45.585083964 (517h17m12.511378657s) [upload=413995]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 22:28:45.666542984 (517h17m12.59283768s) [upload=413995]: Done uploading segments... Jul 14 22:28:45.666566621 (517h17m12.592861292s) [upload=413995]: Upload finished. err: <nil> Jul 14 22:29:23.485493179 (517h17m50.41178787s) [upload=414002]: Starting upload Jul 14 22:29:23.485582455 (517h17m50.411877145s) [upload=414002]: Done starting upload Jul 14 22:29:23.485666072 (517h17m50.411960749s) [upload=414002]: Uploading segments... Jul 14 22:29:23.485712218 (517h17m50.412006915s) [upload=414002]: Got next segment. Inline: true Jul 14 22:29:23.485718174 (517h17m50.412012864s) [upload=414002]: Waiting for error group managing segments... Jul 14 22:29:23.485767949 (517h17m50.412062638s) [upload=414002]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 22:29:23.485774222 (517h17m50.41206891s) [upload=414002]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 22:29:23.485778499 (517h17m50.412073187s) [upload=414002]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 22:29:23.557452042 (517h17m50.483746729s) [upload=414002]: Done uploading segments... Jul 14 22:29:23.557477187 (517h17m50.483771871s) [upload=414002]: Upload finished. err: <nil> Jul 14 22:29:57.800795373 (517h18m24.727090065s) [upload=414009]: Starting upload Jul 14 22:29:57.800873771 (517h18m24.727168461s) [upload=414009]: Done starting upload Jul 14 22:29:57.800968200 (517h18m24.727262892s) [upload=414009]: Uploading segments... Jul 14 22:29:57.801037227 (517h18m24.727331924s) [upload=414009]: Got next segment. Inline: true Jul 14 22:29:57.801043900 (517h18m24.727338593s) [upload=414009]: Waiting for error group managing segments... Jul 14 22:29:57.801223002 (517h18m24.727517698s) [upload=414009]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 22:29:57.801231410 (517h18m24.727526105s) [upload=414009]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 22:29:57.801235144 (517h18m24.727529842s) [upload=414009]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 22:29:57.921237592 (517h18m24.847532271s) [upload=414009]: Done uploading segments... Jul 14 22:29:57.921255617 (517h18m24.847550294s) [upload=414009]: Upload finished. err: <nil> Jul 14 22:30:40.751239620 (517h19m7.677534312s) [upload=414020]: Starting upload Jul 14 22:30:40.751356985 (517h19m7.677651677s) [upload=414020]: Done starting upload Jul 14 22:30:40.751466075 (517h19m7.677760754s) [upload=414020]: Uploading segments... Jul 14 22:30:40.751537099 (517h19m7.677831785s) [upload=414020]: Got next segment. Inline: true Jul 14 22:30:40.751543083 (517h19m7.677837782s) [upload=414020]: Waiting for error group managing segments... Jul 14 22:30:40.751595558 (517h19m7.677890244s) [upload=414020]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 22:30:40.751601057 (517h19m7.677909364s) [upload=414020]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 22:30:40.751618636 (517h19m7.677913323s) [upload=414020]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 22:30:40.832783883 (517h19m7.759078561s) [upload=414020]: Done uploading segments... Jul 14 22:30:40.832802237 (517h19m7.759096913s) [upload=414020]: Upload finished. err: <nil> Jul 14 22:31:18.985603543 (517h19m45.91189822s) [upload=414026]: Starting upload Jul 14 22:31:18.985676609 (517h19m45.911971298s) [upload=414026]: Done starting upload Jul 14 22:31:18.985753552 (517h19m45.912048227s) [upload=414026]: Uploading segments... Jul 14 22:31:18.985802665 (517h19m45.912097361s) [upload=414026]: Got next segment. Inline: true Jul 14 22:31:18.985807556 (517h19m45.912102242s) [upload=414026]: Waiting for error group managing segments... Jul 14 22:31:18.985848046 (517h19m45.912142722s) [upload=414026]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 22:31:18.985863675 (517h19m45.912158362s) [upload=414026]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 22:31:18.985867346 (517h19m45.912162022s) [upload=414026]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 22:31:19.053410124 (517h19m45.979704813s) [upload=414026]: Done uploading segments... Jul 14 22:31:19.053433703 (517h19m45.979728394s) [upload=414026]: Upload finished. err: <nil> Jul 14 22:32:25.786471107 (517h20m52.712765794s) [upload=414043]: Starting upload Jul 14 22:32:25.786529855 (517h20m52.712824533s) [upload=414043]: Done starting upload Jul 14 22:32:25.786625449 (517h20m52.712920125s) [upload=414043]: Uploading segments... Jul 14 22:32:25.786720270 (517h20m52.713014968s) [upload=414043]: Got next segment. Inline: true Jul 14 22:32:25.786730995 (517h20m52.713025684s) [upload=414043]: Waiting for error group managing segments... Jul 14 22:32:25.786797543 (517h20m52.713092232s) [upload=414043]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 22:32:25.786804665 (517h20m52.71309936s) [upload=414043]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 22:32:25.786808867 (517h20m52.713103557s) [upload=414043]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 22:32:25.853631744 (517h20m52.77992643s) [upload=414043]: Done uploading segments... Jul 14 22:32:25.853649513 (517h20m52.77994419s) [upload=414043]: Upload finished. err: <nil> Jul 14 22:33:00.951507975 (517h21m27.877802664s) [upload=414050]: Starting upload Jul 14 22:33:00.951580949 (517h21m27.877875638s) [upload=414050]: Done starting upload Jul 14 22:33:00.951675541 (517h21m27.877970229s) [upload=414050]: Uploading segments... Jul 14 22:33:00.951758272 (517h21m27.878052967s) [upload=414050]: Got next segment. Inline: true Jul 14 22:33:00.951764061 (517h21m27.878058737s) [upload=414050]: Waiting for error group managing segments... Jul 14 22:33:00.951823347 (517h21m27.878118023s) [upload=414050]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jul 14 22:33:00.951829639 (517h21m27.878124315s) [upload=414050]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jul 14 22:33:00.951833041 (517h21m27.878127717s) [upload=414050]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jul 14 22:33:01.021815420 (517h21m27.948110103s) [upload=414050]: Done uploading segments... Jul 14 22:33:01.021850031 (517h21m27.948144718s) [upload=414050]: Upload finished. err: <nil>
Duration: 291.170418ms
core
uplink
us-select-1
|
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777"} failure (antiblip window 3/60): (authservice.eu1)antiblip:
2 out of 60 is failed (threshold=3) [...................X..X.....................................] duration:
357.070632ms
FAIL (2025-07-14 22:54:38 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp: lookup auth.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:50971->127.0.0.1:53: i/o timeout
Duration: 357.103993ms
edge
authservice
http
bgp
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)antiblip:
2 out of 60 is failed (threshold=3) [..................................X..................X......] duration:
850.122852ms
FAIL (2025-07-14 22:54:32 UTC)
orbiter warning: uplink: bucket: metaclient: context deadline exceeded
Duration: 850.157193ms
core
uplink
eu1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
5 out of 60 is failed (threshold=3) [....................X............XXX.........X..............] duration:
1.77905657s
FAIL (2025-07-14 22:54:30 UTC)
uplink: bucket: metaclient: context deadline exceeded
Duration: 1.779097907s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [...............................XX...........................] duration:
510.989194ms
FAIL (2025-07-14 22:54:28 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
22:32:36.255563160 (517h21m3.181857848s) [noise=true , address=45.91.116.5:4297 , piece_id=4DXCPSF4YTD77FXCVOUGH4GWVJBS3JM3WYNERF6DS4UNEBOU2QWQ, seg_pos={0 0}, upload=414047]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF
Jul 14 22:32:36.255577193 (517h21m3.181871868s) [seg_pos={0 0}, upload=414047]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 14 22:32:36.255645270 (517h21m3.181939946s) [upload=414047]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 14 22:32:36.255650241 (517h21m3.181944931s) [upload=414047]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 14 22:32:36.314863469 (517h21m3.241158149s) [upload=414047]: Done uploading segments...
Jul 14 22:32:36.314882474 (517h21m3.241177151s) [upload=414047]: Upload finished. err: <nil>
Jul 14 22:33:11.378492041 (517h21m38.304786743s) [upload=414053]: Starting upload
Jul 14 22:33:11.378577342 (517h21m38.304872032s) [upload=414053]: Done starting upload
Jul 14 22:33:11.378744513 (517h21m38.305039193s) [upload=414053]: Uploading segments...
Jul 14 22:33:11.378953197 (517h21m38.305247877s) [upload=414053]: Got next segment. Inline: false
Jul 14 22:33:11.378994874 (517h21m38.30528956s) [upload=414053]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 14 22:33:11.379000311 (517h21m38.305294992s) [upload=414053]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 14 22:33:11.409521121 (517h21m38.335815814s) [seg_pos={0 0}, upload=414053]: Begin upload segment...
Jul 14 22:33:11.409580940 (517h21m38.335875637s) [seg_pos={0 0}, upload=414053]: Done begin upload segment.
Jul 14 22:33:11.409628148 (517h21m38.335922844s) [noise=true , address=104.36.82.16:4294 , piece_id=QLMMGAWGTTQZXFKV4XVD3YDR2T355TB556XQKWGSLHYTIAZIVDDQ, seg_pos={0 0}, upload=414053]: Uploading piece...
Jul 14 22:33:11.409651610 (517h21m38.335946308s) [noise=true , address=45.91.116.5:4299 , piece_id=5F57Y5UYILBHBF3DV7M5POSK6RFAL3Z7CFA24WMKBEUQAZAL7ODQ, seg_pos={0 0}, upload=414053]: Uploading piece...
Jul 14 22:33:11.410094293 (517h21m38.336388984s) [noise=true , address=45.91.116.9:4263 , piece_id=IO6KT3CIHLSEYQEMZ4DAX4JT4MP4OYARWASXXGV35Q7UHMLIIBOA, seg_pos={0 0}, upload=414053]: Uploading piece...
Jul 14 22:33:11.410493120 (517h21m38.33678781s) [noise=true , address=77.90.31.18:4806 , piece_id=WH2NYTMISPD6II4BSWZDNNPM3H4G2JXZMUCCYKG4GM5LGFAQ2WPQ, seg_pos={0 0}, upload=414053]: Uploading piece...
Jul 14 22:33:11.410954124 (517h21m38.337248825s) [noise=true , address=104.36.82.15:4257 , piece_id=WJRQPXCD4ZS73SBUUPSKGG3VSDLYTDQ7ZQE6B4YW2OIPLAHGIDJA, seg_pos={0 0}, upload=414053]: Uploading piece...
Jul 14 22:33:11.411449799 (517h21m38.337744496s) [noise=true , address=45.91.116.4:4939 , piece_id=ISKKXMSPSMOSP4FWOUKSZ6IMIJFPO33IJDDZ73VNPNSDZ7KQ7SEA, seg_pos={0 0}, upload=414053]: Uploading piece...
Jul 14 22:33:11.411485810 (517h21m38.337780506s) [noise=true , address=104.36.82.11:4113 , piece_id=AK7SBE2BGLIXBKRJGGFRVTRXZGKILEP364KUPGDUS5D7Y74VHJLA, seg_pos={0 0}, upload=414053]: Uploading piece...
Jul 14 22:33:11.411850807 (517h21m38.338145504s) [noise=true , address=45.91.116.12:4225 , piece_id=MDOSVRR4VTFU6A4NIU3HZFTVMJLYTQDKPI2OTVR5SBQ6WGMWBLWA, seg_pos={0 0}, upload=414053]: Uploading piece...
Jul 14 22:33:11.411890633 (517h21m38.338185327s) [noise=true , address=77.90.31.10:4518 , piece_id=KRK47EJXTN4I7TWP7KXWPNYVDUBPOUQGNOG5VN6HDBJ36WE26RHA, seg_pos={0 0}, upload=414053]: Uploading piece...
Jul 14 22:33:11.412220793 (517h21m38.338515486s) [noise=true , address=104.36.82.8:4972 , piece_id=E46WB6ZEOOK23SMTLGJQPGGKDUN3FKBXHSKOY7QLLY24YPPIQVLQ, seg_pos={0 0}, upload=414053]: Uploading piece...
Jul 14 22:33:11.412279103 (517h21m38.338573797s) [noise=true , address=104.36.82.16:4295 , piece_id=NA6OQP5GZJKGH7OMU7OJLTME4OC5LLJUPE3AWBRISBEP7TFSIZIQ, seg_pos={0 0}, upload=414053]: Uploading piece...
Jul 14 22:33:11.412647456 (517h21m38.338942154s) [noise=true , address=104.36.82.4:4804 , piece_id=XO52GRQZ3KWIAQYBDMJTT7D6ADQK35EP232Z4JERAMI6HBEVIXNQ, seg_pos={0 0}, upload=414053]: Uploading piece...
Jul 14 22:33:11.413168375 (517h21m38.33946307s) [noise=true , address=45.91.116.11:4118 , piece_id=VYLKVTNKUCCC6QYXJKGQVRWAJS7X4PEMUDJKJIIGYKHUQBADXQIA, seg_pos={0 0}, upload=414053]: Uploading piece...
Jul 14 22:33:11.413438644 (517h21m38.33973334s) [noise=true , address=45.91.116.14:4191 , piece_id=Q7A6WPWJ3F6D7ZBT27AARER3R3MDQLTUXU3GVBKMUJCUTIE2P3KA, seg_pos={0 0}, upload=414053]: Uploading piece...
Jul 14 22:33:11.413572133 (517h21m38.339866829s) [noise=true , address=77.90.31.17:4772 , piece_id=2JZOVZDHXZ4J5DZYX6EWQ4WSJ6NTMSVQN3WF3VD6AFFHC3J7AFJA, seg_pos={0 0}, upload=414053]: Uploading piece...
Jul 14 22:33:11.413947381 (517h21m38.340242075s) [noise=true , address=104.36.82.13:4186 , piece_id=P7HJGYDOOI4GVIB6AQ3AHCLLXKHSWSR3VDWBIMHTLEVOYRTEV7FQ, seg_pos={0 0}, upload=414053]: Uploading piece...
Jul 14 22:33:11.524370644 (517h21m38.450665324s) [upload=414053]: Next returned nil segment
Jul 14 22:33:11.524388534 (517h21m38.450683224s) [upload=414053]: Waiting for error group managing segments...
Jul 14 22:33:11.529388413 (517h21m38.455683093s) [noise=true , address=77.90.31.10:4518 , piece_id=KRK47EJXTN4I7TWP7KXWPNYVDUBPOUQGNOG5VN6HDBJ36WE26RHA, seg_pos={0 0}, upload=414053]: Done uploading piece. err: <nil>
Jul 14 22:33:11.529842618 (517h21m38.456137297s) [noise=true , address=77.90.31.18:4806 , piece_id=WH2NYTMISPD6II4BSWZDNNPM3H4G2JXZMUCCYKG4GM5LGFAQ2WPQ, seg_pos={0 0}, upload=414053]: Done uploading piece. err: <nil>
Jul 14 22:33:11.530343771 (517h21m38.456638451s) [noise=true , address=45.91.116.5:4299 , piece_id=5F57Y5UYILBHBF3DV7M5POSK6RFAL3Z7CFA24WMKBEUQAZAL7ODQ, seg_pos={0 0}, upload=414053]: Done uploading piece. err: <nil>
Jul 14 22:33:11.530866496 (517h21m38.457161176s) [noise=true , address=45.91.116.12:4225 , piece_id=MDOSVRR4VTFU6A4NIU3HZFTVMJLYTQDKPI2OTVR5SBQ6WGMWBLWA, seg_pos={0 0}, upload=414053]: Done uploading piece. err: <nil>
Jul 14 22:33:11.530939893 (517h21m38.45723458s) [noise=true , address=45.91.116.4:4939 , piece_id=ISKKXMSPSMOSP4FWOUKSZ6IMIJFPO33IJDDZ73VNPNSDZ7KQ7SEA, seg_pos={0 0}, upload=414053]: Done uploading piece. err: <nil>
Jul 14 22:33:11.531326057 (517h21m38.457620733s) [noise=true , address=45.91.116.14:4191 , piece_id=Q7A6WPWJ3F6D7ZBT27AARER3R3MDQLTUXU3GVBKMUJCUTIE2P3KA, seg_pos={0 0}, upload=414053]: Done uploading piece. err: <nil>
Jul 14 22:33:11.531369084 (517h21m38.457663773s) [noise=true , address=45.91.116.11:4118 , piece_id=VYLKVTNKUCCC6QYXJKGQVRWAJS7X4PEMUDJKJIIGYKHUQBADXQIA, seg_pos={0 0}, upload=414053]: Done uploading piece. err: <nil>
Jul 14 22:33:11.531856839 (517h21m38.458151512s) [noise=true , address=45.91.116.9:4263 , piece_id=IO6KT3CIHLSEYQEMZ4DAX4JT4MP4OYARWASXXGV35Q7UHMLIIBOA, seg_pos={0 0}, upload=414053]: Done uploading piece. err: <nil>
Jul 14 22:33:11.534154262 (517h21m38.460448936s) [noise=true , address=104.36.82.13:4186 , piece_id=P7HJGYDOOI4GVIB6AQ3AHCLLXKHSWSR3VDWBIMHTLEVOYRTEV7FQ, seg_pos={0 0}, upload=414053]: Done uploading piece. err: <nil>
Jul 14 22:33:11.534476285 (517h21m38.460770973s) [noise=true , address=104.36.82.16:4295 , piece_id=NA6OQP5GZJKGH7OMU7OJLTME4OC5LLJUPE3AWBRISBEP7TFSIZIQ, seg_pos={0 0}, upload=414053]: Done uploading piece. err: <nil>
Jul 14 22:33:11.536382666 (517h21m38.462677357s) [noise=true , address=104.36.82.8:4972 , piece_id=E46WB6ZEOOK23SMTLGJQPGGKDUN3FKBXHSKOY7QLLY24YPPIQVLQ, seg_pos={0 0}, upload=414053]: Done uploading piece. err: <nil>
Jul 14 22:33:11.536961196 (517h21m38.463255883s) [noise=true , address=104.36.82.15:4257 , piece_id=WJRQPXCD4ZS73SBUUPSKGG3VSDLYTDQ7ZQE6B4YW2OIPLAHGIDJA, seg_pos={0 0}, upload=414053]: Done uploading piece. err: <nil>
Jul 14 22:33:11.537568434 (517h21m38.463863128s) [noise=true , address=104.36.82.16:4294 , piece_id=QLMMGAWGTTQZXFKV4XVD3YDR2T355TB556XQKWGSLHYTIAZIVDDQ, seg_pos={0 0}, upload=414053]: Done uploading piece. err: <nil>
Jul 14 22:33:11.537582802 (517h21m38.463877487s) [seg_pos={0 0}, upload=414053]: Segment reached optimal threshold of 13 pieces.
Jul 14 22:33:11.537702706 (517h21m38.463997382s) [noise=true , address=104.36.82.4:4804 , piece_id=XO52GRQZ3KWIAQYBDMJTT7D6ADQK35EP232Z4JERAMI6HBEVIXNQ, seg_pos={0 0}, upload=414053]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g): protocol: expected piece hash; context canceled; EOF
Jul 14 22:33:11.537835025 (517h21m38.464129712s) [noise=true , address=104.36.82.11:4113 , piece_id=AK7SBE2BGLIXBKRJGGFRVTRXZGKILEP364KUPGDUS5D7Y74VHJLA, seg_pos={0 0}, upload=414053]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV): protocol: expected piece hash; context canceled; EOF
Jul 14 22:33:11.539014185 (517h21m38.465308871s) [noise=true , address=77.90.31.17:4772 , piece_id=2JZOVZDHXZ4J5DZYX6EWQ4WSJ6NTMSVQN3WF3VD6AFFHC3J7AFJA, seg_pos={0 0}, upload=414053]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE): protocol: expected piece hash; context canceled; EOF
Jul 14 22:33:11.539064812 (517h21m38.465359489s) [seg_pos={0 0}, upload=414053]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 14 22:33:11.539134433 (517h21m38.465429119s) [upload=414053]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 14 22:33:11.539139366 (517h21m38.465434043s) [upload=414053]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 14 22:33:11.600764137 (517h21m38.527058814s) [upload=414053]: Done uploading segments...
Jul 14 22:33:11.600783410 (517h21m38.527078087s) [upload=414053]: Upload finished. err: <nil>
Duration: 511.021143ms
core
uplink
us-select-1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)antiblip:
3 out of 60 is failed (threshold=6) [....................X............X.X........................] duration:
113.247951ms target:
https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt
FAIL (2025-07-14 22:54:28 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": dial tcp: lookup link.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:46744->127.0.0.1:53: i/o timeout
Duration: 113.277608ms
edge
linksharing
bgp
eu1
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
2 out of 60 is failed (threshold=3) [..................X....................X....................] duration:
2.278155899s
FAIL (2025-07-14 22:54:26 UTC)
orbiter warning: uplink: bucket: metaclient: context deadline exceeded
Duration: 2.278188353s
core
uplink
ap1
|
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)antiblip:
3 out of 60 is failed (threshold=3) [................X..................................XX.......] duration:
215.94872ms target:
https://www.storj-ipfs.com/api/v0/add
FAIL (2025-07-14 22:54:26 UTC)
orbiter warning: Post "https://www.storj-ipfs.com/api/v0/add": dial tcp: lookup www.storj-ipfs.com on 127.0.0.1:53: read udp 127.0.0.1:54967->127.0.0.1:53: i/o timeout
Duration: 215.979003ms
edge
ipfs
us1
|
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)antiblip:
2 out of 60 is failed (threshold=3) [.....................XX.....................................] duration:
146.607174ms
FAIL (2025-07-14 22:54:25 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp: lookup auth.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:53370->127.0.0.1:53: i/o timeout
Duration: 146.642044ms
edge
authservice
http
bgp
us1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)antiblip:
4 out of 60 is failed (threshold=6) [....................X..............X..X........X............] duration:
743.698461ms target:
https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt
FAIL (2025-07-14 22:54:24 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": dial tcp: lookup link.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:51321->127.0.0.1:53: i/o timeout
Duration: 743.724504ms
edge
linksharing
bgp
ap1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
2 out of 60 is failed (threshold=3) [.....................X........X.............................] duration:
37.242251ms target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222
FAIL (2025-07-14 22:54:23 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": dial tcp: lookup uptime-checks.gateway.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:40390->127.0.0.1:53: i/o timeout
Duration: 37.285164ms
edge
gateway
vhost
bgp
us1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
2 out of 60 is failed (threshold=3) [......................X......X..............................] duration:
108.316825ms target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318
FAIL (2025-07-14 22:54:21 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318": dial tcp: lookup uptime-checks.gateway.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:40390->127.0.0.1:53: i/o timeout
Duration: 108.357381ms
edge
gateway
vhost
bgp
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
2 out of 60 is failed (threshold=3) [....................................X.................X.....] duration:
2.730252586s
FAIL (2025-07-14 22:54:18 UTC)
orbiter warning: uplink: bucket: metaclient: context deadline exceeded
Duration: 2.730282237s
core
uplink
eu1
|
checkers.HTTP{"GET" "http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt", ExpectedStatus: 308, ExpectedRedirect: "https://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt"} failure (antiblip window 3/60): (http.us_select_1.linksharing)antiblip:
5 out of 60 is failed (threshold=3) [................X............X........X...XX................] duration:
19.417347ms target:
http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt
FAIL (2025-07-14 22:54:17 UTC)
orbiter warning: Get "http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt": dial tcp: lookup link.us-select-1.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:59356->127.0.0.1:53: i/o timeout
Duration: 19.441492ms
edge
linksharing
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [.................X...X......................................] duration:
670.44799ms
FAIL (2025-07-14 22:54:17 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp: lookup auth.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:50870->127.0.0.1:53: i/o timeout
Duration: 670.488729ms
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [....................X.......................................] duration:
42.912442ms target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410
FAIL (2025-07-14 22:54:15 UTC)
orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": dial tcp: lookup uptime-checks-us-select.s3.us-select-1.storjapi.io on 127.0.0.1:53: read udp 127.0.0.1:58761->127.0.0.1:53: i/o timeout
Duration: 42.937441ms
edge
gateway
vhost
us-select-1
|
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)antiblip:
2 out of 60 is failed (threshold=3) [...................................X...................X....] duration:
254.833745ms target:
https://eu1.storj.io/login
FAIL (2025-07-14 22:54:15 UTC)
orbiter warning: Get "https://eu1.storj.io/login": dial tcp: lookup eu1.storj.io on 127.0.0.1:53: read udp 127.0.0.1:37393->127.0.0.1:53: i/o timeout
Duration: 254.858861ms
core
satellite
eu1
|
checkers.HTTP{"GET" "https://ap1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.ap1.satellite_login)antiblip:
4 out of 60 is failed (threshold=3) [........................XX.............XX...................] duration:
728.469177ms target:
https://ap1.storj.io/login
FAIL (2025-07-14 22:54:15 UTC)
orbiter warning: Get "https://ap1.storj.io/login": dial tcp: lookup ap1.storj.io on 127.0.0.1:53: read udp 127.0.0.1:45980->127.0.0.1:53: i/o timeout
Duration: 728.495278ms
core
satellite
ap1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................................X........................] duration:
466.617765ms
FAIL (2025-07-14 22:54:15 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 466.643383ms
edge
gateway
us-select-1
|