Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure

Orbiter Check History

Showing 50 of 150186 reports (Page 2754 of 3004)
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)
antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..]
duration:
850.749954ms
FAIL (2025-07-14 22:36:07 UTC)
orbiter warning: uplink: bucket: metaclient: context deadline exceeded
Duration: 850.78357ms core uplink eu1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)
antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...]
duration:
292.475536ms
FAIL (2025-07-14 22:36:07 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 292.560292ms edge gateway us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..]
duration:
2.802911543s
FAIL (2025-07-14 22:36:05 UTC)
orbiter warning: uplink: bucket: metaclient: context deadline exceeded
Duration: 2.802954354s core uplink eu1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)
antiblip:
2 out of 60 is failed (threshold=3) [..............................................X......X......]
duration:
339.100125ms
target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318
FAIL (2025-07-14 22:36:03 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318": dial tcp: lookup uptime-checks.gateway.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:40390->127.0.0.1:53: i/o timeout
Duration: 339.135739ms edge gateway vhost bgp eu1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)
antiblip:
2 out of 60 is failed (threshold=3) [............................................X........X......]
duration:
28.428904ms
target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222
FAIL (2025-07-14 22:36:02 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": dial tcp: lookup uptime-checks.gateway.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:40390->127.0.0.1:53: i/o timeout
Duration: 28.450282ms edge gateway vhost bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
2 out of 60 is failed (threshold=3) [........................................................XX..]
duration:
493.979458ms
FAIL (2025-07-14 22:35:59 UTC)
orbiter warning: 
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
22:32:36.255563160 (517h21m3.181857848s) [noise=true , address=45.91.116.5:4297     , piece_id=4DXCPSF4YTD77FXCVOUGH4GWVJBS3JM3WYNERF6DS4UNEBOU2QWQ, seg_pos={0 0}, upload=414047]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF
Jul 14 22:32:36.255577193 (517h21m3.181871868s) [seg_pos={0 0}, upload=414047]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 14 22:32:36.255645270 (517h21m3.181939946s) [upload=414047]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 14 22:32:36.255650241 (517h21m3.181944931s) [upload=414047]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 14 22:32:36.314863469 (517h21m3.241158149s) [upload=414047]: Done uploading segments...
Jul 14 22:32:36.314882474 (517h21m3.241177151s) [upload=414047]: Upload finished. err: <nil>
Jul 14 22:33:11.378492041 (517h21m38.304786743s) [upload=414053]: Starting upload
Jul 14 22:33:11.378577342 (517h21m38.304872032s) [upload=414053]: Done starting upload
Jul 14 22:33:11.378744513 (517h21m38.305039193s) [upload=414053]: Uploading segments...
Jul 14 22:33:11.378953197 (517h21m38.305247877s) [upload=414053]: Got next segment. Inline: false
Jul 14 22:33:11.378994874 (517h21m38.30528956s) [upload=414053]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 14 22:33:11.379000311 (517h21m38.305294992s) [upload=414053]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 14 22:33:11.409521121 (517h21m38.335815814s) [seg_pos={0 0}, upload=414053]: Begin upload segment...
Jul 14 22:33:11.409580940 (517h21m38.335875637s) [seg_pos={0 0}, upload=414053]: Done begin upload segment.
Jul 14 22:33:11.409628148 (517h21m38.335922844s) [noise=true , address=104.36.82.16:4294    , piece_id=QLMMGAWGTTQZXFKV4XVD3YDR2T355TB556XQKWGSLHYTIAZIVDDQ, seg_pos={0 0}, upload=414053]: Uploading piece...
Jul 14 22:33:11.409651610 (517h21m38.335946308s) [noise=true , address=45.91.116.5:4299     , piece_id=5F57Y5UYILBHBF3DV7M5POSK6RFAL3Z7CFA24WMKBEUQAZAL7ODQ, seg_pos={0 0}, upload=414053]: Uploading piece...
Jul 14 22:33:11.410094293 (517h21m38.336388984s) [noise=true , address=45.91.116.9:4263     , piece_id=IO6KT3CIHLSEYQEMZ4DAX4JT4MP4OYARWASXXGV35Q7UHMLIIBOA, seg_pos={0 0}, upload=414053]: Uploading piece...
Jul 14 22:33:11.410493120 (517h21m38.33678781s) [noise=true , address=77.90.31.18:4806     , piece_id=WH2NYTMISPD6II4BSWZDNNPM3H4G2JXZMUCCYKG4GM5LGFAQ2WPQ, seg_pos={0 0}, upload=414053]: Uploading piece...
Jul 14 22:33:11.410954124 (517h21m38.337248825s) [noise=true , address=104.36.82.15:4257    , piece_id=WJRQPXCD4ZS73SBUUPSKGG3VSDLYTDQ7ZQE6B4YW2OIPLAHGIDJA, seg_pos={0 0}, upload=414053]: Uploading piece...
Jul 14 22:33:11.411449799 (517h21m38.337744496s) [noise=true , address=45.91.116.4:4939     , piece_id=ISKKXMSPSMOSP4FWOUKSZ6IMIJFPO33IJDDZ73VNPNSDZ7KQ7SEA, seg_pos={0 0}, upload=414053]: Uploading piece...
Jul 14 22:33:11.411485810 (517h21m38.337780506s) [noise=true , address=104.36.82.11:4113    , piece_id=AK7SBE2BGLIXBKRJGGFRVTRXZGKILEP364KUPGDUS5D7Y74VHJLA, seg_pos={0 0}, upload=414053]: Uploading piece...
Jul 14 22:33:11.411850807 (517h21m38.338145504s) [noise=true , address=45.91.116.12:4225    , piece_id=MDOSVRR4VTFU6A4NIU3HZFTVMJLYTQDKPI2OTVR5SBQ6WGMWBLWA, seg_pos={0 0}, upload=414053]: Uploading piece...
Jul 14 22:33:11.411890633 (517h21m38.338185327s) [noise=true , address=77.90.31.10:4518     , piece_id=KRK47EJXTN4I7TWP7KXWPNYVDUBPOUQGNOG5VN6HDBJ36WE26RHA, seg_pos={0 0}, upload=414053]: Uploading piece...
Jul 14 22:33:11.412220793 (517h21m38.338515486s) [noise=true , address=104.36.82.8:4972     , piece_id=E46WB6ZEOOK23SMTLGJQPGGKDUN3FKBXHSKOY7QLLY24YPPIQVLQ, seg_pos={0 0}, upload=414053]: Uploading piece...
Jul 14 22:33:11.412279103 (517h21m38.338573797s) [noise=true , address=104.36.82.16:4295    , piece_id=NA6OQP5GZJKGH7OMU7OJLTME4OC5LLJUPE3AWBRISBEP7TFSIZIQ, seg_pos={0 0}, upload=414053]: Uploading piece...
Jul 14 22:33:11.412647456 (517h21m38.338942154s) [noise=true , address=104.36.82.4:4804     , piece_id=XO52GRQZ3KWIAQYBDMJTT7D6ADQK35EP232Z4JERAMI6HBEVIXNQ, seg_pos={0 0}, upload=414053]: Uploading piece...
Jul 14 22:33:11.413168375 (517h21m38.33946307s) [noise=true , address=45.91.116.11:4118    , piece_id=VYLKVTNKUCCC6QYXJKGQVRWAJS7X4PEMUDJKJIIGYKHUQBADXQIA, seg_pos={0 0}, upload=414053]: Uploading piece...
Jul 14 22:33:11.413438644 (517h21m38.33973334s) [noise=true , address=45.91.116.14:4191    , piece_id=Q7A6WPWJ3F6D7ZBT27AARER3R3MDQLTUXU3GVBKMUJCUTIE2P3KA, seg_pos={0 0}, upload=414053]: Uploading piece...
Jul 14 22:33:11.413572133 (517h21m38.339866829s) [noise=true , address=77.90.31.17:4772     , piece_id=2JZOVZDHXZ4J5DZYX6EWQ4WSJ6NTMSVQN3WF3VD6AFFHC3J7AFJA, seg_pos={0 0}, upload=414053]: Uploading piece...
Jul 14 22:33:11.413947381 (517h21m38.340242075s) [noise=true , address=104.36.82.13:4186    , piece_id=P7HJGYDOOI4GVIB6AQ3AHCLLXKHSWSR3VDWBIMHTLEVOYRTEV7FQ, seg_pos={0 0}, upload=414053]: Uploading piece...
Jul 14 22:33:11.524370644 (517h21m38.450665324s) [upload=414053]: Next returned nil segment
Jul 14 22:33:11.524388534 (517h21m38.450683224s) [upload=414053]: Waiting for error group managing segments...
Jul 14 22:33:11.529388413 (517h21m38.455683093s) [noise=true , address=77.90.31.10:4518     , piece_id=KRK47EJXTN4I7TWP7KXWPNYVDUBPOUQGNOG5VN6HDBJ36WE26RHA, seg_pos={0 0}, upload=414053]: Done uploading piece. err: <nil>
Jul 14 22:33:11.529842618 (517h21m38.456137297s) [noise=true , address=77.90.31.18:4806     , piece_id=WH2NYTMISPD6II4BSWZDNNPM3H4G2JXZMUCCYKG4GM5LGFAQ2WPQ, seg_pos={0 0}, upload=414053]: Done uploading piece. err: <nil>
Jul 14 22:33:11.530343771 (517h21m38.456638451s) [noise=true , address=45.91.116.5:4299     , piece_id=5F57Y5UYILBHBF3DV7M5POSK6RFAL3Z7CFA24WMKBEUQAZAL7ODQ, seg_pos={0 0}, upload=414053]: Done uploading piece. err: <nil>
Jul 14 22:33:11.530866496 (517h21m38.457161176s) [noise=true , address=45.91.116.12:4225    , piece_id=MDOSVRR4VTFU6A4NIU3HZFTVMJLYTQDKPI2OTVR5SBQ6WGMWBLWA, seg_pos={0 0}, upload=414053]: Done uploading piece. err: <nil>
Jul 14 22:33:11.530939893 (517h21m38.45723458s) [noise=true , address=45.91.116.4:4939     , piece_id=ISKKXMSPSMOSP4FWOUKSZ6IMIJFPO33IJDDZ73VNPNSDZ7KQ7SEA, seg_pos={0 0}, upload=414053]: Done uploading piece. err: <nil>
Jul 14 22:33:11.531326057 (517h21m38.457620733s) [noise=true , address=45.91.116.14:4191    , piece_id=Q7A6WPWJ3F6D7ZBT27AARER3R3MDQLTUXU3GVBKMUJCUTIE2P3KA, seg_pos={0 0}, upload=414053]: Done uploading piece. err: <nil>
Jul 14 22:33:11.531369084 (517h21m38.457663773s) [noise=true , address=45.91.116.11:4118    , piece_id=VYLKVTNKUCCC6QYXJKGQVRWAJS7X4PEMUDJKJIIGYKHUQBADXQIA, seg_pos={0 0}, upload=414053]: Done uploading piece. err: <nil>
Jul 14 22:33:11.531856839 (517h21m38.458151512s) [noise=true , address=45.91.116.9:4263     , piece_id=IO6KT3CIHLSEYQEMZ4DAX4JT4MP4OYARWASXXGV35Q7UHMLIIBOA, seg_pos={0 0}, upload=414053]: Done uploading piece. err: <nil>
Jul 14 22:33:11.534154262 (517h21m38.460448936s) [noise=true , address=104.36.82.13:4186    , piece_id=P7HJGYDOOI4GVIB6AQ3AHCLLXKHSWSR3VDWBIMHTLEVOYRTEV7FQ, seg_pos={0 0}, upload=414053]: Done uploading piece. err: <nil>
Jul 14 22:33:11.534476285 (517h21m38.460770973s) [noise=true , address=104.36.82.16:4295    , piece_id=NA6OQP5GZJKGH7OMU7OJLTME4OC5LLJUPE3AWBRISBEP7TFSIZIQ, seg_pos={0 0}, upload=414053]: Done uploading piece. err: <nil>
Jul 14 22:33:11.536382666 (517h21m38.462677357s) [noise=true , address=104.36.82.8:4972     , piece_id=E46WB6ZEOOK23SMTLGJQPGGKDUN3FKBXHSKOY7QLLY24YPPIQVLQ, seg_pos={0 0}, upload=414053]: Done uploading piece. err: <nil>
Jul 14 22:33:11.536961196 (517h21m38.463255883s) [noise=true , address=104.36.82.15:4257    , piece_id=WJRQPXCD4ZS73SBUUPSKGG3VSDLYTDQ7ZQE6B4YW2OIPLAHGIDJA, seg_pos={0 0}, upload=414053]: Done uploading piece. err: <nil>
Jul 14 22:33:11.537568434 (517h21m38.463863128s) [noise=true , address=104.36.82.16:4294    , piece_id=QLMMGAWGTTQZXFKV4XVD3YDR2T355TB556XQKWGSLHYTIAZIVDDQ, seg_pos={0 0}, upload=414053]: Done uploading piece. err: <nil>
Jul 14 22:33:11.537582802 (517h21m38.463877487s) [seg_pos={0 0}, upload=414053]: Segment reached optimal threshold of 13 pieces.
Jul 14 22:33:11.537702706 (517h21m38.463997382s) [noise=true , address=104.36.82.4:4804     , piece_id=XO52GRQZ3KWIAQYBDMJTT7D6ADQK35EP232Z4JERAMI6HBEVIXNQ, seg_pos={0 0}, upload=414053]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g): protocol: expected piece hash; context canceled; EOF
Jul 14 22:33:11.537835025 (517h21m38.464129712s) [noise=true , address=104.36.82.11:4113    , piece_id=AK7SBE2BGLIXBKRJGGFRVTRXZGKILEP364KUPGDUS5D7Y74VHJLA, seg_pos={0 0}, upload=414053]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV): protocol: expected piece hash; context canceled; EOF
Jul 14 22:33:11.539014185 (517h21m38.465308871s) [noise=true , address=77.90.31.17:4772     , piece_id=2JZOVZDHXZ4J5DZYX6EWQ4WSJ6NTMSVQN3WF3VD6AFFHC3J7AFJA, seg_pos={0 0}, upload=414053]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE): protocol: expected piece hash; context canceled; EOF
Jul 14 22:33:11.539064812 (517h21m38.465359489s) [seg_pos={0 0}, upload=414053]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 14 22:33:11.539134433 (517h21m38.465429119s) [upload=414053]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 14 22:33:11.539139366 (517h21m38.465434043s) [upload=414053]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 14 22:33:11.600764137 (517h21m38.527058814s) [upload=414053]: Done uploading segments...
Jul 14 22:33:11.600783410 (517h21m38.527078087s) [upload=414053]: Upload finished. err: <nil>
Duration: 494.003267ms core uplink us-select-1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777"} failure (antiblip window 3/60): (authservice.eu1)
antiblip:
2 out of 60 is failed (threshold=3) [............................................X..X............]
duration:
361.346945ms
FAIL (2025-07-14 22:35:59 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp: lookup auth.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:50971->127.0.0.1:53: i/o timeout
Duration: 361.371712ms edge authservice http bgp eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)
antiblip:
3 out of 60 is failed (threshold=3) [........................................................XXX.]
duration:
23.798199585s
FAIL (2025-07-14 22:35:58 UTC)
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
ad
Jul 14 22:26:22.101876911 (517h14m49.028171602s) [upload=413961]: Done starting upload
Jul 14 22:26:22.101939016 (517h14m49.028233702s) [upload=413961]: Uploading segments...
Jul 14 22:26:22.102013964 (517h14m49.028308659s) [upload=413961]: Got next segment. Inline: true
Jul 14 22:26:22.102020563 (517h14m49.02831525s) [upload=413961]: Waiting for error group managing segments...
Jul 14 22:26:22.102066345 (517h14m49.028361031s) [upload=413961]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 14 22:26:22.102074758 (517h14m49.028369446s) [upload=413961]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Jul 14 22:26:22.102078292 (517h14m49.028372977s) [upload=413961]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 14 22:26:22.191721055 (517h14m49.118015741s) [upload=413961]: Done uploading segments...
Jul 14 22:26:22.191739109 (517h14m49.118033784s) [upload=413961]: Upload finished. err: <nil>
Jul 14 22:27:05.187541018 (517h15m32.113835711s) [upload=413970]: Starting upload
Jul 14 22:27:05.187627985 (517h15m32.113922676s) [upload=413970]: Done starting upload
Jul 14 22:27:05.187699511 (517h15m32.113994192s) [upload=413970]: Uploading segments...
Jul 14 22:27:05.187769044 (517h15m32.114063739s) [upload=413970]: Got next segment. Inline: true
Jul 14 22:27:05.187775252 (517h15m32.11406995s) [upload=413970]: Waiting for error group managing segments...
Jul 14 22:27:05.187827226 (517h15m32.114121924s) [upload=413970]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 14 22:27:05.187833426 (517h15m32.114128119s) [upload=413970]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Jul 14 22:27:05.187837312 (517h15m32.114132002s) [upload=413970]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 14 22:27:05.268547355 (517h15m32.194842047s) [upload=413970]: Done uploading segments...
Jul 14 22:27:05.268567502 (517h15m32.194862178s) [upload=413970]: Upload finished. err: <nil>
Jul 14 22:28:14.458618300 (517h16m41.384912975s) [upload=413987]: Starting upload
Jul 14 22:28:14.458745109 (517h16m41.385039799s) [upload=413987]: Done starting upload
Jul 14 22:28:14.458859196 (517h16m41.385153874s) [upload=413987]: Uploading segments...
Jul 14 22:28:14.458961146 (517h16m41.385255841s) [upload=413987]: Got next segment. Inline: true
Jul 14 22:28:14.458967913 (517h16m41.38526261s) [upload=413987]: Waiting for error group managing segments...
Jul 14 22:28:14.459061851 (517h16m41.385356542s) [upload=413987]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 14 22:28:14.459069295 (517h16m41.385363981s) [upload=413987]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Jul 14 22:28:14.459073440 (517h16m41.38536813s) [upload=413987]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 14 22:28:14.527793388 (517h16m41.454088085s) [upload=413987]: Done uploading segments...
Jul 14 22:28:14.527816718 (517h16m41.454111407s) [upload=413987]: Upload finished. err: <nil>
Jul 14 22:28:45.584768006 (517h17m12.511062681s) [upload=413995]: Starting upload
Jul 14 22:28:45.584835645 (517h17m12.511130321s) [upload=413995]: Done starting upload
Jul 14 22:28:45.584904793 (517h17m12.511199473s) [upload=413995]: Uploading segments...
Jul 14 22:28:45.584979419 (517h17m12.511274114s) [upload=413995]: Got next segment. Inline: true
Jul 14 22:28:45.584989608 (517h17m12.511284295s) [upload=413995]: Waiting for error group managing segments...
Jul 14 22:28:45.585050237 (517h17m12.511344923s) [upload=413995]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 14 22:28:45.585055965 (517h17m12.511350652s) [upload=413995]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Jul 14 22:28:45.585083964 (517h17m12.511378657s) [upload=413995]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 14 22:28:45.666542984 (517h17m12.59283768s) [upload=413995]: Done uploading segments...
Jul 14 22:28:45.666566621 (517h17m12.592861292s) [upload=413995]: Upload finished. err: <nil>
Jul 14 22:29:23.485493179 (517h17m50.41178787s) [upload=414002]: Starting upload
Jul 14 22:29:23.485582455 (517h17m50.411877145s) [upload=414002]: Done starting upload
Jul 14 22:29:23.485666072 (517h17m50.411960749s) [upload=414002]: Uploading segments...
Jul 14 22:29:23.485712218 (517h17m50.412006915s) [upload=414002]: Got next segment. Inline: true
Jul 14 22:29:23.485718174 (517h17m50.412012864s) [upload=414002]: Waiting for error group managing segments...
Jul 14 22:29:23.485767949 (517h17m50.412062638s) [upload=414002]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 14 22:29:23.485774222 (517h17m50.41206891s) [upload=414002]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Jul 14 22:29:23.485778499 (517h17m50.412073187s) [upload=414002]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 14 22:29:23.557452042 (517h17m50.483746729s) [upload=414002]: Done uploading segments...
Jul 14 22:29:23.557477187 (517h17m50.483771871s) [upload=414002]: Upload finished. err: <nil>
Jul 14 22:29:57.800795373 (517h18m24.727090065s) [upload=414009]: Starting upload
Jul 14 22:29:57.800873771 (517h18m24.727168461s) [upload=414009]: Done starting upload
Jul 14 22:29:57.800968200 (517h18m24.727262892s) [upload=414009]: Uploading segments...
Jul 14 22:29:57.801037227 (517h18m24.727331924s) [upload=414009]: Got next segment. Inline: true
Jul 14 22:29:57.801043900 (517h18m24.727338593s) [upload=414009]: Waiting for error group managing segments...
Jul 14 22:29:57.801223002 (517h18m24.727517698s) [upload=414009]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 14 22:29:57.801231410 (517h18m24.727526105s) [upload=414009]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Jul 14 22:29:57.801235144 (517h18m24.727529842s) [upload=414009]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 14 22:29:57.921237592 (517h18m24.847532271s) [upload=414009]: Done uploading segments...
Jul 14 22:29:57.921255617 (517h18m24.847550294s) [upload=414009]: Upload finished. err: <nil>
Jul 14 22:30:40.751239620 (517h19m7.677534312s) [upload=414020]: Starting upload
Jul 14 22:30:40.751356985 (517h19m7.677651677s) [upload=414020]: Done starting upload
Jul 14 22:30:40.751466075 (517h19m7.677760754s) [upload=414020]: Uploading segments...
Jul 14 22:30:40.751537099 (517h19m7.677831785s) [upload=414020]: Got next segment. Inline: true
Jul 14 22:30:40.751543083 (517h19m7.677837782s) [upload=414020]: Waiting for error group managing segments...
Jul 14 22:30:40.751595558 (517h19m7.677890244s) [upload=414020]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 14 22:30:40.751601057 (517h19m7.677909364s) [upload=414020]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Jul 14 22:30:40.751618636 (517h19m7.677913323s) [upload=414020]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 14 22:30:40.832783883 (517h19m7.759078561s) [upload=414020]: Done uploading segments...
Jul 14 22:30:40.832802237 (517h19m7.759096913s) [upload=414020]: Upload finished. err: <nil>
Jul 14 22:31:18.985603543 (517h19m45.91189822s) [upload=414026]: Starting upload
Jul 14 22:31:18.985676609 (517h19m45.911971298s) [upload=414026]: Done starting upload
Jul 14 22:31:18.985753552 (517h19m45.912048227s) [upload=414026]: Uploading segments...
Jul 14 22:31:18.985802665 (517h19m45.912097361s) [upload=414026]: Got next segment. Inline: true
Jul 14 22:31:18.985807556 (517h19m45.912102242s) [upload=414026]: Waiting for error group managing segments...
Jul 14 22:31:18.985848046 (517h19m45.912142722s) [upload=414026]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 14 22:31:18.985863675 (517h19m45.912158362s) [upload=414026]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Jul 14 22:31:18.985867346 (517h19m45.912162022s) [upload=414026]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 14 22:31:19.053410124 (517h19m45.979704813s) [upload=414026]: Done uploading segments...
Jul 14 22:31:19.053433703 (517h19m45.979728394s) [upload=414026]: Upload finished. err: <nil>
Jul 14 22:32:25.786471107 (517h20m52.712765794s) [upload=414043]: Starting upload
Jul 14 22:32:25.786529855 (517h20m52.712824533s) [upload=414043]: Done starting upload
Jul 14 22:32:25.786625449 (517h20m52.712920125s) [upload=414043]: Uploading segments...
Jul 14 22:32:25.786720270 (517h20m52.713014968s) [upload=414043]: Got next segment. Inline: true
Jul 14 22:32:25.786730995 (517h20m52.713025684s) [upload=414043]: Waiting for error group managing segments...
Jul 14 22:32:25.786797543 (517h20m52.713092232s) [upload=414043]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 14 22:32:25.786804665 (517h20m52.71309936s) [upload=414043]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Jul 14 22:32:25.786808867 (517h20m52.713103557s) [upload=414043]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 14 22:32:25.853631744 (517h20m52.77992643s) [upload=414043]: Done uploading segments...
Jul 14 22:32:25.853649513 (517h20m52.77994419s) [upload=414043]: Upload finished. err: <nil>
Jul 14 22:33:00.951507975 (517h21m27.877802664s) [upload=414050]: Starting upload
Jul 14 22:33:00.951580949 (517h21m27.877875638s) [upload=414050]: Done starting upload
Jul 14 22:33:00.951675541 (517h21m27.877970229s) [upload=414050]: Uploading segments...
Jul 14 22:33:00.951758272 (517h21m27.878052967s) [upload=414050]: Got next segment. Inline: true
Jul 14 22:33:00.951764061 (517h21m27.878058737s) [upload=414050]: Waiting for error group managing segments...
Jul 14 22:33:00.951823347 (517h21m27.878118023s) [upload=414050]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 14 22:33:00.951829639 (517h21m27.878124315s) [upload=414050]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Jul 14 22:33:00.951833041 (517h21m27.878127717s) [upload=414050]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 14 22:33:01.021815420 (517h21m27.948110103s) [upload=414050]: Done uploading segments...
Jul 14 22:33:01.021850031 (517h21m27.948144718s) [upload=414050]: Upload finished. err: <nil>
Duration: 23.798216183s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)
antiblip:
2 out of 60 is failed (threshold=3) [..........................................................XX]
duration:
1m0.000589252s
FAIL (2025-07-14 22:35:58 UTC)
orbiter warning: 
======ERROR======
uplink: bucket: metaclient: rpc: tcp connector failed: rpc: context deadline exceeded
======LOGS======
RequestItem_ObjectBegin
Jul 14 22:33:20.920268234 (517h21m47.84656293s) [upload=414055]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 14 22:33:20.956049835 (517h21m47.882344525s) [seg_pos={0 0}, upload=414055]: Begin upload segment...
Jul 14 22:33:20.956138063 (517h21m47.882432739s) [seg_pos={0 0}, upload=414055]: Done begin upload segment.
Jul 14 22:33:20.956148696 (517h21m47.88244339s) [noise=true , address=104.36.82.19:4432    , piece_id=ZNUQB26NCFHHETSPPLIX52BSR23AI2VO3J2BJA5O4FPUDRD6ILEA, seg_pos={0 0}, upload=414055]: Uploading piece...
Jul 14 22:33:20.956179183 (517h21m47.882473875s) [noise=true , address=45.91.95.4:4001      , piece_id=C4KRMZGJX4PWVYSTRKXHCRK34AABH2ZLHVXXRKADJIOFOXX45LTA, seg_pos={0 0}, upload=414055]: Uploading piece...
Jul 14 22:33:20.956739778 (517h21m47.883034473s) [noise=true , address=104.36.82.17:4334    , piece_id=6QFH4V2XP5HENEXYSPBLCGWESKAO2M7S243GVWQCSGYYANMDC5BQ, seg_pos={0 0}, upload=414055]: Uploading piece...
Jul 14 22:33:20.957112661 (517h21m47.883407356s) [noise=true , address=45.91.116.14:4215    , piece_id=JB6J36RX7ZLVP4SBF3ZOJSXEZWILQXZOFUQON5WIWMRIAFDZYARA, seg_pos={0 0}, upload=414055]: Uploading piece...
Jul 14 22:33:20.957321762 (517h21m47.883616454s) [noise=true , address=77.90.31.19:4873     , piece_id=ALRBAJG2MS2FX2JQQORO5ZLLXGFQGMBIXLKRTQB3OKIICVAAQTKA, seg_pos={0 0}, upload=414055]: Uploading piece...
Jul 14 22:33:20.957534791 (517h21m47.883829483s) [noise=true , address=104.36.82.51:4074    , piece_id=HOC7FQM2ROFQMQOOQMXPD5HVLON2C6U2UVSS5HMLGRIWJIIO3MUQ, seg_pos={0 0}, upload=414055]: Uploading piece...
Jul 14 22:33:20.957685297 (517h21m47.883979986s) [noise=true , address=45.91.116.10:4098    , piece_id=QAD3V2YB62ZWM7NR5YTZD5YBHIZONG64A27OM6C7WS6R6XE5IAKA, seg_pos={0 0}, upload=414055]: Uploading piece...
Jul 14 22:33:20.957863790 (517h21m47.884158483s) [noise=true , address=77.90.31.18:4810     , piece_id=L2G6QQTX3FVYCW7Z7FMLM6SAA7XZI64BRADRPCOUBLBRVL223KWQ, seg_pos={0 0}, upload=414055]: Uploading piece...
Jul 14 22:33:20.958078206 (517h21m47.884372898s) [noise=true , address=77.90.31.11:4556     , piece_id=4WXTBJSV6F4WCYX3ETHYA7LKJEHEAFNNOD7E5X6GY65ZYUYEAC3A, seg_pos={0 0}, upload=414055]: Uploading piece...
Jul 14 22:33:20.958206699 (517h21m47.884501391s) [noise=true , address=45.91.116.6:4005     , piece_id=KZTESMI5CMWF5U7S6PXTUK5Z4HSVSXXA6CJ2VAVL5SLK2IVJG3DQ, seg_pos={0 0}, upload=414055]: Uploading piece...
Jul 14 22:33:20.958449262 (517h21m47.884743949s) [noise=true , address=104.36.82.10:4101    , piece_id=6I325W4T6GYA7B6TXAEA33PGTNXQSWH5AB6MRRWPJM6O66LXTDAA, seg_pos={0 0}, upload=414055]: Uploading piece...
Jul 14 22:33:20.958598885 (517h21m47.884893577s) [noise=true , address=45.91.95.3:4028      , piece_id=WPAZ53X2MORZF3ZAL42BEZW2HQ2FC7YEBX7YGGFNLOBALDZB7JUQ, seg_pos={0 0}, upload=414055]: Uploading piece...
Jul 14 22:33:20.958933707 (517h21m47.885228398s) [noise=true , address=77.90.31.15:4704     , piece_id=K7MIP54PNDQCEH6FKFEHFDYAYQ2ZWOHHARJ26JJYLD6YKJCRX64A, seg_pos={0 0}, upload=414055]: Uploading piece...
Jul 14 22:33:20.959332655 (517h21m47.885627352s) [noise=true , address=77.90.31.12:4604     , piece_id=ZCE56GJ3B52UXNUIYQ6CFVR2CGCXDKZYXGY5CWY5235HAQHXRX6A, seg_pos={0 0}, upload=414055]: Uploading piece...
Jul 14 22:33:20.959715836 (517h21m47.886010533s) [noise=true , address=45.91.116.10:4111    , piece_id=LCZSVS34DIVD4LVRF73L6ATATHWMR4DVJQ4GN3HADJSZTPLCFBTA, seg_pos={0 0}, upload=414055]: Uploading piece...
Jul 14 22:33:20.960115490 (517h21m47.886410182s) [noise=true , address=45.91.95.2:4005      , piece_id=XTY4LNMWW23XE7HEZ6GB2WOS37K3ZAOMY2EVCHIQLDB6IQDMWZ6A, seg_pos={0 0}, upload=414055]: Uploading piece...
Jul 14 22:33:20.960349946 (517h21m47.886644642s) [noise=true , address=77.90.31.14:4694     , piece_id=GXYYM54EJCQQHN7UNLTOP52UVHPQ2E7I36ETL64JMQWH6FT5KEGA, seg_pos={0 0}, upload=414055]: Uploading piece...
Jul 14 22:33:20.960857778 (517h21m47.88715247s) [noise=true , address=104.36.82.34:4562    , piece_id=HZ7BIXYA7TOMLS3ANW4SSS5WTPOYTRF6JOKO26KJR6TMBSJJPA4Q, seg_pos={0 0}, upload=414055]: Uploading piece...
Jul 14 22:33:21.054689329 (517h21m47.980984015s) [upload=414055]: Next returned nil segment
Jul 14 22:33:21.054703519 (517h21m47.980998192s) [upload=414055]: Waiting for error group managing segments...
Jul 14 22:33:21.067731227 (517h21m47.994025919s) [noise=true , address=104.36.82.34:4562    , piece_id=HZ7BIXYA7TOMLS3ANW4SSS5WTPOYTRF6JOKO26KJR6TMBSJJPA4Q, seg_pos={0 0}, upload=414055]: Done uploading piece. err: <nil>
Jul 14 22:33:21.068562910 (517h21m47.994857586s) [noise=true , address=77.90.31.18:4810     , piece_id=L2G6QQTX3FVYCW7Z7FMLM6SAA7XZI64BRADRPCOUBLBRVL223KWQ, seg_pos={0 0}, upload=414055]: Done uploading piece. err: <nil>
Jul 14 22:33:21.070198720 (517h21m47.996493396s) [noise=true , address=45.91.116.10:4111    , piece_id=LCZSVS34DIVD4LVRF73L6ATATHWMR4DVJQ4GN3HADJSZTPLCFBTA, seg_pos={0 0}, upload=414055]: Done uploading piece. err: <nil>
Jul 14 22:33:21.071365233 (517h21m47.997659906s) [noise=true , address=45.91.116.14:4215    , piece_id=JB6J36RX7ZLVP4SBF3ZOJSXEZWILQXZOFUQON5WIWMRIAFDZYARA, seg_pos={0 0}, upload=414055]: Done uploading piece. err: <nil>
Jul 14 22:33:21.071792066 (517h21m47.998086738s) [noise=true , address=45.91.116.6:4005     , piece_id=KZTESMI5CMWF5U7S6PXTUK5Z4HSVSXXA6CJ2VAVL5SLK2IVJG3DQ, seg_pos={0 0}, upload=414055]: Done uploading piece. err: <nil>
Jul 14 22:33:21.074850939 (517h21m48.001145615s) [noise=true , address=77.90.31.11:4556     , piece_id=4WXTBJSV6F4WCYX3ETHYA7LKJEHEAFNNOD7E5X6GY65ZYUYEAC3A, seg_pos={0 0}, upload=414055]: Done uploading piece. err: <nil>
Jul 14 22:33:21.075297712 (517h21m48.001592384s) [noise=true , address=45.91.116.10:4098    , piece_id=QAD3V2YB62ZWM7NR5YTZD5YBHIZONG64A27OM6C7WS6R6XE5IAKA, seg_pos={0 0}, upload=414055]: Done uploading piece. err: <nil>
Jul 14 22:33:21.076301394 (517h21m48.002596084s) [noise=true , address=104.36.82.17:4334    , piece_id=6QFH4V2XP5HENEXYSPBLCGWESKAO2M7S243GVWQCSGYYANMDC5BQ, seg_pos={0 0}, upload=414055]: Done uploading piece. err: <nil>
Jul 14 22:33:21.078308059 (517h21m48.00460275s) [noise=true , address=104.36.82.19:4432    , piece_id=ZNUQB26NCFHHETSPPLIX52BSR23AI2VO3J2BJA5O4FPUDRD6ILEA, seg_pos={0 0}, upload=414055]: Done uploading piece. err: <nil>
Jul 14 22:33:21.079380196 (517h21m48.005674875s) [noise=true , address=77.90.31.14:4694     , piece_id=GXYYM54EJCQQHN7UNLTOP52UVHPQ2E7I36ETL64JMQWH6FT5KEGA, seg_pos={0 0}, upload=414055]: Done uploading piece. err: <nil>
Jul 14 22:33:21.086892708 (517h21m48.013187388s) [noise=true , address=104.36.82.10:4101    , piece_id=6I325W4T6GYA7B6TXAEA33PGTNXQSWH5AB6MRRWPJM6O66LXTDAA, seg_pos={0 0}, upload=414055]: Done uploading piece. err: <nil>
Jul 14 22:33:21.096904024 (517h21m48.023198705s) [noise=true , address=77.90.31.19:4873     , piece_id=ALRBAJG2MS2FX2JQQORO5ZLLXGFQGMBIXLKRTQB3OKIICVAAQTKA, seg_pos={0 0}, upload=414055]: Done uploading piece. err: <nil>
Jul 14 22:33:21.097936100 (517h21m48.024230776s) [noise=true , address=77.90.31.12:4604     , piece_id=ZCE56GJ3B52UXNUIYQ6CFVR2CGCXDKZYXGY5CWY5235HAQHXRX6A, seg_pos={0 0}, upload=414055]: Done uploading piece. err: <nil>
Jul 14 22:33:21.107109151 (517h21m48.03340383s) [noise=true , address=77.90.31.15:4704     , piece_id=K7MIP54PNDQCEH6FKFEHFDYAYQ2ZWOHHARJ26JJYLD6YKJCRX64A, seg_pos={0 0}, upload=414055]: Done uploading piece. err: <nil>
Jul 14 22:33:21.107140287 (517h21m48.033434963s) [seg_pos={0 0}, upload=414055]: Segment reached optimal threshold of 14 pieces.
Jul 14 22:33:21.107798713 (517h21m48.034093407s) [noise=true , address=45.91.95.2:4005      , piece_id=XTY4LNMWW23XE7HEZ6GB2WOS37K3ZAOMY2EVCHIQLDB6IQDMWZ6A, seg_pos={0 0}, upload=414055]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bxFpBmz99gV4rL6bMtnHuthK2Xg4EusbfMAWF37RTRtPkqhQS): protocol: context canceled
Jul 14 22:33:21.107904677 (517h21m48.034199363s) [noise=true , address=104.36.82.51:4074    , piece_id=HOC7FQM2ROFQMQOOQMXPD5HVLON2C6U2UVSS5HMLGRIWJIIO3MUQ, seg_pos={0 0}, upload=414055]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1DBQpuE1EC1ENCy67We5QhMT6zsa2AbeMDSv8hmQLjokGRvoxc): protocol: write tcp 159.223.143.39:44794->104.36.82.51:4074: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 14 22:33:21.107928716 (517h21m48.034223405s) [noise=true , address=45.91.95.3:4028      , piece_id=WPAZ53X2MORZF3ZAL42BEZW2HQ2FC7YEBX7YGGFNLOBALDZB7JUQ, seg_pos={0 0}, upload=414055]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Cn2vydXYjCSFYkLuBck57avzeCstHowf3cDQwtYn2RV2Wskh2): protocol: write tcp 159.223.143.39:33072->45.91.95.3:4028: use of closed network connection; write tcp 159.223.143.39:33072->45.91.95.3:4028: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:33072->45.91.95.3:4028: use of closed network connection
Jul 14 22:33:21.108008084 (517h21m48.034302763s) [noise=true , address=45.91.95.4:4001      , piece_id=C4KRMZGJX4PWVYSTRKXHCRK34AABH2ZLHVXXRKADJIOFOXX45LTA, seg_pos={0 0}, upload=414055]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PdimTdB4ysD3SQ8hKMAGAXkzzC71irLv6rW9FArf5qmrvZ5MY): protocol: read tcp 159.223.143.39:55404->45.91.95.4:4001: use of closed network connection; context canceled
Jul 14 22:33:21.108022527 (517h21m48.034317213s) [seg_pos={0 0}, upload=414055]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jul 14 22:33:21.108148869 (517h21m48.034443556s) [upload=414055]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 14 22:33:21.108154867 (517h21m48.034449552s) [upload=414055]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 14 22:33:21.175128505 (517h21m48.101423192s) [upload=414055]: Done uploading segments...
Jul 14 22:33:21.175148328 (517h21m48.101443004s) [upload=414055]: Upload finished. err: <nil>
Duration: 1m0.000604096s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)
antiblip:
2 out of 60 is failed (threshold=3) [.........................................................XX.]
duration:
19.510568875s
FAIL (2025-07-14 22:35:58 UTC)
orbiter warning: uplink: bucket: metaclient: context deadline exceeded
Duration: 19.510595773s core uplink us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)
antiblip:
3 out of 60 is failed (threshold=3) [........................................................XXX.]
duration:
4.575688519s
target:
https://us1.storj.io/login
FAIL (2025-07-14 22:35:58 UTC)
Get "https://us1.storj.io/login": dial tcp: lookup us1.storj.io on 127.0.0.1:53: read udp 127.0.0.1:56644->127.0.0.1:53: i/o timeout
Duration: 4.5757102s core satellite us1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)
antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............]
duration:
530.108063ms
target:
https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi
FAIL (2025-07-14 22:35:54 UTC)
orbiter warning: Get "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi": dial tcp: lookup www.storj-ipfs.com on 127.0.0.1:53: read udp 127.0.0.1:36811->127.0.0.1:53: i/o timeout
Duration: 530.148789ms edge ipfs us1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)
antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............]
duration:
255.762006ms
target:
https://www.storj-ipfs.com/api/v0/add
FAIL (2025-07-14 22:35:50 UTC)
orbiter warning: Post "https://www.storj-ipfs.com/api/v0/add": dial tcp: lookup www.storj-ipfs.com on 127.0.0.1:53: read udp 127.0.0.1:60626->127.0.0.1:53: i/o timeout
Duration: 255.777734ms edge ipfs us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip:
2 out of 60 is failed (threshold=3) [...........................................X...X............]
duration:
896.992597ms
FAIL (2025-07-14 22:35:50 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp: lookup auth.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:50870->127.0.0.1:53: i/o timeout
Duration: 897.025389ms core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://ap1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.ap1.satellite_login)
antiblip:
2 out of 60 is failed (threshold=3) [...............................................XX...........]
duration:
717.417906ms
target:
https://ap1.storj.io/login
FAIL (2025-07-14 22:35:47 UTC)
orbiter warning: Get "https://ap1.storj.io/login": dial tcp: lookup ap1.storj.io on 127.0.0.1:53: read udp 127.0.0.1:53169->127.0.0.1:53: i/o timeout
Duration: 717.443538ms core satellite ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)
antiblip:
4 out of 60 is failed (threshold=3) [............................................X............XXX]
duration:
30.000424815s
FAIL (2025-07-14 22:35:45 UTC)
uplink: bucket: metaclient: context deadline exceeded
Duration: 30.000449472s core uplink us1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)
antiblip:
2 out of 60 is failed (threshold=3) [...............................................X......X.....]
duration:
108.141029ms
target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318
FAIL (2025-07-14 22:35:42 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318": dial tcp: lookup uptime-checks.gateway.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:40390->127.0.0.1:53: i/o timeout
Duration: 108.165324ms edge gateway vhost bgp eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)
antiblip:
2 out of 60 is failed (threshold=3) [..........................................................XX]
duration:
30.000381969s
FAIL (2025-07-14 22:35:39 UTC)
orbiter warning: uplink: bucket: metaclient: context deadline exceeded
Duration: 30.000403482s core uplink us1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)
antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.]
duration:
1.202892794s
FAIL (2025-07-14 22:35:38 UTC)
orbiter warning: uplink: bucket: metaclient: context deadline exceeded
Duration: 1.202920223s core uplink eu1
checkers.HTTP{"GET" "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi", ExpectedContent: "Storj IPFS!"} failure (antiblip window 3/60): (ipfs.get)
antiblip:
1 out of 60 is failed (threshold=3) [..............................................X.............]
duration:
342.043035ms
target:
https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi
FAIL (2025-07-14 22:35:36 UTC)
orbiter warning: Get "https://www.storj-ipfs.com/ipfs/Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi": dial tcp: lookup www.storj-ipfs.com on 127.0.0.1:53: read udp 127.0.0.1:36811->127.0.0.1:53: i/o timeout
Duration: 342.100823ms edge ipfs us1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777"} failure (antiblip window 3/60): (authservice.ap1)
antiblip:
2 out of 60 is failed (threshold=3) [...............................................X.X..........]
duration:
636.528418ms
FAIL (2025-07-14 22:35:35 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp: lookup auth.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:50971->127.0.0.1:53: i/o timeout
Duration: 636.556183ms edge authservice http bgp ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)
antiblip:
3 out of 60 is failed (threshold=3) [.........................................................XXX]
duration:
30.001377672s
FAIL (2025-07-14 22:35:34 UTC)
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
ad
Jul 14 22:26:22.101876911 (517h14m49.028171602s) [upload=413961]: Done starting upload
Jul 14 22:26:22.101939016 (517h14m49.028233702s) [upload=413961]: Uploading segments...
Jul 14 22:26:22.102013964 (517h14m49.028308659s) [upload=413961]: Got next segment. Inline: true
Jul 14 22:26:22.102020563 (517h14m49.02831525s) [upload=413961]: Waiting for error group managing segments...
Jul 14 22:26:22.102066345 (517h14m49.028361031s) [upload=413961]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 14 22:26:22.102074758 (517h14m49.028369446s) [upload=413961]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Jul 14 22:26:22.102078292 (517h14m49.028372977s) [upload=413961]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 14 22:26:22.191721055 (517h14m49.118015741s) [upload=413961]: Done uploading segments...
Jul 14 22:26:22.191739109 (517h14m49.118033784s) [upload=413961]: Upload finished. err: <nil>
Jul 14 22:27:05.187541018 (517h15m32.113835711s) [upload=413970]: Starting upload
Jul 14 22:27:05.187627985 (517h15m32.113922676s) [upload=413970]: Done starting upload
Jul 14 22:27:05.187699511 (517h15m32.113994192s) [upload=413970]: Uploading segments...
Jul 14 22:27:05.187769044 (517h15m32.114063739s) [upload=413970]: Got next segment. Inline: true
Jul 14 22:27:05.187775252 (517h15m32.11406995s) [upload=413970]: Waiting for error group managing segments...
Jul 14 22:27:05.187827226 (517h15m32.114121924s) [upload=413970]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 14 22:27:05.187833426 (517h15m32.114128119s) [upload=413970]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Jul 14 22:27:05.187837312 (517h15m32.114132002s) [upload=413970]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 14 22:27:05.268547355 (517h15m32.194842047s) [upload=413970]: Done uploading segments...
Jul 14 22:27:05.268567502 (517h15m32.194862178s) [upload=413970]: Upload finished. err: <nil>
Jul 14 22:28:14.458618300 (517h16m41.384912975s) [upload=413987]: Starting upload
Jul 14 22:28:14.458745109 (517h16m41.385039799s) [upload=413987]: Done starting upload
Jul 14 22:28:14.458859196 (517h16m41.385153874s) [upload=413987]: Uploading segments...
Jul 14 22:28:14.458961146 (517h16m41.385255841s) [upload=413987]: Got next segment. Inline: true
Jul 14 22:28:14.458967913 (517h16m41.38526261s) [upload=413987]: Waiting for error group managing segments...
Jul 14 22:28:14.459061851 (517h16m41.385356542s) [upload=413987]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 14 22:28:14.459069295 (517h16m41.385363981s) [upload=413987]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Jul 14 22:28:14.459073440 (517h16m41.38536813s) [upload=413987]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 14 22:28:14.527793388 (517h16m41.454088085s) [upload=413987]: Done uploading segments...
Jul 14 22:28:14.527816718 (517h16m41.454111407s) [upload=413987]: Upload finished. err: <nil>
Jul 14 22:28:45.584768006 (517h17m12.511062681s) [upload=413995]: Starting upload
Jul 14 22:28:45.584835645 (517h17m12.511130321s) [upload=413995]: Done starting upload
Jul 14 22:28:45.584904793 (517h17m12.511199473s) [upload=413995]: Uploading segments...
Jul 14 22:28:45.584979419 (517h17m12.511274114s) [upload=413995]: Got next segment. Inline: true
Jul 14 22:28:45.584989608 (517h17m12.511284295s) [upload=413995]: Waiting for error group managing segments...
Jul 14 22:28:45.585050237 (517h17m12.511344923s) [upload=413995]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 14 22:28:45.585055965 (517h17m12.511350652s) [upload=413995]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Jul 14 22:28:45.585083964 (517h17m12.511378657s) [upload=413995]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 14 22:28:45.666542984 (517h17m12.59283768s) [upload=413995]: Done uploading segments...
Jul 14 22:28:45.666566621 (517h17m12.592861292s) [upload=413995]: Upload finished. err: <nil>
Jul 14 22:29:23.485493179 (517h17m50.41178787s) [upload=414002]: Starting upload
Jul 14 22:29:23.485582455 (517h17m50.411877145s) [upload=414002]: Done starting upload
Jul 14 22:29:23.485666072 (517h17m50.411960749s) [upload=414002]: Uploading segments...
Jul 14 22:29:23.485712218 (517h17m50.412006915s) [upload=414002]: Got next segment. Inline: true
Jul 14 22:29:23.485718174 (517h17m50.412012864s) [upload=414002]: Waiting for error group managing segments...
Jul 14 22:29:23.485767949 (517h17m50.412062638s) [upload=414002]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 14 22:29:23.485774222 (517h17m50.41206891s) [upload=414002]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Jul 14 22:29:23.485778499 (517h17m50.412073187s) [upload=414002]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 14 22:29:23.557452042 (517h17m50.483746729s) [upload=414002]: Done uploading segments...
Jul 14 22:29:23.557477187 (517h17m50.483771871s) [upload=414002]: Upload finished. err: <nil>
Jul 14 22:29:57.800795373 (517h18m24.727090065s) [upload=414009]: Starting upload
Jul 14 22:29:57.800873771 (517h18m24.727168461s) [upload=414009]: Done starting upload
Jul 14 22:29:57.800968200 (517h18m24.727262892s) [upload=414009]: Uploading segments...
Jul 14 22:29:57.801037227 (517h18m24.727331924s) [upload=414009]: Got next segment. Inline: true
Jul 14 22:29:57.801043900 (517h18m24.727338593s) [upload=414009]: Waiting for error group managing segments...
Jul 14 22:29:57.801223002 (517h18m24.727517698s) [upload=414009]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 14 22:29:57.801231410 (517h18m24.727526105s) [upload=414009]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Jul 14 22:29:57.801235144 (517h18m24.727529842s) [upload=414009]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 14 22:29:57.921237592 (517h18m24.847532271s) [upload=414009]: Done uploading segments...
Jul 14 22:29:57.921255617 (517h18m24.847550294s) [upload=414009]: Upload finished. err: <nil>
Jul 14 22:30:40.751239620 (517h19m7.677534312s) [upload=414020]: Starting upload
Jul 14 22:30:40.751356985 (517h19m7.677651677s) [upload=414020]: Done starting upload
Jul 14 22:30:40.751466075 (517h19m7.677760754s) [upload=414020]: Uploading segments...
Jul 14 22:30:40.751537099 (517h19m7.677831785s) [upload=414020]: Got next segment. Inline: true
Jul 14 22:30:40.751543083 (517h19m7.677837782s) [upload=414020]: Waiting for error group managing segments...
Jul 14 22:30:40.751595558 (517h19m7.677890244s) [upload=414020]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 14 22:30:40.751601057 (517h19m7.677909364s) [upload=414020]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Jul 14 22:30:40.751618636 (517h19m7.677913323s) [upload=414020]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 14 22:30:40.832783883 (517h19m7.759078561s) [upload=414020]: Done uploading segments...
Jul 14 22:30:40.832802237 (517h19m7.759096913s) [upload=414020]: Upload finished. err: <nil>
Jul 14 22:31:18.985603543 (517h19m45.91189822s) [upload=414026]: Starting upload
Jul 14 22:31:18.985676609 (517h19m45.911971298s) [upload=414026]: Done starting upload
Jul 14 22:31:18.985753552 (517h19m45.912048227s) [upload=414026]: Uploading segments...
Jul 14 22:31:18.985802665 (517h19m45.912097361s) [upload=414026]: Got next segment. Inline: true
Jul 14 22:31:18.985807556 (517h19m45.912102242s) [upload=414026]: Waiting for error group managing segments...
Jul 14 22:31:18.985848046 (517h19m45.912142722s) [upload=414026]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 14 22:31:18.985863675 (517h19m45.912158362s) [upload=414026]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Jul 14 22:31:18.985867346 (517h19m45.912162022s) [upload=414026]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 14 22:31:19.053410124 (517h19m45.979704813s) [upload=414026]: Done uploading segments...
Jul 14 22:31:19.053433703 (517h19m45.979728394s) [upload=414026]: Upload finished. err: <nil>
Jul 14 22:32:25.786471107 (517h20m52.712765794s) [upload=414043]: Starting upload
Jul 14 22:32:25.786529855 (517h20m52.712824533s) [upload=414043]: Done starting upload
Jul 14 22:32:25.786625449 (517h20m52.712920125s) [upload=414043]: Uploading segments...
Jul 14 22:32:25.786720270 (517h20m52.713014968s) [upload=414043]: Got next segment. Inline: true
Jul 14 22:32:25.786730995 (517h20m52.713025684s) [upload=414043]: Waiting for error group managing segments...
Jul 14 22:32:25.786797543 (517h20m52.713092232s) [upload=414043]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 14 22:32:25.786804665 (517h20m52.71309936s) [upload=414043]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Jul 14 22:32:25.786808867 (517h20m52.713103557s) [upload=414043]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 14 22:32:25.853631744 (517h20m52.77992643s) [upload=414043]: Done uploading segments...
Jul 14 22:32:25.853649513 (517h20m52.77994419s) [upload=414043]: Upload finished. err: <nil>
Jul 14 22:33:00.951507975 (517h21m27.877802664s) [upload=414050]: Starting upload
Jul 14 22:33:00.951580949 (517h21m27.877875638s) [upload=414050]: Done starting upload
Jul 14 22:33:00.951675541 (517h21m27.877970229s) [upload=414050]: Uploading segments...
Jul 14 22:33:00.951758272 (517h21m27.878052967s) [upload=414050]: Got next segment. Inline: true
Jul 14 22:33:00.951764061 (517h21m27.878058737s) [upload=414050]: Waiting for error group managing segments...
Jul 14 22:33:00.951823347 (517h21m27.878118023s) [upload=414050]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 14 22:33:00.951829639 (517h21m27.878124315s) [upload=414050]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Jul 14 22:33:00.951833041 (517h21m27.878127717s) [upload=414050]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 14 22:33:01.021815420 (517h21m27.948110103s) [upload=414050]: Done uploading segments...
Jul 14 22:33:01.021850031 (517h21m27.948144718s) [upload=414050]: Upload finished. err: <nil>
Duration: 30.001392941s core uplink us-select-1
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)
antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.]
duration:
251.548042ms
target:
https://eu1.storj.io/login
FAIL (2025-07-14 22:35:33 UTC)
orbiter warning: Get "https://eu1.storj.io/login": dial tcp: lookup eu1.storj.io on 127.0.0.1:53: read udp 127.0.0.1:33844->127.0.0.1:53: i/o timeout
Duration: 251.574487ms core satellite eu1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)
antiblip:
1 out of 60 is failed (threshold=3) [..........................................X.................]
duration:
2.551473489s
FAIL (2025-07-14 22:35:31 UTC)
orbiter warning: uplink: bucket: metaclient: context deadline exceeded
Duration: 2.551501069s core uplink ap1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777"} failure (antiblip window 3/60): (authservice.eu1)
antiblip:
2 out of 60 is failed (threshold=3) [.............................................X..X...........]
duration:
353.377844ms
FAIL (2025-07-14 22:35:31 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp: lookup auth.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:50971->127.0.0.1:53: i/o timeout
Duration: 353.400172ms edge authservice http bgp eu1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)
antiblip:
2 out of 60 is failed (threshold=3) [.............................................X........X.....]
duration:
289.784536ms
target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011
FAIL (2025-07-14 22:35:30 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011": dial tcp: lookup uptime-checks.gateway.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:40445->127.0.0.1:53: i/o timeout
Duration: 289.819516ms edge gateway vhost bgp ap1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)
antiblip:
2 out of 60 is failed (threshold=6) [..........................................X..............X..]
duration:
255.5635ms
target:
https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt
FAIL (2025-07-14 22:35:27 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": dial tcp: lookup link.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:48688->127.0.0.1:53: i/o timeout
Duration: 255.59272ms edge linksharing bgp ap1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.]
duration:
18.249620329s
FAIL (2025-07-14 22:35:25 UTC)
orbiter warning: uplink: bucket: metaclient: context deadline exceeded
Duration: 18.249640314s core uplink eu1
checkers.HTTP{"GET" "https://ap1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.ap1.satellite_login)
antiblip:
2 out of 60 is failed (threshold=3) [................................................XX..........]
duration:
727.782841ms
target:
https://ap1.storj.io/login
FAIL (2025-07-14 22:35:25 UTC)
orbiter warning: Get "https://ap1.storj.io/login": dial tcp: lookup ap1.storj.io on 127.0.0.1:53: read udp 127.0.0.1:53169->127.0.0.1:53: i/o timeout
Duration: 727.811589ms core satellite ap1
checkers.HTTP{"GET" "http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt", ExpectedStatus: 308, ExpectedRedirect: "https://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt"} failure (antiblip window 3/60): (http.us_select_1.linksharing)
antiblip:
2 out of 60 is failed (threshold=3) [...........................................X............X...]
duration:
55.859453ms
target:
http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt
FAIL (2025-07-14 22:35:23 UTC)
orbiter warning: Get "http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt": dial tcp: lookup link.us-select-1.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:48860->127.0.0.1:53: i/o timeout
Duration: 55.884299ms edge linksharing us-select-1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)
antiblip:
2 out of 60 is failed (threshold=6) [.............................................X............X.]
duration:
115.701351ms
target:
https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt
FAIL (2025-07-14 22:35:22 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": dial tcp: lookup link.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:48688->127.0.0.1:53: i/o timeout
Duration: 115.724502ms edge linksharing bgp eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)
antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X]
duration:
30.000820539s
FAIL (2025-07-14 22:35:17 UTC)
orbiter warning: uplink: bucket: metaclient: context deadline exceeded
Duration: 30.000841733s core uplink eu1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)
antiblip:
2 out of 60 is failed (threshold=3) [............................................XX..............]
duration:
156.82716ms
FAIL (2025-07-14 22:35:17 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp: lookup auth.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:53370->127.0.0.1:53: i/o timeout
Duration: 156.871155ms edge authservice http bgp us1
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)
antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X]
duration:
8.839937525s
target:
https://eu1.storj.io/login
FAIL (2025-07-14 22:35:07 UTC)
orbiter warning: Get "https://eu1.storj.io/login": dial tcp: lookup eu1.storj.io on 127.0.0.1:53: read udp 127.0.0.1:33844->127.0.0.1:53: i/o timeout
Duration: 8.839964839s core satellite eu1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)
antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..]
duration:
413.986521ms
FAIL (2025-07-14 22:35:07 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 414.008987ms edge gateway us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X]
duration:
30.001008208s
FAIL (2025-07-14 22:35:06 UTC)
orbiter warning: uplink: bucket: metaclient: context deadline exceeded
Duration: 30.001033941s core uplink eu1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)
antiblip:
2 out of 60 is failed (threshold=6) [...........................................X..............X.]
duration:
784.550725ms
target:
https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt
FAIL (2025-07-14 22:35:06 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": dial tcp: lookup link.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:48688->127.0.0.1:53: i/o timeout
Duration: 784.581496ms edge linksharing bgp ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
2 out of 60 is failed (threshold=3) [..........................................................XX]
duration:
30.001270544s
FAIL (2025-07-14 22:35:05 UTC)
orbiter warning: 
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
22:32:36.255563160 (517h21m3.181857848s) [noise=true , address=45.91.116.5:4297     , piece_id=4DXCPSF4YTD77FXCVOUGH4GWVJBS3JM3WYNERF6DS4UNEBOU2QWQ, seg_pos={0 0}, upload=414047]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF
Jul 14 22:32:36.255577193 (517h21m3.181871868s) [seg_pos={0 0}, upload=414047]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 14 22:32:36.255645270 (517h21m3.181939946s) [upload=414047]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 14 22:32:36.255650241 (517h21m3.181944931s) [upload=414047]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 14 22:32:36.314863469 (517h21m3.241158149s) [upload=414047]: Done uploading segments...
Jul 14 22:32:36.314882474 (517h21m3.241177151s) [upload=414047]: Upload finished. err: <nil>
Jul 14 22:33:11.378492041 (517h21m38.304786743s) [upload=414053]: Starting upload
Jul 14 22:33:11.378577342 (517h21m38.304872032s) [upload=414053]: Done starting upload
Jul 14 22:33:11.378744513 (517h21m38.305039193s) [upload=414053]: Uploading segments...
Jul 14 22:33:11.378953197 (517h21m38.305247877s) [upload=414053]: Got next segment. Inline: false
Jul 14 22:33:11.378994874 (517h21m38.30528956s) [upload=414053]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 14 22:33:11.379000311 (517h21m38.305294992s) [upload=414053]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 14 22:33:11.409521121 (517h21m38.335815814s) [seg_pos={0 0}, upload=414053]: Begin upload segment...
Jul 14 22:33:11.409580940 (517h21m38.335875637s) [seg_pos={0 0}, upload=414053]: Done begin upload segment.
Jul 14 22:33:11.409628148 (517h21m38.335922844s) [noise=true , address=104.36.82.16:4294    , piece_id=QLMMGAWGTTQZXFKV4XVD3YDR2T355TB556XQKWGSLHYTIAZIVDDQ, seg_pos={0 0}, upload=414053]: Uploading piece...
Jul 14 22:33:11.409651610 (517h21m38.335946308s) [noise=true , address=45.91.116.5:4299     , piece_id=5F57Y5UYILBHBF3DV7M5POSK6RFAL3Z7CFA24WMKBEUQAZAL7ODQ, seg_pos={0 0}, upload=414053]: Uploading piece...
Jul 14 22:33:11.410094293 (517h21m38.336388984s) [noise=true , address=45.91.116.9:4263     , piece_id=IO6KT3CIHLSEYQEMZ4DAX4JT4MP4OYARWASXXGV35Q7UHMLIIBOA, seg_pos={0 0}, upload=414053]: Uploading piece...
Jul 14 22:33:11.410493120 (517h21m38.33678781s) [noise=true , address=77.90.31.18:4806     , piece_id=WH2NYTMISPD6II4BSWZDNNPM3H4G2JXZMUCCYKG4GM5LGFAQ2WPQ, seg_pos={0 0}, upload=414053]: Uploading piece...
Jul 14 22:33:11.410954124 (517h21m38.337248825s) [noise=true , address=104.36.82.15:4257    , piece_id=WJRQPXCD4ZS73SBUUPSKGG3VSDLYTDQ7ZQE6B4YW2OIPLAHGIDJA, seg_pos={0 0}, upload=414053]: Uploading piece...
Jul 14 22:33:11.411449799 (517h21m38.337744496s) [noise=true , address=45.91.116.4:4939     , piece_id=ISKKXMSPSMOSP4FWOUKSZ6IMIJFPO33IJDDZ73VNPNSDZ7KQ7SEA, seg_pos={0 0}, upload=414053]: Uploading piece...
Jul 14 22:33:11.411485810 (517h21m38.337780506s) [noise=true , address=104.36.82.11:4113    , piece_id=AK7SBE2BGLIXBKRJGGFRVTRXZGKILEP364KUPGDUS5D7Y74VHJLA, seg_pos={0 0}, upload=414053]: Uploading piece...
Jul 14 22:33:11.411850807 (517h21m38.338145504s) [noise=true , address=45.91.116.12:4225    , piece_id=MDOSVRR4VTFU6A4NIU3HZFTVMJLYTQDKPI2OTVR5SBQ6WGMWBLWA, seg_pos={0 0}, upload=414053]: Uploading piece...
Jul 14 22:33:11.411890633 (517h21m38.338185327s) [noise=true , address=77.90.31.10:4518     , piece_id=KRK47EJXTN4I7TWP7KXWPNYVDUBPOUQGNOG5VN6HDBJ36WE26RHA, seg_pos={0 0}, upload=414053]: Uploading piece...
Jul 14 22:33:11.412220793 (517h21m38.338515486s) [noise=true , address=104.36.82.8:4972     , piece_id=E46WB6ZEOOK23SMTLGJQPGGKDUN3FKBXHSKOY7QLLY24YPPIQVLQ, seg_pos={0 0}, upload=414053]: Uploading piece...
Jul 14 22:33:11.412279103 (517h21m38.338573797s) [noise=true , address=104.36.82.16:4295    , piece_id=NA6OQP5GZJKGH7OMU7OJLTME4OC5LLJUPE3AWBRISBEP7TFSIZIQ, seg_pos={0 0}, upload=414053]: Uploading piece...
Jul 14 22:33:11.412647456 (517h21m38.338942154s) [noise=true , address=104.36.82.4:4804     , piece_id=XO52GRQZ3KWIAQYBDMJTT7D6ADQK35EP232Z4JERAMI6HBEVIXNQ, seg_pos={0 0}, upload=414053]: Uploading piece...
Jul 14 22:33:11.413168375 (517h21m38.33946307s) [noise=true , address=45.91.116.11:4118    , piece_id=VYLKVTNKUCCC6QYXJKGQVRWAJS7X4PEMUDJKJIIGYKHUQBADXQIA, seg_pos={0 0}, upload=414053]: Uploading piece...
Jul 14 22:33:11.413438644 (517h21m38.33973334s) [noise=true , address=45.91.116.14:4191    , piece_id=Q7A6WPWJ3F6D7ZBT27AARER3R3MDQLTUXU3GVBKMUJCUTIE2P3KA, seg_pos={0 0}, upload=414053]: Uploading piece...
Jul 14 22:33:11.413572133 (517h21m38.339866829s) [noise=true , address=77.90.31.17:4772     , piece_id=2JZOVZDHXZ4J5DZYX6EWQ4WSJ6NTMSVQN3WF3VD6AFFHC3J7AFJA, seg_pos={0 0}, upload=414053]: Uploading piece...
Jul 14 22:33:11.413947381 (517h21m38.340242075s) [noise=true , address=104.36.82.13:4186    , piece_id=P7HJGYDOOI4GVIB6AQ3AHCLLXKHSWSR3VDWBIMHTLEVOYRTEV7FQ, seg_pos={0 0}, upload=414053]: Uploading piece...
Jul 14 22:33:11.524370644 (517h21m38.450665324s) [upload=414053]: Next returned nil segment
Jul 14 22:33:11.524388534 (517h21m38.450683224s) [upload=414053]: Waiting for error group managing segments...
Jul 14 22:33:11.529388413 (517h21m38.455683093s) [noise=true , address=77.90.31.10:4518     , piece_id=KRK47EJXTN4I7TWP7KXWPNYVDUBPOUQGNOG5VN6HDBJ36WE26RHA, seg_pos={0 0}, upload=414053]: Done uploading piece. err: <nil>
Jul 14 22:33:11.529842618 (517h21m38.456137297s) [noise=true , address=77.90.31.18:4806     , piece_id=WH2NYTMISPD6II4BSWZDNNPM3H4G2JXZMUCCYKG4GM5LGFAQ2WPQ, seg_pos={0 0}, upload=414053]: Done uploading piece. err: <nil>
Jul 14 22:33:11.530343771 (517h21m38.456638451s) [noise=true , address=45.91.116.5:4299     , piece_id=5F57Y5UYILBHBF3DV7M5POSK6RFAL3Z7CFA24WMKBEUQAZAL7ODQ, seg_pos={0 0}, upload=414053]: Done uploading piece. err: <nil>
Jul 14 22:33:11.530866496 (517h21m38.457161176s) [noise=true , address=45.91.116.12:4225    , piece_id=MDOSVRR4VTFU6A4NIU3HZFTVMJLYTQDKPI2OTVR5SBQ6WGMWBLWA, seg_pos={0 0}, upload=414053]: Done uploading piece. err: <nil>
Jul 14 22:33:11.530939893 (517h21m38.45723458s) [noise=true , address=45.91.116.4:4939     , piece_id=ISKKXMSPSMOSP4FWOUKSZ6IMIJFPO33IJDDZ73VNPNSDZ7KQ7SEA, seg_pos={0 0}, upload=414053]: Done uploading piece. err: <nil>
Jul 14 22:33:11.531326057 (517h21m38.457620733s) [noise=true , address=45.91.116.14:4191    , piece_id=Q7A6WPWJ3F6D7ZBT27AARER3R3MDQLTUXU3GVBKMUJCUTIE2P3KA, seg_pos={0 0}, upload=414053]: Done uploading piece. err: <nil>
Jul 14 22:33:11.531369084 (517h21m38.457663773s) [noise=true , address=45.91.116.11:4118    , piece_id=VYLKVTNKUCCC6QYXJKGQVRWAJS7X4PEMUDJKJIIGYKHUQBADXQIA, seg_pos={0 0}, upload=414053]: Done uploading piece. err: <nil>
Jul 14 22:33:11.531856839 (517h21m38.458151512s) [noise=true , address=45.91.116.9:4263     , piece_id=IO6KT3CIHLSEYQEMZ4DAX4JT4MP4OYARWASXXGV35Q7UHMLIIBOA, seg_pos={0 0}, upload=414053]: Done uploading piece. err: <nil>
Jul 14 22:33:11.534154262 (517h21m38.460448936s) [noise=true , address=104.36.82.13:4186    , piece_id=P7HJGYDOOI4GVIB6AQ3AHCLLXKHSWSR3VDWBIMHTLEVOYRTEV7FQ, seg_pos={0 0}, upload=414053]: Done uploading piece. err: <nil>
Jul 14 22:33:11.534476285 (517h21m38.460770973s) [noise=true , address=104.36.82.16:4295    , piece_id=NA6OQP5GZJKGH7OMU7OJLTME4OC5LLJUPE3AWBRISBEP7TFSIZIQ, seg_pos={0 0}, upload=414053]: Done uploading piece. err: <nil>
Jul 14 22:33:11.536382666 (517h21m38.462677357s) [noise=true , address=104.36.82.8:4972     , piece_id=E46WB6ZEOOK23SMTLGJQPGGKDUN3FKBXHSKOY7QLLY24YPPIQVLQ, seg_pos={0 0}, upload=414053]: Done uploading piece. err: <nil>
Jul 14 22:33:11.536961196 (517h21m38.463255883s) [noise=true , address=104.36.82.15:4257    , piece_id=WJRQPXCD4ZS73SBUUPSKGG3VSDLYTDQ7ZQE6B4YW2OIPLAHGIDJA, seg_pos={0 0}, upload=414053]: Done uploading piece. err: <nil>
Jul 14 22:33:11.537568434 (517h21m38.463863128s) [noise=true , address=104.36.82.16:4294    , piece_id=QLMMGAWGTTQZXFKV4XVD3YDR2T355TB556XQKWGSLHYTIAZIVDDQ, seg_pos={0 0}, upload=414053]: Done uploading piece. err: <nil>
Jul 14 22:33:11.537582802 (517h21m38.463877487s) [seg_pos={0 0}, upload=414053]: Segment reached optimal threshold of 13 pieces.
Jul 14 22:33:11.537702706 (517h21m38.463997382s) [noise=true , address=104.36.82.4:4804     , piece_id=XO52GRQZ3KWIAQYBDMJTT7D6ADQK35EP232Z4JERAMI6HBEVIXNQ, seg_pos={0 0}, upload=414053]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g): protocol: expected piece hash; context canceled; EOF
Jul 14 22:33:11.537835025 (517h21m38.464129712s) [noise=true , address=104.36.82.11:4113    , piece_id=AK7SBE2BGLIXBKRJGGFRVTRXZGKILEP364KUPGDUS5D7Y74VHJLA, seg_pos={0 0}, upload=414053]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV): protocol: expected piece hash; context canceled; EOF
Jul 14 22:33:11.539014185 (517h21m38.465308871s) [noise=true , address=77.90.31.17:4772     , piece_id=2JZOVZDHXZ4J5DZYX6EWQ4WSJ6NTMSVQN3WF3VD6AFFHC3J7AFJA, seg_pos={0 0}, upload=414053]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE): protocol: expected piece hash; context canceled; EOF
Jul 14 22:33:11.539064812 (517h21m38.465359489s) [seg_pos={0 0}, upload=414053]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 14 22:33:11.539134433 (517h21m38.465429119s) [upload=414053]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 14 22:33:11.539139366 (517h21m38.465434043s) [upload=414053]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 14 22:33:11.600764137 (517h21m38.527058814s) [upload=414053]: Done uploading segments...
Jul 14 22:33:11.600783410 (517h21m38.527078087s) [upload=414053]: Upload finished. err: <nil>
Duration: 30.001291676s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)
antiblip:
2 out of 60 is failed (threshold=3) [..........................................................XX]
duration:
30.002070363s
FAIL (2025-07-14 22:35:04 UTC)
orbiter warning: 
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
ad
Jul 14 22:26:22.101876911 (517h14m49.028171602s) [upload=413961]: Done starting upload
Jul 14 22:26:22.101939016 (517h14m49.028233702s) [upload=413961]: Uploading segments...
Jul 14 22:26:22.102013964 (517h14m49.028308659s) [upload=413961]: Got next segment. Inline: true
Jul 14 22:26:22.102020563 (517h14m49.02831525s) [upload=413961]: Waiting for error group managing segments...
Jul 14 22:26:22.102066345 (517h14m49.028361031s) [upload=413961]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 14 22:26:22.102074758 (517h14m49.028369446s) [upload=413961]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Jul 14 22:26:22.102078292 (517h14m49.028372977s) [upload=413961]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 14 22:26:22.191721055 (517h14m49.118015741s) [upload=413961]: Done uploading segments...
Jul 14 22:26:22.191739109 (517h14m49.118033784s) [upload=413961]: Upload finished. err: <nil>
Jul 14 22:27:05.187541018 (517h15m32.113835711s) [upload=413970]: Starting upload
Jul 14 22:27:05.187627985 (517h15m32.113922676s) [upload=413970]: Done starting upload
Jul 14 22:27:05.187699511 (517h15m32.113994192s) [upload=413970]: Uploading segments...
Jul 14 22:27:05.187769044 (517h15m32.114063739s) [upload=413970]: Got next segment. Inline: true
Jul 14 22:27:05.187775252 (517h15m32.11406995s) [upload=413970]: Waiting for error group managing segments...
Jul 14 22:27:05.187827226 (517h15m32.114121924s) [upload=413970]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 14 22:27:05.187833426 (517h15m32.114128119s) [upload=413970]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Jul 14 22:27:05.187837312 (517h15m32.114132002s) [upload=413970]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 14 22:27:05.268547355 (517h15m32.194842047s) [upload=413970]: Done uploading segments...
Jul 14 22:27:05.268567502 (517h15m32.194862178s) [upload=413970]: Upload finished. err: <nil>
Jul 14 22:28:14.458618300 (517h16m41.384912975s) [upload=413987]: Starting upload
Jul 14 22:28:14.458745109 (517h16m41.385039799s) [upload=413987]: Done starting upload
Jul 14 22:28:14.458859196 (517h16m41.385153874s) [upload=413987]: Uploading segments...
Jul 14 22:28:14.458961146 (517h16m41.385255841s) [upload=413987]: Got next segment. Inline: true
Jul 14 22:28:14.458967913 (517h16m41.38526261s) [upload=413987]: Waiting for error group managing segments...
Jul 14 22:28:14.459061851 (517h16m41.385356542s) [upload=413987]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 14 22:28:14.459069295 (517h16m41.385363981s) [upload=413987]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Jul 14 22:28:14.459073440 (517h16m41.38536813s) [upload=413987]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 14 22:28:14.527793388 (517h16m41.454088085s) [upload=413987]: Done uploading segments...
Jul 14 22:28:14.527816718 (517h16m41.454111407s) [upload=413987]: Upload finished. err: <nil>
Jul 14 22:28:45.584768006 (517h17m12.511062681s) [upload=413995]: Starting upload
Jul 14 22:28:45.584835645 (517h17m12.511130321s) [upload=413995]: Done starting upload
Jul 14 22:28:45.584904793 (517h17m12.511199473s) [upload=413995]: Uploading segments...
Jul 14 22:28:45.584979419 (517h17m12.511274114s) [upload=413995]: Got next segment. Inline: true
Jul 14 22:28:45.584989608 (517h17m12.511284295s) [upload=413995]: Waiting for error group managing segments...
Jul 14 22:28:45.585050237 (517h17m12.511344923s) [upload=413995]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 14 22:28:45.585055965 (517h17m12.511350652s) [upload=413995]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Jul 14 22:28:45.585083964 (517h17m12.511378657s) [upload=413995]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 14 22:28:45.666542984 (517h17m12.59283768s) [upload=413995]: Done uploading segments...
Jul 14 22:28:45.666566621 (517h17m12.592861292s) [upload=413995]: Upload finished. err: <nil>
Jul 14 22:29:23.485493179 (517h17m50.41178787s) [upload=414002]: Starting upload
Jul 14 22:29:23.485582455 (517h17m50.411877145s) [upload=414002]: Done starting upload
Jul 14 22:29:23.485666072 (517h17m50.411960749s) [upload=414002]: Uploading segments...
Jul 14 22:29:23.485712218 (517h17m50.412006915s) [upload=414002]: Got next segment. Inline: true
Jul 14 22:29:23.485718174 (517h17m50.412012864s) [upload=414002]: Waiting for error group managing segments...
Jul 14 22:29:23.485767949 (517h17m50.412062638s) [upload=414002]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 14 22:29:23.485774222 (517h17m50.41206891s) [upload=414002]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Jul 14 22:29:23.485778499 (517h17m50.412073187s) [upload=414002]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 14 22:29:23.557452042 (517h17m50.483746729s) [upload=414002]: Done uploading segments...
Jul 14 22:29:23.557477187 (517h17m50.483771871s) [upload=414002]: Upload finished. err: <nil>
Jul 14 22:29:57.800795373 (517h18m24.727090065s) [upload=414009]: Starting upload
Jul 14 22:29:57.800873771 (517h18m24.727168461s) [upload=414009]: Done starting upload
Jul 14 22:29:57.800968200 (517h18m24.727262892s) [upload=414009]: Uploading segments...
Jul 14 22:29:57.801037227 (517h18m24.727331924s) [upload=414009]: Got next segment. Inline: true
Jul 14 22:29:57.801043900 (517h18m24.727338593s) [upload=414009]: Waiting for error group managing segments...
Jul 14 22:29:57.801223002 (517h18m24.727517698s) [upload=414009]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 14 22:29:57.801231410 (517h18m24.727526105s) [upload=414009]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Jul 14 22:29:57.801235144 (517h18m24.727529842s) [upload=414009]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 14 22:29:57.921237592 (517h18m24.847532271s) [upload=414009]: Done uploading segments...
Jul 14 22:29:57.921255617 (517h18m24.847550294s) [upload=414009]: Upload finished. err: <nil>
Jul 14 22:30:40.751239620 (517h19m7.677534312s) [upload=414020]: Starting upload
Jul 14 22:30:40.751356985 (517h19m7.677651677s) [upload=414020]: Done starting upload
Jul 14 22:30:40.751466075 (517h19m7.677760754s) [upload=414020]: Uploading segments...
Jul 14 22:30:40.751537099 (517h19m7.677831785s) [upload=414020]: Got next segment. Inline: true
Jul 14 22:30:40.751543083 (517h19m7.677837782s) [upload=414020]: Waiting for error group managing segments...
Jul 14 22:30:40.751595558 (517h19m7.677890244s) [upload=414020]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 14 22:30:40.751601057 (517h19m7.677909364s) [upload=414020]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Jul 14 22:30:40.751618636 (517h19m7.677913323s) [upload=414020]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 14 22:30:40.832783883 (517h19m7.759078561s) [upload=414020]: Done uploading segments...
Jul 14 22:30:40.832802237 (517h19m7.759096913s) [upload=414020]: Upload finished. err: <nil>
Jul 14 22:31:18.985603543 (517h19m45.91189822s) [upload=414026]: Starting upload
Jul 14 22:31:18.985676609 (517h19m45.911971298s) [upload=414026]: Done starting upload
Jul 14 22:31:18.985753552 (517h19m45.912048227s) [upload=414026]: Uploading segments...
Jul 14 22:31:18.985802665 (517h19m45.912097361s) [upload=414026]: Got next segment. Inline: true
Jul 14 22:31:18.985807556 (517h19m45.912102242s) [upload=414026]: Waiting for error group managing segments...
Jul 14 22:31:18.985848046 (517h19m45.912142722s) [upload=414026]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 14 22:31:18.985863675 (517h19m45.912158362s) [upload=414026]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Jul 14 22:31:18.985867346 (517h19m45.912162022s) [upload=414026]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 14 22:31:19.053410124 (517h19m45.979704813s) [upload=414026]: Done uploading segments...
Jul 14 22:31:19.053433703 (517h19m45.979728394s) [upload=414026]: Upload finished. err: <nil>
Jul 14 22:32:25.786471107 (517h20m52.712765794s) [upload=414043]: Starting upload
Jul 14 22:32:25.786529855 (517h20m52.712824533s) [upload=414043]: Done starting upload
Jul 14 22:32:25.786625449 (517h20m52.712920125s) [upload=414043]: Uploading segments...
Jul 14 22:32:25.786720270 (517h20m52.713014968s) [upload=414043]: Got next segment. Inline: true
Jul 14 22:32:25.786730995 (517h20m52.713025684s) [upload=414043]: Waiting for error group managing segments...
Jul 14 22:32:25.786797543 (517h20m52.713092232s) [upload=414043]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 14 22:32:25.786804665 (517h20m52.71309936s) [upload=414043]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Jul 14 22:32:25.786808867 (517h20m52.713103557s) [upload=414043]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 14 22:32:25.853631744 (517h20m52.77992643s) [upload=414043]: Done uploading segments...
Jul 14 22:32:25.853649513 (517h20m52.77994419s) [upload=414043]: Upload finished. err: <nil>
Jul 14 22:33:00.951507975 (517h21m27.877802664s) [upload=414050]: Starting upload
Jul 14 22:33:00.951580949 (517h21m27.877875638s) [upload=414050]: Done starting upload
Jul 14 22:33:00.951675541 (517h21m27.877970229s) [upload=414050]: Uploading segments...
Jul 14 22:33:00.951758272 (517h21m27.878052967s) [upload=414050]: Got next segment. Inline: true
Jul 14 22:33:00.951764061 (517h21m27.878058737s) [upload=414050]: Waiting for error group managing segments...
Jul 14 22:33:00.951823347 (517h21m27.878118023s) [upload=414050]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 14 22:33:00.951829639 (517h21m27.878124315s) [upload=414050]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Jul 14 22:33:00.951833041 (517h21m27.878127717s) [upload=414050]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 14 22:33:01.021815420 (517h21m27.948110103s) [upload=414050]: Done uploading segments...
Jul 14 22:33:01.021850031 (517h21m27.948144718s) [upload=414050]: Upload finished. err: <nil>
Duration: 30.002086364s core uplink us-select-1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)
antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........]
duration:
5.182895694s
FAIL (2025-07-14 22:35:04 UTC)
orbiter warning: uplink: failed to upload enough pieces (needed at least 49 but got 9)
Duration: 5.182921057s core uplink ap1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)
antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............]
duration:
318.929972ms
target:
https://www.storj-ipfs.com/api/v0/add
FAIL (2025-07-14 22:35:04 UTC)
orbiter warning: Post "https://www.storj-ipfs.com/api/v0/add": dial tcp: lookup www.storj-ipfs.com on 127.0.0.1:53: read udp 127.0.0.1:60626->127.0.0.1:53: i/o timeout
Duration: 318.954106ms edge ipfs us1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)
antiblip:
1 out of 60 is failed (threshold=3) [..............................................X.............]
duration:
55.232512ms
target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410
FAIL (2025-07-14 22:35:02 UTC)
orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": dial tcp: lookup uptime-checks-us-select.s3.us-select-1.storjapi.io on 127.0.0.1:53: read udp 127.0.0.1:58761->127.0.0.1:53: i/o timeout
Duration: 55.256728ms edge gateway vhost us-select-1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing)
antiblip:
2 out of 60 is failed (threshold=6) [.........................................X................X.]
duration:
38.763614ms
target:
https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt
FAIL (2025-07-14 22:35:01 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt": dial tcp: lookup link.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:48688->127.0.0.1:53: i/o timeout
Duration: 38.785716ms edge linksharing bgp us1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)
antiblip:
2 out of 60 is failed (threshold=3) [.............................................XX.............]
duration:
176.459016ms
FAIL (2025-07-14 22:35:00 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp: lookup auth.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:53370->127.0.0.1:53: i/o timeout
Duration: 176.489295ms edge authservice http bgp us1
checkers.HTTP{"GET" "http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt", ExpectedStatus: 308, ExpectedRedirect: "https://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt"} failure (antiblip window 3/60): (http.us_select_1.linksharing)
antiblip:
2 out of 60 is failed (threshold=3) [............................................X............X..]
duration:
19.521976ms
target:
http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt
FAIL (2025-07-14 22:34:59 UTC)
orbiter warning: Get "http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt": dial tcp: lookup link.us-select-1.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:48860->127.0.0.1:53: i/o timeout
Duration: 19.544733ms edge linksharing us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)
antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X]
duration:
1m0.000845113s
FAIL (2025-07-14 22:34:57 UTC)
orbiter warning: 
======ERROR======
uplink: bucket: metaclient: rpc: tcp connector failed: rpc: context deadline exceeded
======LOGS======
RequestItem_ObjectBegin
Jul 14 22:33:20.920268234 (517h21m47.84656293s) [upload=414055]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 14 22:33:20.956049835 (517h21m47.882344525s) [seg_pos={0 0}, upload=414055]: Begin upload segment...
Jul 14 22:33:20.956138063 (517h21m47.882432739s) [seg_pos={0 0}, upload=414055]: Done begin upload segment.
Jul 14 22:33:20.956148696 (517h21m47.88244339s) [noise=true , address=104.36.82.19:4432    , piece_id=ZNUQB26NCFHHETSPPLIX52BSR23AI2VO3J2BJA5O4FPUDRD6ILEA, seg_pos={0 0}, upload=414055]: Uploading piece...
Jul 14 22:33:20.956179183 (517h21m47.882473875s) [noise=true , address=45.91.95.4:4001      , piece_id=C4KRMZGJX4PWVYSTRKXHCRK34AABH2ZLHVXXRKADJIOFOXX45LTA, seg_pos={0 0}, upload=414055]: Uploading piece...
Jul 14 22:33:20.956739778 (517h21m47.883034473s) [noise=true , address=104.36.82.17:4334    , piece_id=6QFH4V2XP5HENEXYSPBLCGWESKAO2M7S243GVWQCSGYYANMDC5BQ, seg_pos={0 0}, upload=414055]: Uploading piece...
Jul 14 22:33:20.957112661 (517h21m47.883407356s) [noise=true , address=45.91.116.14:4215    , piece_id=JB6J36RX7ZLVP4SBF3ZOJSXEZWILQXZOFUQON5WIWMRIAFDZYARA, seg_pos={0 0}, upload=414055]: Uploading piece...
Jul 14 22:33:20.957321762 (517h21m47.883616454s) [noise=true , address=77.90.31.19:4873     , piece_id=ALRBAJG2MS2FX2JQQORO5ZLLXGFQGMBIXLKRTQB3OKIICVAAQTKA, seg_pos={0 0}, upload=414055]: Uploading piece...
Jul 14 22:33:20.957534791 (517h21m47.883829483s) [noise=true , address=104.36.82.51:4074    , piece_id=HOC7FQM2ROFQMQOOQMXPD5HVLON2C6U2UVSS5HMLGRIWJIIO3MUQ, seg_pos={0 0}, upload=414055]: Uploading piece...
Jul 14 22:33:20.957685297 (517h21m47.883979986s) [noise=true , address=45.91.116.10:4098    , piece_id=QAD3V2YB62ZWM7NR5YTZD5YBHIZONG64A27OM6C7WS6R6XE5IAKA, seg_pos={0 0}, upload=414055]: Uploading piece...
Jul 14 22:33:20.957863790 (517h21m47.884158483s) [noise=true , address=77.90.31.18:4810     , piece_id=L2G6QQTX3FVYCW7Z7FMLM6SAA7XZI64BRADRPCOUBLBRVL223KWQ, seg_pos={0 0}, upload=414055]: Uploading piece...
Jul 14 22:33:20.958078206 (517h21m47.884372898s) [noise=true , address=77.90.31.11:4556     , piece_id=4WXTBJSV6F4WCYX3ETHYA7LKJEHEAFNNOD7E5X6GY65ZYUYEAC3A, seg_pos={0 0}, upload=414055]: Uploading piece...
Jul 14 22:33:20.958206699 (517h21m47.884501391s) [noise=true , address=45.91.116.6:4005     , piece_id=KZTESMI5CMWF5U7S6PXTUK5Z4HSVSXXA6CJ2VAVL5SLK2IVJG3DQ, seg_pos={0 0}, upload=414055]: Uploading piece...
Jul 14 22:33:20.958449262 (517h21m47.884743949s) [noise=true , address=104.36.82.10:4101    , piece_id=6I325W4T6GYA7B6TXAEA33PGTNXQSWH5AB6MRRWPJM6O66LXTDAA, seg_pos={0 0}, upload=414055]: Uploading piece...
Jul 14 22:33:20.958598885 (517h21m47.884893577s) [noise=true , address=45.91.95.3:4028      , piece_id=WPAZ53X2MORZF3ZAL42BEZW2HQ2FC7YEBX7YGGFNLOBALDZB7JUQ, seg_pos={0 0}, upload=414055]: Uploading piece...
Jul 14 22:33:20.958933707 (517h21m47.885228398s) [noise=true , address=77.90.31.15:4704     , piece_id=K7MIP54PNDQCEH6FKFEHFDYAYQ2ZWOHHARJ26JJYLD6YKJCRX64A, seg_pos={0 0}, upload=414055]: Uploading piece...
Jul 14 22:33:20.959332655 (517h21m47.885627352s) [noise=true , address=77.90.31.12:4604     , piece_id=ZCE56GJ3B52UXNUIYQ6CFVR2CGCXDKZYXGY5CWY5235HAQHXRX6A, seg_pos={0 0}, upload=414055]: Uploading piece...
Jul 14 22:33:20.959715836 (517h21m47.886010533s) [noise=true , address=45.91.116.10:4111    , piece_id=LCZSVS34DIVD4LVRF73L6ATATHWMR4DVJQ4GN3HADJSZTPLCFBTA, seg_pos={0 0}, upload=414055]: Uploading piece...
Jul 14 22:33:20.960115490 (517h21m47.886410182s) [noise=true , address=45.91.95.2:4005      , piece_id=XTY4LNMWW23XE7HEZ6GB2WOS37K3ZAOMY2EVCHIQLDB6IQDMWZ6A, seg_pos={0 0}, upload=414055]: Uploading piece...
Jul 14 22:33:20.960349946 (517h21m47.886644642s) [noise=true , address=77.90.31.14:4694     , piece_id=GXYYM54EJCQQHN7UNLTOP52UVHPQ2E7I36ETL64JMQWH6FT5KEGA, seg_pos={0 0}, upload=414055]: Uploading piece...
Jul 14 22:33:20.960857778 (517h21m47.88715247s) [noise=true , address=104.36.82.34:4562    , piece_id=HZ7BIXYA7TOMLS3ANW4SSS5WTPOYTRF6JOKO26KJR6TMBSJJPA4Q, seg_pos={0 0}, upload=414055]: Uploading piece...
Jul 14 22:33:21.054689329 (517h21m47.980984015s) [upload=414055]: Next returned nil segment
Jul 14 22:33:21.054703519 (517h21m47.980998192s) [upload=414055]: Waiting for error group managing segments...
Jul 14 22:33:21.067731227 (517h21m47.994025919s) [noise=true , address=104.36.82.34:4562    , piece_id=HZ7BIXYA7TOMLS3ANW4SSS5WTPOYTRF6JOKO26KJR6TMBSJJPA4Q, seg_pos={0 0}, upload=414055]: Done uploading piece. err: <nil>
Jul 14 22:33:21.068562910 (517h21m47.994857586s) [noise=true , address=77.90.31.18:4810     , piece_id=L2G6QQTX3FVYCW7Z7FMLM6SAA7XZI64BRADRPCOUBLBRVL223KWQ, seg_pos={0 0}, upload=414055]: Done uploading piece. err: <nil>
Jul 14 22:33:21.070198720 (517h21m47.996493396s) [noise=true , address=45.91.116.10:4111    , piece_id=LCZSVS34DIVD4LVRF73L6ATATHWMR4DVJQ4GN3HADJSZTPLCFBTA, seg_pos={0 0}, upload=414055]: Done uploading piece. err: <nil>
Jul 14 22:33:21.071365233 (517h21m47.997659906s) [noise=true , address=45.91.116.14:4215    , piece_id=JB6J36RX7ZLVP4SBF3ZOJSXEZWILQXZOFUQON5WIWMRIAFDZYARA, seg_pos={0 0}, upload=414055]: Done uploading piece. err: <nil>
Jul 14 22:33:21.071792066 (517h21m47.998086738s) [noise=true , address=45.91.116.6:4005     , piece_id=KZTESMI5CMWF5U7S6PXTUK5Z4HSVSXXA6CJ2VAVL5SLK2IVJG3DQ, seg_pos={0 0}, upload=414055]: Done uploading piece. err: <nil>
Jul 14 22:33:21.074850939 (517h21m48.001145615s) [noise=true , address=77.90.31.11:4556     , piece_id=4WXTBJSV6F4WCYX3ETHYA7LKJEHEAFNNOD7E5X6GY65ZYUYEAC3A, seg_pos={0 0}, upload=414055]: Done uploading piece. err: <nil>
Jul 14 22:33:21.075297712 (517h21m48.001592384s) [noise=true , address=45.91.116.10:4098    , piece_id=QAD3V2YB62ZWM7NR5YTZD5YBHIZONG64A27OM6C7WS6R6XE5IAKA, seg_pos={0 0}, upload=414055]: Done uploading piece. err: <nil>
Jul 14 22:33:21.076301394 (517h21m48.002596084s) [noise=true , address=104.36.82.17:4334    , piece_id=6QFH4V2XP5HENEXYSPBLCGWESKAO2M7S243GVWQCSGYYANMDC5BQ, seg_pos={0 0}, upload=414055]: Done uploading piece. err: <nil>
Jul 14 22:33:21.078308059 (517h21m48.00460275s) [noise=true , address=104.36.82.19:4432    , piece_id=ZNUQB26NCFHHETSPPLIX52BSR23AI2VO3J2BJA5O4FPUDRD6ILEA, seg_pos={0 0}, upload=414055]: Done uploading piece. err: <nil>
Jul 14 22:33:21.079380196 (517h21m48.005674875s) [noise=true , address=77.90.31.14:4694     , piece_id=GXYYM54EJCQQHN7UNLTOP52UVHPQ2E7I36ETL64JMQWH6FT5KEGA, seg_pos={0 0}, upload=414055]: Done uploading piece. err: <nil>
Jul 14 22:33:21.086892708 (517h21m48.013187388s) [noise=true , address=104.36.82.10:4101    , piece_id=6I325W4T6GYA7B6TXAEA33PGTNXQSWH5AB6MRRWPJM6O66LXTDAA, seg_pos={0 0}, upload=414055]: Done uploading piece. err: <nil>
Jul 14 22:33:21.096904024 (517h21m48.023198705s) [noise=true , address=77.90.31.19:4873     , piece_id=ALRBAJG2MS2FX2JQQORO5ZLLXGFQGMBIXLKRTQB3OKIICVAAQTKA, seg_pos={0 0}, upload=414055]: Done uploading piece. err: <nil>
Jul 14 22:33:21.097936100 (517h21m48.024230776s) [noise=true , address=77.90.31.12:4604     , piece_id=ZCE56GJ3B52UXNUIYQ6CFVR2CGCXDKZYXGY5CWY5235HAQHXRX6A, seg_pos={0 0}, upload=414055]: Done uploading piece. err: <nil>
Jul 14 22:33:21.107109151 (517h21m48.03340383s) [noise=true , address=77.90.31.15:4704     , piece_id=K7MIP54PNDQCEH6FKFEHFDYAYQ2ZWOHHARJ26JJYLD6YKJCRX64A, seg_pos={0 0}, upload=414055]: Done uploading piece. err: <nil>
Jul 14 22:33:21.107140287 (517h21m48.033434963s) [seg_pos={0 0}, upload=414055]: Segment reached optimal threshold of 14 pieces.
Jul 14 22:33:21.107798713 (517h21m48.034093407s) [noise=true , address=45.91.95.2:4005      , piece_id=XTY4LNMWW23XE7HEZ6GB2WOS37K3ZAOMY2EVCHIQLDB6IQDMWZ6A, seg_pos={0 0}, upload=414055]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bxFpBmz99gV4rL6bMtnHuthK2Xg4EusbfMAWF37RTRtPkqhQS): protocol: context canceled
Jul 14 22:33:21.107904677 (517h21m48.034199363s) [noise=true , address=104.36.82.51:4074    , piece_id=HOC7FQM2ROFQMQOOQMXPD5HVLON2C6U2UVSS5HMLGRIWJIIO3MUQ, seg_pos={0 0}, upload=414055]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1DBQpuE1EC1ENCy67We5QhMT6zsa2AbeMDSv8hmQLjokGRvoxc): protocol: write tcp 159.223.143.39:44794->104.36.82.51:4074: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 14 22:33:21.107928716 (517h21m48.034223405s) [noise=true , address=45.91.95.3:4028      , piece_id=WPAZ53X2MORZF3ZAL42BEZW2HQ2FC7YEBX7YGGFNLOBALDZB7JUQ, seg_pos={0 0}, upload=414055]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Cn2vydXYjCSFYkLuBck57avzeCstHowf3cDQwtYn2RV2Wskh2): protocol: write tcp 159.223.143.39:33072->45.91.95.3:4028: use of closed network connection; write tcp 159.223.143.39:33072->45.91.95.3:4028: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:33072->45.91.95.3:4028: use of closed network connection
Jul 14 22:33:21.108008084 (517h21m48.034302763s) [noise=true , address=45.91.95.4:4001      , piece_id=C4KRMZGJX4PWVYSTRKXHCRK34AABH2ZLHVXXRKADJIOFOXX45LTA, seg_pos={0 0}, upload=414055]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PdimTdB4ysD3SQ8hKMAGAXkzzC71irLv6rW9FArf5qmrvZ5MY): protocol: read tcp 159.223.143.39:55404->45.91.95.4:4001: use of closed network connection; context canceled
Jul 14 22:33:21.108022527 (517h21m48.034317213s) [seg_pos={0 0}, upload=414055]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jul 14 22:33:21.108148869 (517h21m48.034443556s) [upload=414055]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 14 22:33:21.108154867 (517h21m48.034449552s) [upload=414055]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 14 22:33:21.175128505 (517h21m48.101423192s) [upload=414055]: Done uploading segments...
Jul 14 22:33:21.175148328 (517h21m48.101443004s) [upload=414055]: Upload finished. err: <nil>
Duration: 1m0.000868079s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip:
2 out of 60 is failed (threshold=3) [............................................X...X...........]
duration:
747.182979ms
FAIL (2025-07-14 22:34:56 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp: lookup auth.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:50870->127.0.0.1:53: i/o timeout
Duration: 747.209828ms core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)
antiblip:
3 out of 60 is failed (threshold=3) [.........................................................XXX]
duration:
4.871182942s
target:
https://us1.storj.io/login
FAIL (2025-07-14 22:34:52 UTC)
Get "https://us1.storj.io/login": dial tcp: lookup us1.storj.io on 127.0.0.1:53: read udp 127.0.0.1:56644->127.0.0.1:53: i/o timeout
Duration: 4.871207033s core satellite us1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)
antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................]
duration:
2.134766077s
FAIL (2025-07-14 22:34:51 UTC)
orbiter warning: uplink: bucket: metaclient: context deadline exceeded
Duration: 2.134808081s core uplink ap1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)
antiblip:
2 out of 60 is failed (threshold=3) [.............................................X........X.....]
duration:
32.883338ms
target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222
FAIL (2025-07-14 22:34:48 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": dial tcp: lookup uptime-checks.gateway.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:40390->127.0.0.1:53: i/o timeout
Duration: 32.907801ms edge gateway vhost bgp us1