Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [........XX..................................................] duration:
646.580897ms
FAIL (2025-07-15 01:38:58 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp: lookup auth.us-select-1.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:58822->127.0.0.1:53: i/o timeout
Duration: 646.636987ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.........X..................................................] duration:
294.810233ms
FAIL (2025-07-15 01:38:54 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp: lookup auth.us-select-1.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:42004->127.0.0.1:53: i/o timeout
Duration: 294.835223ms
edge
gateway
us-select-1
|
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)antiblip:
1 out of 60 is failed (threshold=3) [............X...............................................] duration:
76.186608ms
FAIL (2025-07-15 01:38:45 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp: lookup auth.us-select-1.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:58822->127.0.0.1:53: i/o timeout
Duration: 76.221604ms
edge
authservice
http
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
230.660147ms
FAIL (2025-07-15 01:38:39 UTC)
orbiter warning: uplink: bucket: metaclient: context deadline exceeded
Duration: 230.689036ms
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
570.429839ms
FAIL (2025-07-15 01:38:19 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF
Jul 15 00:50:55.285687709 (519h39m22.211982402s) [noise=true , address=104.36.82.6:4875 , piece_id=EI2GYFUMGM4RBVMZ4YX4XBHU2F4DGW7JU3WFO7ZJ3PB5TA5JEPAA, seg_pos={0 0}, upload=415855]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF
Jul 15 00:50:55.285704441 (519h39m22.211999129s) [seg_pos={0 0}, upload=415855]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 15 00:50:55.285782799 (519h39m22.212077495s) [upload=415855]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 15 00:50:55.285789426 (519h39m22.212084124s) [upload=415855]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 15 00:50:55.339105588 (519h39m22.265400284s) [upload=415855]: Done uploading segments...
Jul 15 00:50:55.339127659 (519h39m22.265422364s) [upload=415855]: Upload finished. err: <nil>
Jul 15 00:52:03.633999496 (519h40m30.560294172s) [upload=415871]: Starting upload
Jul 15 00:52:03.634082952 (519h40m30.560377645s) [upload=415871]: Done starting upload
Jul 15 00:52:03.634209535 (519h40m30.560504222s) [upload=415871]: Uploading segments...
Jul 15 00:52:03.634480377 (519h40m30.560775053s) [upload=415871]: Got next segment. Inline: false
Jul 15 00:52:03.634530708 (519h40m30.560825395s) [upload=415871]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 15 00:52:03.634536382 (519h40m30.560831069s) [upload=415871]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 15 00:52:03.695685280 (519h40m30.621979973s) [seg_pos={0 0}, upload=415871]: Begin upload segment...
Jul 15 00:52:03.695768486 (519h40m30.622063158s) [seg_pos={0 0}, upload=415871]: Done begin upload segment.
Jul 15 00:52:03.695803916 (519h40m30.622098609s) [noise=true , address=77.90.31.14:4663 , piece_id=FTJPZQIFAM7DTUAQPLMWWQKA6ECPZCHYVRM7CVQRYYZWGOZKPE2Q, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.696411043 (519h40m30.622705739s) [noise=true , address=104.36.82.2:4729 , piece_id=AJMUHTXD4T27L6QOQT2MESVEJ2P5S7AITD77NA5D5KHSWLUSLO6Q, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.696879328 (519h40m30.623174024s) [noise=true , address=104.36.82.7:4909 , piece_id=4LWX2WM4TEUJK7NLMIEN45CODLCYMX5QJ75DIIODXYKH5JEWUFJA, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.697539086 (519h40m30.623833786s) [noise=true , address=104.36.82.10:4078 , piece_id=OZJCT2NLKSVSJLZNTUZKP4FHNJPTZ3WAMKFLWH3YZEOLLDRJ3IOA, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.698495186 (519h40m30.624789876s) [noise=true , address=104.36.82.9:4006 , piece_id=BFYMJD5WM7LZRLRDOU42ICLG5GXJIKTHSCFTJZIDV5BXBXTUPWYQ, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.698964985 (519h40m30.625259674s) [noise=true , address=104.36.82.15:4258 , piece_id=LHV6F4ZT6OFPKGOVGBSE3KHNND2SF3WWM4XRRE7QGNLG22ZQ5KGQ, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.699380429 (519h40m30.625675118s) [noise=true , address=104.36.82.14:4222 , piece_id=JF7XOB4JLIJYZS2KDBVBATWQWRD4OS43IVFTSPJYVCCLAF2OWOYA, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.699832706 (519h40m30.626127395s) [noise=true , address=77.90.31.10:4518 , piece_id=BBO7ZS4WLRV2F3DSPQEG3VDH2BCPL6NHSVHVMIF7JDYZOU2PGPFQ, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.701385016 (519h40m30.627679726s) [noise=true , address=104.36.82.5:4839 , piece_id=6SLBVGQTJFRJP7QA2CVG2ARGYYFWLRBOCBN33O5WN4NUTRFT6DUA, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.701879105 (519h40m30.628173794s) [noise=true , address=104.36.82.6:4875 , piece_id=UXCSD7HDZCRJ5WW6235ZQAT2FICARKHZWI7A2YE3JVNWA5W7GSOQ, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.702867315 (519h40m30.629162012s) [noise=true , address=45.91.116.13:4155 , piece_id=7HXRG74ZSSX42CIXL6SVROLXE22DFC4BLPIXAQOOGOIGD6CPRYPA, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.703883641 (519h40m30.63017834s) [noise=true , address=104.36.82.4:4801 , piece_id=W4ZHXNHCSFTQ4PNBUE44FFD6BML36MXLADIK7W4EMUNWIJW2FZ3A, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.704552977 (519h40m30.630847673s) [noise=true , address=77.90.31.12:4590 , piece_id=W7ZVFJDIJNFOJ2ONAIL5FB6QIDW75FV23MTQYIDISNTI4HMWP72Q, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.705047441 (519h40m30.631342118s) [noise=true , address=45.91.116.10:4083 , piece_id=GBI3BQEAT7HCN3DP775SJ3FERFVDOXOTVCW6RZWKCP7P6T5K7DWQ, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.705519581 (519h40m30.631814267s) [noise=true , address=45.91.116.14:4189 , piece_id=Y52DGD267UU5OQKDFDZYYUL3DWIMFDI6TX533SOWBCZCLRMMTE2Q, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.705848528 (519h40m30.6321432s) [noise=true , address=77.90.31.15:4697 , piece_id=E3ZD7PJOQTOETE66TKCAMXM33JDTJUVTHUGGZ4EAN25DFFNCAUTA, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.827107120 (519h40m30.753401817s) [upload=415871]: Next returned nil segment
Jul 15 00:52:03.827125125 (519h40m30.753419825s) [upload=415871]: Waiting for error group managing segments...
Jul 15 00:52:03.831917682 (519h40m30.758212372s) [noise=true , address=45.91.116.10:4083 , piece_id=GBI3BQEAT7HCN3DP775SJ3FERFVDOXOTVCW6RZWKCP7P6T5K7DWQ, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.832572362 (519h40m30.758867042s) [noise=true , address=77.90.31.10:4518 , piece_id=BBO7ZS4WLRV2F3DSPQEG3VDH2BCPL6NHSVHVMIF7JDYZOU2PGPFQ, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.833108591 (519h40m30.759403279s) [noise=true , address=77.90.31.12:4590 , piece_id=W7ZVFJDIJNFOJ2ONAIL5FB6QIDW75FV23MTQYIDISNTI4HMWP72Q, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.833610214 (519h40m30.7599049s) [noise=true , address=77.90.31.14:4663 , piece_id=FTJPZQIFAM7DTUAQPLMWWQKA6ECPZCHYVRM7CVQRYYZWGOZKPE2Q, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.834676903 (519h40m30.760971596s) [noise=true , address=45.91.116.14:4189 , piece_id=Y52DGD267UU5OQKDFDZYYUL3DWIMFDI6TX533SOWBCZCLRMMTE2Q, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.839879741 (519h40m30.766174428s) [noise=true , address=104.36.82.14:4222 , piece_id=JF7XOB4JLIJYZS2KDBVBATWQWRD4OS43IVFTSPJYVCCLAF2OWOYA, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.840357933 (519h40m30.76665261s) [noise=true , address=104.36.82.7:4909 , piece_id=4LWX2WM4TEUJK7NLMIEN45CODLCYMX5QJ75DIIODXYKH5JEWUFJA, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.841075253 (519h40m30.767369934s) [noise=true , address=104.36.82.5:4839 , piece_id=6SLBVGQTJFRJP7QA2CVG2ARGYYFWLRBOCBN33O5WN4NUTRFT6DUA, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.841610950 (519h40m30.767905627s) [noise=true , address=104.36.82.9:4006 , piece_id=BFYMJD5WM7LZRLRDOU42ICLG5GXJIKTHSCFTJZIDV5BXBXTUPWYQ, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.842107860 (519h40m30.768402541s) [noise=true , address=104.36.82.15:4258 , piece_id=LHV6F4ZT6OFPKGOVGBSE3KHNND2SF3WWM4XRRE7QGNLG22ZQ5KGQ, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.842764021 (519h40m30.769058699s) [noise=true , address=77.90.31.15:4697 , piece_id=E3ZD7PJOQTOETE66TKCAMXM33JDTJUVTHUGGZ4EAN25DFFNCAUTA, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.843339192 (519h40m30.769633866s) [noise=true , address=104.36.82.2:4729 , piece_id=AJMUHTXD4T27L6QOQT2MESVEJ2P5S7AITD77NA5D5KHSWLUSLO6Q, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.843944968 (519h40m30.77023966s) [noise=true , address=104.36.82.6:4875 , piece_id=UXCSD7HDZCRJ5WW6235ZQAT2FICARKHZWI7A2YE3JVNWA5W7GSOQ, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.843958762 (519h40m30.770253441s) [seg_pos={0 0}, upload=415871]: Segment reached optimal threshold of 13 pieces.
Jul 15 00:52:03.844132370 (519h40m30.770427052s) [noise=true , address=104.36.82.4:4801 , piece_id=W4ZHXNHCSFTQ4PNBUE44FFD6BML36MXLADIK7W4EMUNWIJW2FZ3A, seg_pos={0 0}, upload=415871]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: expected piece hash; context canceled; EOF
Jul 15 00:52:03.844340698 (519h40m30.770635385s) [noise=true , address=45.91.116.13:4155 , piece_id=7HXRG74ZSSX42CIXL6SVROLXE22DFC4BLPIXAQOOGOIGD6CPRYPA, seg_pos={0 0}, upload=415871]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf): protocol: expected piece hash; context canceled; EOF
Jul 15 00:52:03.844694597 (519h40m30.770989276s) [noise=true , address=104.36.82.10:4078 , piece_id=OZJCT2NLKSVSJLZNTUZKP4FHNJPTZ3WAMKFLWH3YZEOLLDRJ3IOA, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.844732096 (519h40m30.77102683s) [seg_pos={0 0}, upload=415871]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 15 00:52:03.844815170 (519h40m30.771109852s) [upload=415871]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 15 00:52:03.844820406 (519h40m30.771115095s) [upload=415871]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 15 00:52:03.899279901 (519h40m30.825574576s) [upload=415871]: Done uploading segments...
Jul 15 00:52:03.899298008 (519h40m30.825592687s) [upload=415871]: Upload finished. err: <nil>
Duration: 570.450404ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................] duration:
277.228205ms
FAIL (2025-07-15 01:38:10 UTC)
orbiter warning: uplink: bucket: metaclient: context deadline exceeded
Duration: 277.260854ms
core
uplink
us1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..........X.................................................] duration:
271.703064ms
FAIL (2025-07-15 01:37:49 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp: lookup auth.us-select-1.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:42004->127.0.0.1:53: i/o timeout
Duration: 271.731805ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.........XX.................................................] duration:
502.950583ms
FAIL (2025-07-15 01:37:46 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp: lookup auth.us-select-1.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:58822->127.0.0.1:53: i/o timeout
Duration: 502.980006ms
edge
gateway
us-select-1
|
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)antiblip:
1 out of 60 is failed (threshold=3) [.............X..............................................] duration:
64.617267ms
FAIL (2025-07-15 01:37:36 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp: lookup auth.us-select-1.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:58822->127.0.0.1:53: i/o timeout
Duration: 64.64521ms
edge
authservice
http
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
474.188167ms
FAIL (2025-07-15 01:37:15 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF
Jul 15 00:50:55.285687709 (519h39m22.211982402s) [noise=true , address=104.36.82.6:4875 , piece_id=EI2GYFUMGM4RBVMZ4YX4XBHU2F4DGW7JU3WFO7ZJ3PB5TA5JEPAA, seg_pos={0 0}, upload=415855]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF
Jul 15 00:50:55.285704441 (519h39m22.211999129s) [seg_pos={0 0}, upload=415855]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 15 00:50:55.285782799 (519h39m22.212077495s) [upload=415855]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 15 00:50:55.285789426 (519h39m22.212084124s) [upload=415855]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 15 00:50:55.339105588 (519h39m22.265400284s) [upload=415855]: Done uploading segments...
Jul 15 00:50:55.339127659 (519h39m22.265422364s) [upload=415855]: Upload finished. err: <nil>
Jul 15 00:52:03.633999496 (519h40m30.560294172s) [upload=415871]: Starting upload
Jul 15 00:52:03.634082952 (519h40m30.560377645s) [upload=415871]: Done starting upload
Jul 15 00:52:03.634209535 (519h40m30.560504222s) [upload=415871]: Uploading segments...
Jul 15 00:52:03.634480377 (519h40m30.560775053s) [upload=415871]: Got next segment. Inline: false
Jul 15 00:52:03.634530708 (519h40m30.560825395s) [upload=415871]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 15 00:52:03.634536382 (519h40m30.560831069s) [upload=415871]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 15 00:52:03.695685280 (519h40m30.621979973s) [seg_pos={0 0}, upload=415871]: Begin upload segment...
Jul 15 00:52:03.695768486 (519h40m30.622063158s) [seg_pos={0 0}, upload=415871]: Done begin upload segment.
Jul 15 00:52:03.695803916 (519h40m30.622098609s) [noise=true , address=77.90.31.14:4663 , piece_id=FTJPZQIFAM7DTUAQPLMWWQKA6ECPZCHYVRM7CVQRYYZWGOZKPE2Q, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.696411043 (519h40m30.622705739s) [noise=true , address=104.36.82.2:4729 , piece_id=AJMUHTXD4T27L6QOQT2MESVEJ2P5S7AITD77NA5D5KHSWLUSLO6Q, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.696879328 (519h40m30.623174024s) [noise=true , address=104.36.82.7:4909 , piece_id=4LWX2WM4TEUJK7NLMIEN45CODLCYMX5QJ75DIIODXYKH5JEWUFJA, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.697539086 (519h40m30.623833786s) [noise=true , address=104.36.82.10:4078 , piece_id=OZJCT2NLKSVSJLZNTUZKP4FHNJPTZ3WAMKFLWH3YZEOLLDRJ3IOA, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.698495186 (519h40m30.624789876s) [noise=true , address=104.36.82.9:4006 , piece_id=BFYMJD5WM7LZRLRDOU42ICLG5GXJIKTHSCFTJZIDV5BXBXTUPWYQ, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.698964985 (519h40m30.625259674s) [noise=true , address=104.36.82.15:4258 , piece_id=LHV6F4ZT6OFPKGOVGBSE3KHNND2SF3WWM4XRRE7QGNLG22ZQ5KGQ, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.699380429 (519h40m30.625675118s) [noise=true , address=104.36.82.14:4222 , piece_id=JF7XOB4JLIJYZS2KDBVBATWQWRD4OS43IVFTSPJYVCCLAF2OWOYA, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.699832706 (519h40m30.626127395s) [noise=true , address=77.90.31.10:4518 , piece_id=BBO7ZS4WLRV2F3DSPQEG3VDH2BCPL6NHSVHVMIF7JDYZOU2PGPFQ, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.701385016 (519h40m30.627679726s) [noise=true , address=104.36.82.5:4839 , piece_id=6SLBVGQTJFRJP7QA2CVG2ARGYYFWLRBOCBN33O5WN4NUTRFT6DUA, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.701879105 (519h40m30.628173794s) [noise=true , address=104.36.82.6:4875 , piece_id=UXCSD7HDZCRJ5WW6235ZQAT2FICARKHZWI7A2YE3JVNWA5W7GSOQ, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.702867315 (519h40m30.629162012s) [noise=true , address=45.91.116.13:4155 , piece_id=7HXRG74ZSSX42CIXL6SVROLXE22DFC4BLPIXAQOOGOIGD6CPRYPA, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.703883641 (519h40m30.63017834s) [noise=true , address=104.36.82.4:4801 , piece_id=W4ZHXNHCSFTQ4PNBUE44FFD6BML36MXLADIK7W4EMUNWIJW2FZ3A, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.704552977 (519h40m30.630847673s) [noise=true , address=77.90.31.12:4590 , piece_id=W7ZVFJDIJNFOJ2ONAIL5FB6QIDW75FV23MTQYIDISNTI4HMWP72Q, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.705047441 (519h40m30.631342118s) [noise=true , address=45.91.116.10:4083 , piece_id=GBI3BQEAT7HCN3DP775SJ3FERFVDOXOTVCW6RZWKCP7P6T5K7DWQ, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.705519581 (519h40m30.631814267s) [noise=true , address=45.91.116.14:4189 , piece_id=Y52DGD267UU5OQKDFDZYYUL3DWIMFDI6TX533SOWBCZCLRMMTE2Q, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.705848528 (519h40m30.6321432s) [noise=true , address=77.90.31.15:4697 , piece_id=E3ZD7PJOQTOETE66TKCAMXM33JDTJUVTHUGGZ4EAN25DFFNCAUTA, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.827107120 (519h40m30.753401817s) [upload=415871]: Next returned nil segment
Jul 15 00:52:03.827125125 (519h40m30.753419825s) [upload=415871]: Waiting for error group managing segments...
Jul 15 00:52:03.831917682 (519h40m30.758212372s) [noise=true , address=45.91.116.10:4083 , piece_id=GBI3BQEAT7HCN3DP775SJ3FERFVDOXOTVCW6RZWKCP7P6T5K7DWQ, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.832572362 (519h40m30.758867042s) [noise=true , address=77.90.31.10:4518 , piece_id=BBO7ZS4WLRV2F3DSPQEG3VDH2BCPL6NHSVHVMIF7JDYZOU2PGPFQ, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.833108591 (519h40m30.759403279s) [noise=true , address=77.90.31.12:4590 , piece_id=W7ZVFJDIJNFOJ2ONAIL5FB6QIDW75FV23MTQYIDISNTI4HMWP72Q, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.833610214 (519h40m30.7599049s) [noise=true , address=77.90.31.14:4663 , piece_id=FTJPZQIFAM7DTUAQPLMWWQKA6ECPZCHYVRM7CVQRYYZWGOZKPE2Q, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.834676903 (519h40m30.760971596s) [noise=true , address=45.91.116.14:4189 , piece_id=Y52DGD267UU5OQKDFDZYYUL3DWIMFDI6TX533SOWBCZCLRMMTE2Q, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.839879741 (519h40m30.766174428s) [noise=true , address=104.36.82.14:4222 , piece_id=JF7XOB4JLIJYZS2KDBVBATWQWRD4OS43IVFTSPJYVCCLAF2OWOYA, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.840357933 (519h40m30.76665261s) [noise=true , address=104.36.82.7:4909 , piece_id=4LWX2WM4TEUJK7NLMIEN45CODLCYMX5QJ75DIIODXYKH5JEWUFJA, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.841075253 (519h40m30.767369934s) [noise=true , address=104.36.82.5:4839 , piece_id=6SLBVGQTJFRJP7QA2CVG2ARGYYFWLRBOCBN33O5WN4NUTRFT6DUA, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.841610950 (519h40m30.767905627s) [noise=true , address=104.36.82.9:4006 , piece_id=BFYMJD5WM7LZRLRDOU42ICLG5GXJIKTHSCFTJZIDV5BXBXTUPWYQ, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.842107860 (519h40m30.768402541s) [noise=true , address=104.36.82.15:4258 , piece_id=LHV6F4ZT6OFPKGOVGBSE3KHNND2SF3WWM4XRRE7QGNLG22ZQ5KGQ, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.842764021 (519h40m30.769058699s) [noise=true , address=77.90.31.15:4697 , piece_id=E3ZD7PJOQTOETE66TKCAMXM33JDTJUVTHUGGZ4EAN25DFFNCAUTA, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.843339192 (519h40m30.769633866s) [noise=true , address=104.36.82.2:4729 , piece_id=AJMUHTXD4T27L6QOQT2MESVEJ2P5S7AITD77NA5D5KHSWLUSLO6Q, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.843944968 (519h40m30.77023966s) [noise=true , address=104.36.82.6:4875 , piece_id=UXCSD7HDZCRJ5WW6235ZQAT2FICARKHZWI7A2YE3JVNWA5W7GSOQ, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.843958762 (519h40m30.770253441s) [seg_pos={0 0}, upload=415871]: Segment reached optimal threshold of 13 pieces.
Jul 15 00:52:03.844132370 (519h40m30.770427052s) [noise=true , address=104.36.82.4:4801 , piece_id=W4ZHXNHCSFTQ4PNBUE44FFD6BML36MXLADIK7W4EMUNWIJW2FZ3A, seg_pos={0 0}, upload=415871]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: expected piece hash; context canceled; EOF
Jul 15 00:52:03.844340698 (519h40m30.770635385s) [noise=true , address=45.91.116.13:4155 , piece_id=7HXRG74ZSSX42CIXL6SVROLXE22DFC4BLPIXAQOOGOIGD6CPRYPA, seg_pos={0 0}, upload=415871]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf): protocol: expected piece hash; context canceled; EOF
Jul 15 00:52:03.844694597 (519h40m30.770989276s) [noise=true , address=104.36.82.10:4078 , piece_id=OZJCT2NLKSVSJLZNTUZKP4FHNJPTZ3WAMKFLWH3YZEOLLDRJ3IOA, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.844732096 (519h40m30.77102683s) [seg_pos={0 0}, upload=415871]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 15 00:52:03.844815170 (519h40m30.771109852s) [upload=415871]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 15 00:52:03.844820406 (519h40m30.771115095s) [upload=415871]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 15 00:52:03.899279901 (519h40m30.825574576s) [upload=415871]: Done uploading segments...
Jul 15 00:52:03.899298008 (519h40m30.825592687s) [upload=415871]: Upload finished. err: <nil>
Duration: 474.240189ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
259.587212ms
FAIL (2025-07-15 01:37:12 UTC)
orbiter warning: uplink: bucket: metaclient: context deadline exceeded
Duration: 259.613767ms
core
uplink
us1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...........X................................................] duration:
297.468953ms
FAIL (2025-07-15 01:37:00 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp: lookup auth.us-select-1.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:42004->127.0.0.1:53: i/o timeout
Duration: 297.571834ms
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
448.585411ms
FAIL (2025-07-15 01:36:52 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF
Jul 15 00:50:55.285687709 (519h39m22.211982402s) [noise=true , address=104.36.82.6:4875 , piece_id=EI2GYFUMGM4RBVMZ4YX4XBHU2F4DGW7JU3WFO7ZJ3PB5TA5JEPAA, seg_pos={0 0}, upload=415855]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF
Jul 15 00:50:55.285704441 (519h39m22.211999129s) [seg_pos={0 0}, upload=415855]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 15 00:50:55.285782799 (519h39m22.212077495s) [upload=415855]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 15 00:50:55.285789426 (519h39m22.212084124s) [upload=415855]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 15 00:50:55.339105588 (519h39m22.265400284s) [upload=415855]: Done uploading segments...
Jul 15 00:50:55.339127659 (519h39m22.265422364s) [upload=415855]: Upload finished. err: <nil>
Jul 15 00:52:03.633999496 (519h40m30.560294172s) [upload=415871]: Starting upload
Jul 15 00:52:03.634082952 (519h40m30.560377645s) [upload=415871]: Done starting upload
Jul 15 00:52:03.634209535 (519h40m30.560504222s) [upload=415871]: Uploading segments...
Jul 15 00:52:03.634480377 (519h40m30.560775053s) [upload=415871]: Got next segment. Inline: false
Jul 15 00:52:03.634530708 (519h40m30.560825395s) [upload=415871]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 15 00:52:03.634536382 (519h40m30.560831069s) [upload=415871]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 15 00:52:03.695685280 (519h40m30.621979973s) [seg_pos={0 0}, upload=415871]: Begin upload segment...
Jul 15 00:52:03.695768486 (519h40m30.622063158s) [seg_pos={0 0}, upload=415871]: Done begin upload segment.
Jul 15 00:52:03.695803916 (519h40m30.622098609s) [noise=true , address=77.90.31.14:4663 , piece_id=FTJPZQIFAM7DTUAQPLMWWQKA6ECPZCHYVRM7CVQRYYZWGOZKPE2Q, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.696411043 (519h40m30.622705739s) [noise=true , address=104.36.82.2:4729 , piece_id=AJMUHTXD4T27L6QOQT2MESVEJ2P5S7AITD77NA5D5KHSWLUSLO6Q, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.696879328 (519h40m30.623174024s) [noise=true , address=104.36.82.7:4909 , piece_id=4LWX2WM4TEUJK7NLMIEN45CODLCYMX5QJ75DIIODXYKH5JEWUFJA, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.697539086 (519h40m30.623833786s) [noise=true , address=104.36.82.10:4078 , piece_id=OZJCT2NLKSVSJLZNTUZKP4FHNJPTZ3WAMKFLWH3YZEOLLDRJ3IOA, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.698495186 (519h40m30.624789876s) [noise=true , address=104.36.82.9:4006 , piece_id=BFYMJD5WM7LZRLRDOU42ICLG5GXJIKTHSCFTJZIDV5BXBXTUPWYQ, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.698964985 (519h40m30.625259674s) [noise=true , address=104.36.82.15:4258 , piece_id=LHV6F4ZT6OFPKGOVGBSE3KHNND2SF3WWM4XRRE7QGNLG22ZQ5KGQ, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.699380429 (519h40m30.625675118s) [noise=true , address=104.36.82.14:4222 , piece_id=JF7XOB4JLIJYZS2KDBVBATWQWRD4OS43IVFTSPJYVCCLAF2OWOYA, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.699832706 (519h40m30.626127395s) [noise=true , address=77.90.31.10:4518 , piece_id=BBO7ZS4WLRV2F3DSPQEG3VDH2BCPL6NHSVHVMIF7JDYZOU2PGPFQ, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.701385016 (519h40m30.627679726s) [noise=true , address=104.36.82.5:4839 , piece_id=6SLBVGQTJFRJP7QA2CVG2ARGYYFWLRBOCBN33O5WN4NUTRFT6DUA, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.701879105 (519h40m30.628173794s) [noise=true , address=104.36.82.6:4875 , piece_id=UXCSD7HDZCRJ5WW6235ZQAT2FICARKHZWI7A2YE3JVNWA5W7GSOQ, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.702867315 (519h40m30.629162012s) [noise=true , address=45.91.116.13:4155 , piece_id=7HXRG74ZSSX42CIXL6SVROLXE22DFC4BLPIXAQOOGOIGD6CPRYPA, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.703883641 (519h40m30.63017834s) [noise=true , address=104.36.82.4:4801 , piece_id=W4ZHXNHCSFTQ4PNBUE44FFD6BML36MXLADIK7W4EMUNWIJW2FZ3A, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.704552977 (519h40m30.630847673s) [noise=true , address=77.90.31.12:4590 , piece_id=W7ZVFJDIJNFOJ2ONAIL5FB6QIDW75FV23MTQYIDISNTI4HMWP72Q, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.705047441 (519h40m30.631342118s) [noise=true , address=45.91.116.10:4083 , piece_id=GBI3BQEAT7HCN3DP775SJ3FERFVDOXOTVCW6RZWKCP7P6T5K7DWQ, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.705519581 (519h40m30.631814267s) [noise=true , address=45.91.116.14:4189 , piece_id=Y52DGD267UU5OQKDFDZYYUL3DWIMFDI6TX533SOWBCZCLRMMTE2Q, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.705848528 (519h40m30.6321432s) [noise=true , address=77.90.31.15:4697 , piece_id=E3ZD7PJOQTOETE66TKCAMXM33JDTJUVTHUGGZ4EAN25DFFNCAUTA, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.827107120 (519h40m30.753401817s) [upload=415871]: Next returned nil segment
Jul 15 00:52:03.827125125 (519h40m30.753419825s) [upload=415871]: Waiting for error group managing segments...
Jul 15 00:52:03.831917682 (519h40m30.758212372s) [noise=true , address=45.91.116.10:4083 , piece_id=GBI3BQEAT7HCN3DP775SJ3FERFVDOXOTVCW6RZWKCP7P6T5K7DWQ, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.832572362 (519h40m30.758867042s) [noise=true , address=77.90.31.10:4518 , piece_id=BBO7ZS4WLRV2F3DSPQEG3VDH2BCPL6NHSVHVMIF7JDYZOU2PGPFQ, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.833108591 (519h40m30.759403279s) [noise=true , address=77.90.31.12:4590 , piece_id=W7ZVFJDIJNFOJ2ONAIL5FB6QIDW75FV23MTQYIDISNTI4HMWP72Q, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.833610214 (519h40m30.7599049s) [noise=true , address=77.90.31.14:4663 , piece_id=FTJPZQIFAM7DTUAQPLMWWQKA6ECPZCHYVRM7CVQRYYZWGOZKPE2Q, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.834676903 (519h40m30.760971596s) [noise=true , address=45.91.116.14:4189 , piece_id=Y52DGD267UU5OQKDFDZYYUL3DWIMFDI6TX533SOWBCZCLRMMTE2Q, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.839879741 (519h40m30.766174428s) [noise=true , address=104.36.82.14:4222 , piece_id=JF7XOB4JLIJYZS2KDBVBATWQWRD4OS43IVFTSPJYVCCLAF2OWOYA, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.840357933 (519h40m30.76665261s) [noise=true , address=104.36.82.7:4909 , piece_id=4LWX2WM4TEUJK7NLMIEN45CODLCYMX5QJ75DIIODXYKH5JEWUFJA, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.841075253 (519h40m30.767369934s) [noise=true , address=104.36.82.5:4839 , piece_id=6SLBVGQTJFRJP7QA2CVG2ARGYYFWLRBOCBN33O5WN4NUTRFT6DUA, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.841610950 (519h40m30.767905627s) [noise=true , address=104.36.82.9:4006 , piece_id=BFYMJD5WM7LZRLRDOU42ICLG5GXJIKTHSCFTJZIDV5BXBXTUPWYQ, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.842107860 (519h40m30.768402541s) [noise=true , address=104.36.82.15:4258 , piece_id=LHV6F4ZT6OFPKGOVGBSE3KHNND2SF3WWM4XRRE7QGNLG22ZQ5KGQ, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.842764021 (519h40m30.769058699s) [noise=true , address=77.90.31.15:4697 , piece_id=E3ZD7PJOQTOETE66TKCAMXM33JDTJUVTHUGGZ4EAN25DFFNCAUTA, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.843339192 (519h40m30.769633866s) [noise=true , address=104.36.82.2:4729 , piece_id=AJMUHTXD4T27L6QOQT2MESVEJ2P5S7AITD77NA5D5KHSWLUSLO6Q, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.843944968 (519h40m30.77023966s) [noise=true , address=104.36.82.6:4875 , piece_id=UXCSD7HDZCRJ5WW6235ZQAT2FICARKHZWI7A2YE3JVNWA5W7GSOQ, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.843958762 (519h40m30.770253441s) [seg_pos={0 0}, upload=415871]: Segment reached optimal threshold of 13 pieces.
Jul 15 00:52:03.844132370 (519h40m30.770427052s) [noise=true , address=104.36.82.4:4801 , piece_id=W4ZHXNHCSFTQ4PNBUE44FFD6BML36MXLADIK7W4EMUNWIJW2FZ3A, seg_pos={0 0}, upload=415871]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: expected piece hash; context canceled; EOF
Jul 15 00:52:03.844340698 (519h40m30.770635385s) [noise=true , address=45.91.116.13:4155 , piece_id=7HXRG74ZSSX42CIXL6SVROLXE22DFC4BLPIXAQOOGOIGD6CPRYPA, seg_pos={0 0}, upload=415871]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf): protocol: expected piece hash; context canceled; EOF
Jul 15 00:52:03.844694597 (519h40m30.770989276s) [noise=true , address=104.36.82.10:4078 , piece_id=OZJCT2NLKSVSJLZNTUZKP4FHNJPTZ3WAMKFLWH3YZEOLLDRJ3IOA, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.844732096 (519h40m30.77102683s) [seg_pos={0 0}, upload=415871]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 15 00:52:03.844815170 (519h40m30.771109852s) [upload=415871]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 15 00:52:03.844820406 (519h40m30.771115095s) [upload=415871]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 15 00:52:03.899279901 (519h40m30.825574576s) [upload=415871]: Done uploading segments...
Jul 15 00:52:03.899298008 (519h40m30.825592687s) [upload=415871]: Upload finished. err: <nil>
Duration: 448.61739ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................] duration:
229.571477ms
FAIL (2025-07-15 01:36:48 UTC)
orbiter warning: uplink: bucket: metaclient: context deadline exceeded
Duration: 229.599442ms
core
uplink
us1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [..........XX................................................] duration:
491.150197ms
FAIL (2025-07-15 01:36:48 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp: lookup auth.us-select-1.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:58822->127.0.0.1:53: i/o timeout
Duration: 491.181537ms
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................] duration:
475.928081ms
FAIL (2025-07-15 01:36:36 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF
Jul 15 00:50:55.285687709 (519h39m22.211982402s) [noise=true , address=104.36.82.6:4875 , piece_id=EI2GYFUMGM4RBVMZ4YX4XBHU2F4DGW7JU3WFO7ZJ3PB5TA5JEPAA, seg_pos={0 0}, upload=415855]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF
Jul 15 00:50:55.285704441 (519h39m22.211999129s) [seg_pos={0 0}, upload=415855]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 15 00:50:55.285782799 (519h39m22.212077495s) [upload=415855]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 15 00:50:55.285789426 (519h39m22.212084124s) [upload=415855]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 15 00:50:55.339105588 (519h39m22.265400284s) [upload=415855]: Done uploading segments...
Jul 15 00:50:55.339127659 (519h39m22.265422364s) [upload=415855]: Upload finished. err: <nil>
Jul 15 00:52:03.633999496 (519h40m30.560294172s) [upload=415871]: Starting upload
Jul 15 00:52:03.634082952 (519h40m30.560377645s) [upload=415871]: Done starting upload
Jul 15 00:52:03.634209535 (519h40m30.560504222s) [upload=415871]: Uploading segments...
Jul 15 00:52:03.634480377 (519h40m30.560775053s) [upload=415871]: Got next segment. Inline: false
Jul 15 00:52:03.634530708 (519h40m30.560825395s) [upload=415871]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 15 00:52:03.634536382 (519h40m30.560831069s) [upload=415871]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 15 00:52:03.695685280 (519h40m30.621979973s) [seg_pos={0 0}, upload=415871]: Begin upload segment...
Jul 15 00:52:03.695768486 (519h40m30.622063158s) [seg_pos={0 0}, upload=415871]: Done begin upload segment.
Jul 15 00:52:03.695803916 (519h40m30.622098609s) [noise=true , address=77.90.31.14:4663 , piece_id=FTJPZQIFAM7DTUAQPLMWWQKA6ECPZCHYVRM7CVQRYYZWGOZKPE2Q, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.696411043 (519h40m30.622705739s) [noise=true , address=104.36.82.2:4729 , piece_id=AJMUHTXD4T27L6QOQT2MESVEJ2P5S7AITD77NA5D5KHSWLUSLO6Q, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.696879328 (519h40m30.623174024s) [noise=true , address=104.36.82.7:4909 , piece_id=4LWX2WM4TEUJK7NLMIEN45CODLCYMX5QJ75DIIODXYKH5JEWUFJA, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.697539086 (519h40m30.623833786s) [noise=true , address=104.36.82.10:4078 , piece_id=OZJCT2NLKSVSJLZNTUZKP4FHNJPTZ3WAMKFLWH3YZEOLLDRJ3IOA, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.698495186 (519h40m30.624789876s) [noise=true , address=104.36.82.9:4006 , piece_id=BFYMJD5WM7LZRLRDOU42ICLG5GXJIKTHSCFTJZIDV5BXBXTUPWYQ, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.698964985 (519h40m30.625259674s) [noise=true , address=104.36.82.15:4258 , piece_id=LHV6F4ZT6OFPKGOVGBSE3KHNND2SF3WWM4XRRE7QGNLG22ZQ5KGQ, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.699380429 (519h40m30.625675118s) [noise=true , address=104.36.82.14:4222 , piece_id=JF7XOB4JLIJYZS2KDBVBATWQWRD4OS43IVFTSPJYVCCLAF2OWOYA, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.699832706 (519h40m30.626127395s) [noise=true , address=77.90.31.10:4518 , piece_id=BBO7ZS4WLRV2F3DSPQEG3VDH2BCPL6NHSVHVMIF7JDYZOU2PGPFQ, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.701385016 (519h40m30.627679726s) [noise=true , address=104.36.82.5:4839 , piece_id=6SLBVGQTJFRJP7QA2CVG2ARGYYFWLRBOCBN33O5WN4NUTRFT6DUA, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.701879105 (519h40m30.628173794s) [noise=true , address=104.36.82.6:4875 , piece_id=UXCSD7HDZCRJ5WW6235ZQAT2FICARKHZWI7A2YE3JVNWA5W7GSOQ, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.702867315 (519h40m30.629162012s) [noise=true , address=45.91.116.13:4155 , piece_id=7HXRG74ZSSX42CIXL6SVROLXE22DFC4BLPIXAQOOGOIGD6CPRYPA, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.703883641 (519h40m30.63017834s) [noise=true , address=104.36.82.4:4801 , piece_id=W4ZHXNHCSFTQ4PNBUE44FFD6BML36MXLADIK7W4EMUNWIJW2FZ3A, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.704552977 (519h40m30.630847673s) [noise=true , address=77.90.31.12:4590 , piece_id=W7ZVFJDIJNFOJ2ONAIL5FB6QIDW75FV23MTQYIDISNTI4HMWP72Q, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.705047441 (519h40m30.631342118s) [noise=true , address=45.91.116.10:4083 , piece_id=GBI3BQEAT7HCN3DP775SJ3FERFVDOXOTVCW6RZWKCP7P6T5K7DWQ, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.705519581 (519h40m30.631814267s) [noise=true , address=45.91.116.14:4189 , piece_id=Y52DGD267UU5OQKDFDZYYUL3DWIMFDI6TX533SOWBCZCLRMMTE2Q, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.705848528 (519h40m30.6321432s) [noise=true , address=77.90.31.15:4697 , piece_id=E3ZD7PJOQTOETE66TKCAMXM33JDTJUVTHUGGZ4EAN25DFFNCAUTA, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.827107120 (519h40m30.753401817s) [upload=415871]: Next returned nil segment
Jul 15 00:52:03.827125125 (519h40m30.753419825s) [upload=415871]: Waiting for error group managing segments...
Jul 15 00:52:03.831917682 (519h40m30.758212372s) [noise=true , address=45.91.116.10:4083 , piece_id=GBI3BQEAT7HCN3DP775SJ3FERFVDOXOTVCW6RZWKCP7P6T5K7DWQ, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.832572362 (519h40m30.758867042s) [noise=true , address=77.90.31.10:4518 , piece_id=BBO7ZS4WLRV2F3DSPQEG3VDH2BCPL6NHSVHVMIF7JDYZOU2PGPFQ, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.833108591 (519h40m30.759403279s) [noise=true , address=77.90.31.12:4590 , piece_id=W7ZVFJDIJNFOJ2ONAIL5FB6QIDW75FV23MTQYIDISNTI4HMWP72Q, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.833610214 (519h40m30.7599049s) [noise=true , address=77.90.31.14:4663 , piece_id=FTJPZQIFAM7DTUAQPLMWWQKA6ECPZCHYVRM7CVQRYYZWGOZKPE2Q, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.834676903 (519h40m30.760971596s) [noise=true , address=45.91.116.14:4189 , piece_id=Y52DGD267UU5OQKDFDZYYUL3DWIMFDI6TX533SOWBCZCLRMMTE2Q, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.839879741 (519h40m30.766174428s) [noise=true , address=104.36.82.14:4222 , piece_id=JF7XOB4JLIJYZS2KDBVBATWQWRD4OS43IVFTSPJYVCCLAF2OWOYA, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.840357933 (519h40m30.76665261s) [noise=true , address=104.36.82.7:4909 , piece_id=4LWX2WM4TEUJK7NLMIEN45CODLCYMX5QJ75DIIODXYKH5JEWUFJA, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.841075253 (519h40m30.767369934s) [noise=true , address=104.36.82.5:4839 , piece_id=6SLBVGQTJFRJP7QA2CVG2ARGYYFWLRBOCBN33O5WN4NUTRFT6DUA, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.841610950 (519h40m30.767905627s) [noise=true , address=104.36.82.9:4006 , piece_id=BFYMJD5WM7LZRLRDOU42ICLG5GXJIKTHSCFTJZIDV5BXBXTUPWYQ, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.842107860 (519h40m30.768402541s) [noise=true , address=104.36.82.15:4258 , piece_id=LHV6F4ZT6OFPKGOVGBSE3KHNND2SF3WWM4XRRE7QGNLG22ZQ5KGQ, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.842764021 (519h40m30.769058699s) [noise=true , address=77.90.31.15:4697 , piece_id=E3ZD7PJOQTOETE66TKCAMXM33JDTJUVTHUGGZ4EAN25DFFNCAUTA, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.843339192 (519h40m30.769633866s) [noise=true , address=104.36.82.2:4729 , piece_id=AJMUHTXD4T27L6QOQT2MESVEJ2P5S7AITD77NA5D5KHSWLUSLO6Q, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.843944968 (519h40m30.77023966s) [noise=true , address=104.36.82.6:4875 , piece_id=UXCSD7HDZCRJ5WW6235ZQAT2FICARKHZWI7A2YE3JVNWA5W7GSOQ, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.843958762 (519h40m30.770253441s) [seg_pos={0 0}, upload=415871]: Segment reached optimal threshold of 13 pieces.
Jul 15 00:52:03.844132370 (519h40m30.770427052s) [noise=true , address=104.36.82.4:4801 , piece_id=W4ZHXNHCSFTQ4PNBUE44FFD6BML36MXLADIK7W4EMUNWIJW2FZ3A, seg_pos={0 0}, upload=415871]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: expected piece hash; context canceled; EOF
Jul 15 00:52:03.844340698 (519h40m30.770635385s) [noise=true , address=45.91.116.13:4155 , piece_id=7HXRG74ZSSX42CIXL6SVROLXE22DFC4BLPIXAQOOGOIGD6CPRYPA, seg_pos={0 0}, upload=415871]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf): protocol: expected piece hash; context canceled; EOF
Jul 15 00:52:03.844694597 (519h40m30.770989276s) [noise=true , address=104.36.82.10:4078 , piece_id=OZJCT2NLKSVSJLZNTUZKP4FHNJPTZ3WAMKFLWH3YZEOLLDRJ3IOA, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.844732096 (519h40m30.77102683s) [seg_pos={0 0}, upload=415871]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 15 00:52:03.844815170 (519h40m30.771109852s) [upload=415871]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 15 00:52:03.844820406 (519h40m30.771115095s) [upload=415871]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 15 00:52:03.899279901 (519h40m30.825574576s) [upload=415871]: Done uploading segments...
Jul 15 00:52:03.899298008 (519h40m30.825592687s) [upload=415871]: Upload finished. err: <nil>
Duration: 475.962313ms
core
uplink
us-select-1
|
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)antiblip:
1 out of 60 is failed (threshold=3) [..............X.............................................] duration:
57.052807ms
FAIL (2025-07-15 01:36:27 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp: lookup auth.us-select-1.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:58822->127.0.0.1:53: i/o timeout
Duration: 57.082554ms
edge
authservice
http
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [............X...............................................] duration:
294.495534ms
FAIL (2025-07-15 01:36:25 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp: lookup auth.us-select-1.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:42004->127.0.0.1:53: i/o timeout
Duration: 294.525487ms
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [......X.....................................................] duration:
254.137556ms
FAIL (2025-07-15 01:36:09 UTC)
orbiter warning: uplink: bucket: metaclient: context deadline exceeded
Duration: 254.1711ms
core
uplink
us1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [...........XX...............................................] duration:
461.852551ms
FAIL (2025-07-15 01:36:07 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp: lookup auth.us-select-1.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:58822->127.0.0.1:53: i/o timeout
Duration: 462.263491ms
edge
gateway
us-select-1
|
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)antiblip:
1 out of 60 is failed (threshold=3) [...............X............................................] duration:
58.142265ms
FAIL (2025-07-15 01:36:06 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp: lookup auth.us-select-1.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:58822->127.0.0.1:53: i/o timeout
Duration: 58.172127ms
edge
authservice
http
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
470.227313ms
FAIL (2025-07-15 01:36:04 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF
Jul 15 00:50:55.285687709 (519h39m22.211982402s) [noise=true , address=104.36.82.6:4875 , piece_id=EI2GYFUMGM4RBVMZ4YX4XBHU2F4DGW7JU3WFO7ZJ3PB5TA5JEPAA, seg_pos={0 0}, upload=415855]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF
Jul 15 00:50:55.285704441 (519h39m22.211999129s) [seg_pos={0 0}, upload=415855]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 15 00:50:55.285782799 (519h39m22.212077495s) [upload=415855]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 15 00:50:55.285789426 (519h39m22.212084124s) [upload=415855]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 15 00:50:55.339105588 (519h39m22.265400284s) [upload=415855]: Done uploading segments...
Jul 15 00:50:55.339127659 (519h39m22.265422364s) [upload=415855]: Upload finished. err: <nil>
Jul 15 00:52:03.633999496 (519h40m30.560294172s) [upload=415871]: Starting upload
Jul 15 00:52:03.634082952 (519h40m30.560377645s) [upload=415871]: Done starting upload
Jul 15 00:52:03.634209535 (519h40m30.560504222s) [upload=415871]: Uploading segments...
Jul 15 00:52:03.634480377 (519h40m30.560775053s) [upload=415871]: Got next segment. Inline: false
Jul 15 00:52:03.634530708 (519h40m30.560825395s) [upload=415871]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 15 00:52:03.634536382 (519h40m30.560831069s) [upload=415871]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 15 00:52:03.695685280 (519h40m30.621979973s) [seg_pos={0 0}, upload=415871]: Begin upload segment...
Jul 15 00:52:03.695768486 (519h40m30.622063158s) [seg_pos={0 0}, upload=415871]: Done begin upload segment.
Jul 15 00:52:03.695803916 (519h40m30.622098609s) [noise=true , address=77.90.31.14:4663 , piece_id=FTJPZQIFAM7DTUAQPLMWWQKA6ECPZCHYVRM7CVQRYYZWGOZKPE2Q, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.696411043 (519h40m30.622705739s) [noise=true , address=104.36.82.2:4729 , piece_id=AJMUHTXD4T27L6QOQT2MESVEJ2P5S7AITD77NA5D5KHSWLUSLO6Q, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.696879328 (519h40m30.623174024s) [noise=true , address=104.36.82.7:4909 , piece_id=4LWX2WM4TEUJK7NLMIEN45CODLCYMX5QJ75DIIODXYKH5JEWUFJA, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.697539086 (519h40m30.623833786s) [noise=true , address=104.36.82.10:4078 , piece_id=OZJCT2NLKSVSJLZNTUZKP4FHNJPTZ3WAMKFLWH3YZEOLLDRJ3IOA, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.698495186 (519h40m30.624789876s) [noise=true , address=104.36.82.9:4006 , piece_id=BFYMJD5WM7LZRLRDOU42ICLG5GXJIKTHSCFTJZIDV5BXBXTUPWYQ, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.698964985 (519h40m30.625259674s) [noise=true , address=104.36.82.15:4258 , piece_id=LHV6F4ZT6OFPKGOVGBSE3KHNND2SF3WWM4XRRE7QGNLG22ZQ5KGQ, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.699380429 (519h40m30.625675118s) [noise=true , address=104.36.82.14:4222 , piece_id=JF7XOB4JLIJYZS2KDBVBATWQWRD4OS43IVFTSPJYVCCLAF2OWOYA, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.699832706 (519h40m30.626127395s) [noise=true , address=77.90.31.10:4518 , piece_id=BBO7ZS4WLRV2F3DSPQEG3VDH2BCPL6NHSVHVMIF7JDYZOU2PGPFQ, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.701385016 (519h40m30.627679726s) [noise=true , address=104.36.82.5:4839 , piece_id=6SLBVGQTJFRJP7QA2CVG2ARGYYFWLRBOCBN33O5WN4NUTRFT6DUA, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.701879105 (519h40m30.628173794s) [noise=true , address=104.36.82.6:4875 , piece_id=UXCSD7HDZCRJ5WW6235ZQAT2FICARKHZWI7A2YE3JVNWA5W7GSOQ, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.702867315 (519h40m30.629162012s) [noise=true , address=45.91.116.13:4155 , piece_id=7HXRG74ZSSX42CIXL6SVROLXE22DFC4BLPIXAQOOGOIGD6CPRYPA, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.703883641 (519h40m30.63017834s) [noise=true , address=104.36.82.4:4801 , piece_id=W4ZHXNHCSFTQ4PNBUE44FFD6BML36MXLADIK7W4EMUNWIJW2FZ3A, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.704552977 (519h40m30.630847673s) [noise=true , address=77.90.31.12:4590 , piece_id=W7ZVFJDIJNFOJ2ONAIL5FB6QIDW75FV23MTQYIDISNTI4HMWP72Q, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.705047441 (519h40m30.631342118s) [noise=true , address=45.91.116.10:4083 , piece_id=GBI3BQEAT7HCN3DP775SJ3FERFVDOXOTVCW6RZWKCP7P6T5K7DWQ, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.705519581 (519h40m30.631814267s) [noise=true , address=45.91.116.14:4189 , piece_id=Y52DGD267UU5OQKDFDZYYUL3DWIMFDI6TX533SOWBCZCLRMMTE2Q, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.705848528 (519h40m30.6321432s) [noise=true , address=77.90.31.15:4697 , piece_id=E3ZD7PJOQTOETE66TKCAMXM33JDTJUVTHUGGZ4EAN25DFFNCAUTA, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.827107120 (519h40m30.753401817s) [upload=415871]: Next returned nil segment
Jul 15 00:52:03.827125125 (519h40m30.753419825s) [upload=415871]: Waiting for error group managing segments...
Jul 15 00:52:03.831917682 (519h40m30.758212372s) [noise=true , address=45.91.116.10:4083 , piece_id=GBI3BQEAT7HCN3DP775SJ3FERFVDOXOTVCW6RZWKCP7P6T5K7DWQ, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.832572362 (519h40m30.758867042s) [noise=true , address=77.90.31.10:4518 , piece_id=BBO7ZS4WLRV2F3DSPQEG3VDH2BCPL6NHSVHVMIF7JDYZOU2PGPFQ, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.833108591 (519h40m30.759403279s) [noise=true , address=77.90.31.12:4590 , piece_id=W7ZVFJDIJNFOJ2ONAIL5FB6QIDW75FV23MTQYIDISNTI4HMWP72Q, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.833610214 (519h40m30.7599049s) [noise=true , address=77.90.31.14:4663 , piece_id=FTJPZQIFAM7DTUAQPLMWWQKA6ECPZCHYVRM7CVQRYYZWGOZKPE2Q, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.834676903 (519h40m30.760971596s) [noise=true , address=45.91.116.14:4189 , piece_id=Y52DGD267UU5OQKDFDZYYUL3DWIMFDI6TX533SOWBCZCLRMMTE2Q, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.839879741 (519h40m30.766174428s) [noise=true , address=104.36.82.14:4222 , piece_id=JF7XOB4JLIJYZS2KDBVBATWQWRD4OS43IVFTSPJYVCCLAF2OWOYA, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.840357933 (519h40m30.76665261s) [noise=true , address=104.36.82.7:4909 , piece_id=4LWX2WM4TEUJK7NLMIEN45CODLCYMX5QJ75DIIODXYKH5JEWUFJA, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.841075253 (519h40m30.767369934s) [noise=true , address=104.36.82.5:4839 , piece_id=6SLBVGQTJFRJP7QA2CVG2ARGYYFWLRBOCBN33O5WN4NUTRFT6DUA, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.841610950 (519h40m30.767905627s) [noise=true , address=104.36.82.9:4006 , piece_id=BFYMJD5WM7LZRLRDOU42ICLG5GXJIKTHSCFTJZIDV5BXBXTUPWYQ, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.842107860 (519h40m30.768402541s) [noise=true , address=104.36.82.15:4258 , piece_id=LHV6F4ZT6OFPKGOVGBSE3KHNND2SF3WWM4XRRE7QGNLG22ZQ5KGQ, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.842764021 (519h40m30.769058699s) [noise=true , address=77.90.31.15:4697 , piece_id=E3ZD7PJOQTOETE66TKCAMXM33JDTJUVTHUGGZ4EAN25DFFNCAUTA, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.843339192 (519h40m30.769633866s) [noise=true , address=104.36.82.2:4729 , piece_id=AJMUHTXD4T27L6QOQT2MESVEJ2P5S7AITD77NA5D5KHSWLUSLO6Q, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.843944968 (519h40m30.77023966s) [noise=true , address=104.36.82.6:4875 , piece_id=UXCSD7HDZCRJ5WW6235ZQAT2FICARKHZWI7A2YE3JVNWA5W7GSOQ, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.843958762 (519h40m30.770253441s) [seg_pos={0 0}, upload=415871]: Segment reached optimal threshold of 13 pieces.
Jul 15 00:52:03.844132370 (519h40m30.770427052s) [noise=true , address=104.36.82.4:4801 , piece_id=W4ZHXNHCSFTQ4PNBUE44FFD6BML36MXLADIK7W4EMUNWIJW2FZ3A, seg_pos={0 0}, upload=415871]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: expected piece hash; context canceled; EOF
Jul 15 00:52:03.844340698 (519h40m30.770635385s) [noise=true , address=45.91.116.13:4155 , piece_id=7HXRG74ZSSX42CIXL6SVROLXE22DFC4BLPIXAQOOGOIGD6CPRYPA, seg_pos={0 0}, upload=415871]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf): protocol: expected piece hash; context canceled; EOF
Jul 15 00:52:03.844694597 (519h40m30.770989276s) [noise=true , address=104.36.82.10:4078 , piece_id=OZJCT2NLKSVSJLZNTUZKP4FHNJPTZ3WAMKFLWH3YZEOLLDRJ3IOA, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.844732096 (519h40m30.77102683s) [seg_pos={0 0}, upload=415871]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 15 00:52:03.844815170 (519h40m30.771109852s) [upload=415871]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 15 00:52:03.844820406 (519h40m30.771115095s) [upload=415871]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 15 00:52:03.899279901 (519h40m30.825574576s) [upload=415871]: Done uploading segments...
Jul 15 00:52:03.899298008 (519h40m30.825592687s) [upload=415871]: Upload finished. err: <nil>
Duration: 470.26213ms
core
uplink
us-select-1
|
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)antiblip:
1 out of 60 is failed (threshold=3) [................X...........................................] duration:
60.892278ms
FAIL (2025-07-15 01:35:50 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp: lookup auth.us-select-1.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:58822->127.0.0.1:53: i/o timeout
Duration: 60.921822ms
edge
authservice
http
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.............X..............................................] duration:
254.980453ms
FAIL (2025-07-15 01:35:43 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp: lookup auth.us-select-1.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:42004->127.0.0.1:53: i/o timeout
Duration: 255.008082ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [............XX..............................................] duration:
463.285364ms
FAIL (2025-07-15 01:35:26 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp: lookup auth.us-select-1.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:58822->127.0.0.1:53: i/o timeout
Duration: 463.306868ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..............X.............................................] duration:
274.879204ms
FAIL (2025-07-15 01:35:15 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp: lookup auth.us-select-1.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:42004->127.0.0.1:53: i/o timeout
Duration: 274.91936ms
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................] duration:
563.059708ms
FAIL (2025-07-15 01:35:11 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF
Jul 15 00:50:55.285687709 (519h39m22.211982402s) [noise=true , address=104.36.82.6:4875 , piece_id=EI2GYFUMGM4RBVMZ4YX4XBHU2F4DGW7JU3WFO7ZJ3PB5TA5JEPAA, seg_pos={0 0}, upload=415855]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF
Jul 15 00:50:55.285704441 (519h39m22.211999129s) [seg_pos={0 0}, upload=415855]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 15 00:50:55.285782799 (519h39m22.212077495s) [upload=415855]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 15 00:50:55.285789426 (519h39m22.212084124s) [upload=415855]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 15 00:50:55.339105588 (519h39m22.265400284s) [upload=415855]: Done uploading segments...
Jul 15 00:50:55.339127659 (519h39m22.265422364s) [upload=415855]: Upload finished. err: <nil>
Jul 15 00:52:03.633999496 (519h40m30.560294172s) [upload=415871]: Starting upload
Jul 15 00:52:03.634082952 (519h40m30.560377645s) [upload=415871]: Done starting upload
Jul 15 00:52:03.634209535 (519h40m30.560504222s) [upload=415871]: Uploading segments...
Jul 15 00:52:03.634480377 (519h40m30.560775053s) [upload=415871]: Got next segment. Inline: false
Jul 15 00:52:03.634530708 (519h40m30.560825395s) [upload=415871]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 15 00:52:03.634536382 (519h40m30.560831069s) [upload=415871]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 15 00:52:03.695685280 (519h40m30.621979973s) [seg_pos={0 0}, upload=415871]: Begin upload segment...
Jul 15 00:52:03.695768486 (519h40m30.622063158s) [seg_pos={0 0}, upload=415871]: Done begin upload segment.
Jul 15 00:52:03.695803916 (519h40m30.622098609s) [noise=true , address=77.90.31.14:4663 , piece_id=FTJPZQIFAM7DTUAQPLMWWQKA6ECPZCHYVRM7CVQRYYZWGOZKPE2Q, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.696411043 (519h40m30.622705739s) [noise=true , address=104.36.82.2:4729 , piece_id=AJMUHTXD4T27L6QOQT2MESVEJ2P5S7AITD77NA5D5KHSWLUSLO6Q, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.696879328 (519h40m30.623174024s) [noise=true , address=104.36.82.7:4909 , piece_id=4LWX2WM4TEUJK7NLMIEN45CODLCYMX5QJ75DIIODXYKH5JEWUFJA, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.697539086 (519h40m30.623833786s) [noise=true , address=104.36.82.10:4078 , piece_id=OZJCT2NLKSVSJLZNTUZKP4FHNJPTZ3WAMKFLWH3YZEOLLDRJ3IOA, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.698495186 (519h40m30.624789876s) [noise=true , address=104.36.82.9:4006 , piece_id=BFYMJD5WM7LZRLRDOU42ICLG5GXJIKTHSCFTJZIDV5BXBXTUPWYQ, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.698964985 (519h40m30.625259674s) [noise=true , address=104.36.82.15:4258 , piece_id=LHV6F4ZT6OFPKGOVGBSE3KHNND2SF3WWM4XRRE7QGNLG22ZQ5KGQ, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.699380429 (519h40m30.625675118s) [noise=true , address=104.36.82.14:4222 , piece_id=JF7XOB4JLIJYZS2KDBVBATWQWRD4OS43IVFTSPJYVCCLAF2OWOYA, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.699832706 (519h40m30.626127395s) [noise=true , address=77.90.31.10:4518 , piece_id=BBO7ZS4WLRV2F3DSPQEG3VDH2BCPL6NHSVHVMIF7JDYZOU2PGPFQ, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.701385016 (519h40m30.627679726s) [noise=true , address=104.36.82.5:4839 , piece_id=6SLBVGQTJFRJP7QA2CVG2ARGYYFWLRBOCBN33O5WN4NUTRFT6DUA, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.701879105 (519h40m30.628173794s) [noise=true , address=104.36.82.6:4875 , piece_id=UXCSD7HDZCRJ5WW6235ZQAT2FICARKHZWI7A2YE3JVNWA5W7GSOQ, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.702867315 (519h40m30.629162012s) [noise=true , address=45.91.116.13:4155 , piece_id=7HXRG74ZSSX42CIXL6SVROLXE22DFC4BLPIXAQOOGOIGD6CPRYPA, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.703883641 (519h40m30.63017834s) [noise=true , address=104.36.82.4:4801 , piece_id=W4ZHXNHCSFTQ4PNBUE44FFD6BML36MXLADIK7W4EMUNWIJW2FZ3A, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.704552977 (519h40m30.630847673s) [noise=true , address=77.90.31.12:4590 , piece_id=W7ZVFJDIJNFOJ2ONAIL5FB6QIDW75FV23MTQYIDISNTI4HMWP72Q, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.705047441 (519h40m30.631342118s) [noise=true , address=45.91.116.10:4083 , piece_id=GBI3BQEAT7HCN3DP775SJ3FERFVDOXOTVCW6RZWKCP7P6T5K7DWQ, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.705519581 (519h40m30.631814267s) [noise=true , address=45.91.116.14:4189 , piece_id=Y52DGD267UU5OQKDFDZYYUL3DWIMFDI6TX533SOWBCZCLRMMTE2Q, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.705848528 (519h40m30.6321432s) [noise=true , address=77.90.31.15:4697 , piece_id=E3ZD7PJOQTOETE66TKCAMXM33JDTJUVTHUGGZ4EAN25DFFNCAUTA, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.827107120 (519h40m30.753401817s) [upload=415871]: Next returned nil segment
Jul 15 00:52:03.827125125 (519h40m30.753419825s) [upload=415871]: Waiting for error group managing segments...
Jul 15 00:52:03.831917682 (519h40m30.758212372s) [noise=true , address=45.91.116.10:4083 , piece_id=GBI3BQEAT7HCN3DP775SJ3FERFVDOXOTVCW6RZWKCP7P6T5K7DWQ, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.832572362 (519h40m30.758867042s) [noise=true , address=77.90.31.10:4518 , piece_id=BBO7ZS4WLRV2F3DSPQEG3VDH2BCPL6NHSVHVMIF7JDYZOU2PGPFQ, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.833108591 (519h40m30.759403279s) [noise=true , address=77.90.31.12:4590 , piece_id=W7ZVFJDIJNFOJ2ONAIL5FB6QIDW75FV23MTQYIDISNTI4HMWP72Q, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.833610214 (519h40m30.7599049s) [noise=true , address=77.90.31.14:4663 , piece_id=FTJPZQIFAM7DTUAQPLMWWQKA6ECPZCHYVRM7CVQRYYZWGOZKPE2Q, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.834676903 (519h40m30.760971596s) [noise=true , address=45.91.116.14:4189 , piece_id=Y52DGD267UU5OQKDFDZYYUL3DWIMFDI6TX533SOWBCZCLRMMTE2Q, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.839879741 (519h40m30.766174428s) [noise=true , address=104.36.82.14:4222 , piece_id=JF7XOB4JLIJYZS2KDBVBATWQWRD4OS43IVFTSPJYVCCLAF2OWOYA, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.840357933 (519h40m30.76665261s) [noise=true , address=104.36.82.7:4909 , piece_id=4LWX2WM4TEUJK7NLMIEN45CODLCYMX5QJ75DIIODXYKH5JEWUFJA, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.841075253 (519h40m30.767369934s) [noise=true , address=104.36.82.5:4839 , piece_id=6SLBVGQTJFRJP7QA2CVG2ARGYYFWLRBOCBN33O5WN4NUTRFT6DUA, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.841610950 (519h40m30.767905627s) [noise=true , address=104.36.82.9:4006 , piece_id=BFYMJD5WM7LZRLRDOU42ICLG5GXJIKTHSCFTJZIDV5BXBXTUPWYQ, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.842107860 (519h40m30.768402541s) [noise=true , address=104.36.82.15:4258 , piece_id=LHV6F4ZT6OFPKGOVGBSE3KHNND2SF3WWM4XRRE7QGNLG22ZQ5KGQ, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.842764021 (519h40m30.769058699s) [noise=true , address=77.90.31.15:4697 , piece_id=E3ZD7PJOQTOETE66TKCAMXM33JDTJUVTHUGGZ4EAN25DFFNCAUTA, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.843339192 (519h40m30.769633866s) [noise=true , address=104.36.82.2:4729 , piece_id=AJMUHTXD4T27L6QOQT2MESVEJ2P5S7AITD77NA5D5KHSWLUSLO6Q, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.843944968 (519h40m30.77023966s) [noise=true , address=104.36.82.6:4875 , piece_id=UXCSD7HDZCRJ5WW6235ZQAT2FICARKHZWI7A2YE3JVNWA5W7GSOQ, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.843958762 (519h40m30.770253441s) [seg_pos={0 0}, upload=415871]: Segment reached optimal threshold of 13 pieces.
Jul 15 00:52:03.844132370 (519h40m30.770427052s) [noise=true , address=104.36.82.4:4801 , piece_id=W4ZHXNHCSFTQ4PNBUE44FFD6BML36MXLADIK7W4EMUNWIJW2FZ3A, seg_pos={0 0}, upload=415871]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: expected piece hash; context canceled; EOF
Jul 15 00:52:03.844340698 (519h40m30.770635385s) [noise=true , address=45.91.116.13:4155 , piece_id=7HXRG74ZSSX42CIXL6SVROLXE22DFC4BLPIXAQOOGOIGD6CPRYPA, seg_pos={0 0}, upload=415871]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf): protocol: expected piece hash; context canceled; EOF
Jul 15 00:52:03.844694597 (519h40m30.770989276s) [noise=true , address=104.36.82.10:4078 , piece_id=OZJCT2NLKSVSJLZNTUZKP4FHNJPTZ3WAMKFLWH3YZEOLLDRJ3IOA, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.844732096 (519h40m30.77102683s) [seg_pos={0 0}, upload=415871]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 15 00:52:03.844815170 (519h40m30.771109852s) [upload=415871]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 15 00:52:03.844820406 (519h40m30.771115095s) [upload=415871]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 15 00:52:03.899279901 (519h40m30.825574576s) [upload=415871]: Done uploading segments...
Jul 15 00:52:03.899298008 (519h40m30.825592687s) [upload=415871]: Upload finished. err: <nil>
Duration: 563.092386ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...............X............................................] duration:
265.177547ms
FAIL (2025-07-15 01:34:57 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp: lookup auth.us-select-1.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:42004->127.0.0.1:53: i/o timeout
Duration: 265.217082ms
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.......X....................................................] duration:
254.227493ms
FAIL (2025-07-15 01:34:55 UTC)
orbiter warning: uplink: bucket: metaclient: context deadline exceeded
Duration: 254.25938ms
core
uplink
us1
|
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)antiblip:
2 out of 60 is failed (threshold=3) [X................X..........................................] duration:
58.268196ms
FAIL (2025-07-15 01:34:55 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp: lookup auth.us-select-1.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:58822->127.0.0.1:53: i/o timeout
Duration: 58.295419ms
edge
authservice
http
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.............XX.............................................] duration:
557.852788ms
FAIL (2025-07-15 01:34:37 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp: lookup auth.us-select-1.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:58822->127.0.0.1:53: i/o timeout
Duration: 557.884428ms
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......X.....................................................] duration:
449.339108ms
FAIL (2025-07-15 01:34:32 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF
Jul 15 00:50:55.285687709 (519h39m22.211982402s) [noise=true , address=104.36.82.6:4875 , piece_id=EI2GYFUMGM4RBVMZ4YX4XBHU2F4DGW7JU3WFO7ZJ3PB5TA5JEPAA, seg_pos={0 0}, upload=415855]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF
Jul 15 00:50:55.285704441 (519h39m22.211999129s) [seg_pos={0 0}, upload=415855]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 15 00:50:55.285782799 (519h39m22.212077495s) [upload=415855]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 15 00:50:55.285789426 (519h39m22.212084124s) [upload=415855]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 15 00:50:55.339105588 (519h39m22.265400284s) [upload=415855]: Done uploading segments...
Jul 15 00:50:55.339127659 (519h39m22.265422364s) [upload=415855]: Upload finished. err: <nil>
Jul 15 00:52:03.633999496 (519h40m30.560294172s) [upload=415871]: Starting upload
Jul 15 00:52:03.634082952 (519h40m30.560377645s) [upload=415871]: Done starting upload
Jul 15 00:52:03.634209535 (519h40m30.560504222s) [upload=415871]: Uploading segments...
Jul 15 00:52:03.634480377 (519h40m30.560775053s) [upload=415871]: Got next segment. Inline: false
Jul 15 00:52:03.634530708 (519h40m30.560825395s) [upload=415871]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 15 00:52:03.634536382 (519h40m30.560831069s) [upload=415871]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 15 00:52:03.695685280 (519h40m30.621979973s) [seg_pos={0 0}, upload=415871]: Begin upload segment...
Jul 15 00:52:03.695768486 (519h40m30.622063158s) [seg_pos={0 0}, upload=415871]: Done begin upload segment.
Jul 15 00:52:03.695803916 (519h40m30.622098609s) [noise=true , address=77.90.31.14:4663 , piece_id=FTJPZQIFAM7DTUAQPLMWWQKA6ECPZCHYVRM7CVQRYYZWGOZKPE2Q, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.696411043 (519h40m30.622705739s) [noise=true , address=104.36.82.2:4729 , piece_id=AJMUHTXD4T27L6QOQT2MESVEJ2P5S7AITD77NA5D5KHSWLUSLO6Q, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.696879328 (519h40m30.623174024s) [noise=true , address=104.36.82.7:4909 , piece_id=4LWX2WM4TEUJK7NLMIEN45CODLCYMX5QJ75DIIODXYKH5JEWUFJA, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.697539086 (519h40m30.623833786s) [noise=true , address=104.36.82.10:4078 , piece_id=OZJCT2NLKSVSJLZNTUZKP4FHNJPTZ3WAMKFLWH3YZEOLLDRJ3IOA, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.698495186 (519h40m30.624789876s) [noise=true , address=104.36.82.9:4006 , piece_id=BFYMJD5WM7LZRLRDOU42ICLG5GXJIKTHSCFTJZIDV5BXBXTUPWYQ, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.698964985 (519h40m30.625259674s) [noise=true , address=104.36.82.15:4258 , piece_id=LHV6F4ZT6OFPKGOVGBSE3KHNND2SF3WWM4XRRE7QGNLG22ZQ5KGQ, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.699380429 (519h40m30.625675118s) [noise=true , address=104.36.82.14:4222 , piece_id=JF7XOB4JLIJYZS2KDBVBATWQWRD4OS43IVFTSPJYVCCLAF2OWOYA, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.699832706 (519h40m30.626127395s) [noise=true , address=77.90.31.10:4518 , piece_id=BBO7ZS4WLRV2F3DSPQEG3VDH2BCPL6NHSVHVMIF7JDYZOU2PGPFQ, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.701385016 (519h40m30.627679726s) [noise=true , address=104.36.82.5:4839 , piece_id=6SLBVGQTJFRJP7QA2CVG2ARGYYFWLRBOCBN33O5WN4NUTRFT6DUA, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.701879105 (519h40m30.628173794s) [noise=true , address=104.36.82.6:4875 , piece_id=UXCSD7HDZCRJ5WW6235ZQAT2FICARKHZWI7A2YE3JVNWA5W7GSOQ, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.702867315 (519h40m30.629162012s) [noise=true , address=45.91.116.13:4155 , piece_id=7HXRG74ZSSX42CIXL6SVROLXE22DFC4BLPIXAQOOGOIGD6CPRYPA, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.703883641 (519h40m30.63017834s) [noise=true , address=104.36.82.4:4801 , piece_id=W4ZHXNHCSFTQ4PNBUE44FFD6BML36MXLADIK7W4EMUNWIJW2FZ3A, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.704552977 (519h40m30.630847673s) [noise=true , address=77.90.31.12:4590 , piece_id=W7ZVFJDIJNFOJ2ONAIL5FB6QIDW75FV23MTQYIDISNTI4HMWP72Q, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.705047441 (519h40m30.631342118s) [noise=true , address=45.91.116.10:4083 , piece_id=GBI3BQEAT7HCN3DP775SJ3FERFVDOXOTVCW6RZWKCP7P6T5K7DWQ, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.705519581 (519h40m30.631814267s) [noise=true , address=45.91.116.14:4189 , piece_id=Y52DGD267UU5OQKDFDZYYUL3DWIMFDI6TX533SOWBCZCLRMMTE2Q, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.705848528 (519h40m30.6321432s) [noise=true , address=77.90.31.15:4697 , piece_id=E3ZD7PJOQTOETE66TKCAMXM33JDTJUVTHUGGZ4EAN25DFFNCAUTA, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.827107120 (519h40m30.753401817s) [upload=415871]: Next returned nil segment
Jul 15 00:52:03.827125125 (519h40m30.753419825s) [upload=415871]: Waiting for error group managing segments...
Jul 15 00:52:03.831917682 (519h40m30.758212372s) [noise=true , address=45.91.116.10:4083 , piece_id=GBI3BQEAT7HCN3DP775SJ3FERFVDOXOTVCW6RZWKCP7P6T5K7DWQ, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.832572362 (519h40m30.758867042s) [noise=true , address=77.90.31.10:4518 , piece_id=BBO7ZS4WLRV2F3DSPQEG3VDH2BCPL6NHSVHVMIF7JDYZOU2PGPFQ, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.833108591 (519h40m30.759403279s) [noise=true , address=77.90.31.12:4590 , piece_id=W7ZVFJDIJNFOJ2ONAIL5FB6QIDW75FV23MTQYIDISNTI4HMWP72Q, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.833610214 (519h40m30.7599049s) [noise=true , address=77.90.31.14:4663 , piece_id=FTJPZQIFAM7DTUAQPLMWWQKA6ECPZCHYVRM7CVQRYYZWGOZKPE2Q, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.834676903 (519h40m30.760971596s) [noise=true , address=45.91.116.14:4189 , piece_id=Y52DGD267UU5OQKDFDZYYUL3DWIMFDI6TX533SOWBCZCLRMMTE2Q, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.839879741 (519h40m30.766174428s) [noise=true , address=104.36.82.14:4222 , piece_id=JF7XOB4JLIJYZS2KDBVBATWQWRD4OS43IVFTSPJYVCCLAF2OWOYA, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.840357933 (519h40m30.76665261s) [noise=true , address=104.36.82.7:4909 , piece_id=4LWX2WM4TEUJK7NLMIEN45CODLCYMX5QJ75DIIODXYKH5JEWUFJA, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.841075253 (519h40m30.767369934s) [noise=true , address=104.36.82.5:4839 , piece_id=6SLBVGQTJFRJP7QA2CVG2ARGYYFWLRBOCBN33O5WN4NUTRFT6DUA, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.841610950 (519h40m30.767905627s) [noise=true , address=104.36.82.9:4006 , piece_id=BFYMJD5WM7LZRLRDOU42ICLG5GXJIKTHSCFTJZIDV5BXBXTUPWYQ, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.842107860 (519h40m30.768402541s) [noise=true , address=104.36.82.15:4258 , piece_id=LHV6F4ZT6OFPKGOVGBSE3KHNND2SF3WWM4XRRE7QGNLG22ZQ5KGQ, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.842764021 (519h40m30.769058699s) [noise=true , address=77.90.31.15:4697 , piece_id=E3ZD7PJOQTOETE66TKCAMXM33JDTJUVTHUGGZ4EAN25DFFNCAUTA, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.843339192 (519h40m30.769633866s) [noise=true , address=104.36.82.2:4729 , piece_id=AJMUHTXD4T27L6QOQT2MESVEJ2P5S7AITD77NA5D5KHSWLUSLO6Q, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.843944968 (519h40m30.77023966s) [noise=true , address=104.36.82.6:4875 , piece_id=UXCSD7HDZCRJ5WW6235ZQAT2FICARKHZWI7A2YE3JVNWA5W7GSOQ, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.843958762 (519h40m30.770253441s) [seg_pos={0 0}, upload=415871]: Segment reached optimal threshold of 13 pieces.
Jul 15 00:52:03.844132370 (519h40m30.770427052s) [noise=true , address=104.36.82.4:4801 , piece_id=W4ZHXNHCSFTQ4PNBUE44FFD6BML36MXLADIK7W4EMUNWIJW2FZ3A, seg_pos={0 0}, upload=415871]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: expected piece hash; context canceled; EOF
Jul 15 00:52:03.844340698 (519h40m30.770635385s) [noise=true , address=45.91.116.13:4155 , piece_id=7HXRG74ZSSX42CIXL6SVROLXE22DFC4BLPIXAQOOGOIGD6CPRYPA, seg_pos={0 0}, upload=415871]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf): protocol: expected piece hash; context canceled; EOF
Jul 15 00:52:03.844694597 (519h40m30.770989276s) [noise=true , address=104.36.82.10:4078 , piece_id=OZJCT2NLKSVSJLZNTUZKP4FHNJPTZ3WAMKFLWH3YZEOLLDRJ3IOA, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.844732096 (519h40m30.77102683s) [seg_pos={0 0}, upload=415871]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 15 00:52:03.844815170 (519h40m30.771109852s) [upload=415871]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 15 00:52:03.844820406 (519h40m30.771115095s) [upload=415871]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 15 00:52:03.899279901 (519h40m30.825574576s) [upload=415871]: Done uploading segments...
Jul 15 00:52:03.899298008 (519h40m30.825592687s) [upload=415871]: Upload finished. err: <nil>
Duration: 449.377711ms
core
uplink
us-select-1
|
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)antiblip:
2 out of 60 is failed (threshold=3) [.X................X.........................................] duration:
98.031633ms
FAIL (2025-07-15 01:34:30 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp: lookup auth.us-select-1.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:58822->127.0.0.1:53: i/o timeout
Duration: 98.056444ms
edge
authservice
http
us-select-1
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
28.379464ms target:
https://us1.storj.io/login
FAIL (2025-07-15 01:34:26 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp: lookup us1.storj.io on 127.0.0.1:53: read udp 127.0.0.1:39250->127.0.0.1:53: i/o timeout
Duration: 28.407954ms
core
satellite
us1
|
checkers.HTTP{"GET" "https://storj.io/", ExpectedStatus: 301, ExpectedRedirect: "https://www.storj.io/"} failure (antiblip window 6/15): (http.storj_io)antiblip:
1 out of 15 is failed (threshold=6) [X..............] duration:
47.621166ms target:
https://storj.io/
FAIL (2025-07-15 01:34:25 UTC)
orbiter warning: Get "https://storj.io/": dial tcp: lookup storj.io on 127.0.0.1:53: read udp 127.0.0.1:54555->127.0.0.1:53: i/o timeout
Duration: 47.705924ms
core
website
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [........X...................................................] duration:
278.015945ms
FAIL (2025-07-15 01:34:21 UTC)
orbiter warning: uplink: bucket: metaclient: context deadline exceeded
Duration: 278.046068ms
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......X....................................................] duration:
594.942413ms
FAIL (2025-07-15 01:34:09 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF
Jul 15 00:50:55.285687709 (519h39m22.211982402s) [noise=true , address=104.36.82.6:4875 , piece_id=EI2GYFUMGM4RBVMZ4YX4XBHU2F4DGW7JU3WFO7ZJ3PB5TA5JEPAA, seg_pos={0 0}, upload=415855]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF
Jul 15 00:50:55.285704441 (519h39m22.211999129s) [seg_pos={0 0}, upload=415855]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 15 00:50:55.285782799 (519h39m22.212077495s) [upload=415855]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 15 00:50:55.285789426 (519h39m22.212084124s) [upload=415855]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 15 00:50:55.339105588 (519h39m22.265400284s) [upload=415855]: Done uploading segments...
Jul 15 00:50:55.339127659 (519h39m22.265422364s) [upload=415855]: Upload finished. err: <nil>
Jul 15 00:52:03.633999496 (519h40m30.560294172s) [upload=415871]: Starting upload
Jul 15 00:52:03.634082952 (519h40m30.560377645s) [upload=415871]: Done starting upload
Jul 15 00:52:03.634209535 (519h40m30.560504222s) [upload=415871]: Uploading segments...
Jul 15 00:52:03.634480377 (519h40m30.560775053s) [upload=415871]: Got next segment. Inline: false
Jul 15 00:52:03.634530708 (519h40m30.560825395s) [upload=415871]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 15 00:52:03.634536382 (519h40m30.560831069s) [upload=415871]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 15 00:52:03.695685280 (519h40m30.621979973s) [seg_pos={0 0}, upload=415871]: Begin upload segment...
Jul 15 00:52:03.695768486 (519h40m30.622063158s) [seg_pos={0 0}, upload=415871]: Done begin upload segment.
Jul 15 00:52:03.695803916 (519h40m30.622098609s) [noise=true , address=77.90.31.14:4663 , piece_id=FTJPZQIFAM7DTUAQPLMWWQKA6ECPZCHYVRM7CVQRYYZWGOZKPE2Q, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.696411043 (519h40m30.622705739s) [noise=true , address=104.36.82.2:4729 , piece_id=AJMUHTXD4T27L6QOQT2MESVEJ2P5S7AITD77NA5D5KHSWLUSLO6Q, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.696879328 (519h40m30.623174024s) [noise=true , address=104.36.82.7:4909 , piece_id=4LWX2WM4TEUJK7NLMIEN45CODLCYMX5QJ75DIIODXYKH5JEWUFJA, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.697539086 (519h40m30.623833786s) [noise=true , address=104.36.82.10:4078 , piece_id=OZJCT2NLKSVSJLZNTUZKP4FHNJPTZ3WAMKFLWH3YZEOLLDRJ3IOA, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.698495186 (519h40m30.624789876s) [noise=true , address=104.36.82.9:4006 , piece_id=BFYMJD5WM7LZRLRDOU42ICLG5GXJIKTHSCFTJZIDV5BXBXTUPWYQ, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.698964985 (519h40m30.625259674s) [noise=true , address=104.36.82.15:4258 , piece_id=LHV6F4ZT6OFPKGOVGBSE3KHNND2SF3WWM4XRRE7QGNLG22ZQ5KGQ, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.699380429 (519h40m30.625675118s) [noise=true , address=104.36.82.14:4222 , piece_id=JF7XOB4JLIJYZS2KDBVBATWQWRD4OS43IVFTSPJYVCCLAF2OWOYA, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.699832706 (519h40m30.626127395s) [noise=true , address=77.90.31.10:4518 , piece_id=BBO7ZS4WLRV2F3DSPQEG3VDH2BCPL6NHSVHVMIF7JDYZOU2PGPFQ, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.701385016 (519h40m30.627679726s) [noise=true , address=104.36.82.5:4839 , piece_id=6SLBVGQTJFRJP7QA2CVG2ARGYYFWLRBOCBN33O5WN4NUTRFT6DUA, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.701879105 (519h40m30.628173794s) [noise=true , address=104.36.82.6:4875 , piece_id=UXCSD7HDZCRJ5WW6235ZQAT2FICARKHZWI7A2YE3JVNWA5W7GSOQ, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.702867315 (519h40m30.629162012s) [noise=true , address=45.91.116.13:4155 , piece_id=7HXRG74ZSSX42CIXL6SVROLXE22DFC4BLPIXAQOOGOIGD6CPRYPA, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.703883641 (519h40m30.63017834s) [noise=true , address=104.36.82.4:4801 , piece_id=W4ZHXNHCSFTQ4PNBUE44FFD6BML36MXLADIK7W4EMUNWIJW2FZ3A, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.704552977 (519h40m30.630847673s) [noise=true , address=77.90.31.12:4590 , piece_id=W7ZVFJDIJNFOJ2ONAIL5FB6QIDW75FV23MTQYIDISNTI4HMWP72Q, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.705047441 (519h40m30.631342118s) [noise=true , address=45.91.116.10:4083 , piece_id=GBI3BQEAT7HCN3DP775SJ3FERFVDOXOTVCW6RZWKCP7P6T5K7DWQ, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.705519581 (519h40m30.631814267s) [noise=true , address=45.91.116.14:4189 , piece_id=Y52DGD267UU5OQKDFDZYYUL3DWIMFDI6TX533SOWBCZCLRMMTE2Q, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.705848528 (519h40m30.6321432s) [noise=true , address=77.90.31.15:4697 , piece_id=E3ZD7PJOQTOETE66TKCAMXM33JDTJUVTHUGGZ4EAN25DFFNCAUTA, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.827107120 (519h40m30.753401817s) [upload=415871]: Next returned nil segment
Jul 15 00:52:03.827125125 (519h40m30.753419825s) [upload=415871]: Waiting for error group managing segments...
Jul 15 00:52:03.831917682 (519h40m30.758212372s) [noise=true , address=45.91.116.10:4083 , piece_id=GBI3BQEAT7HCN3DP775SJ3FERFVDOXOTVCW6RZWKCP7P6T5K7DWQ, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.832572362 (519h40m30.758867042s) [noise=true , address=77.90.31.10:4518 , piece_id=BBO7ZS4WLRV2F3DSPQEG3VDH2BCPL6NHSVHVMIF7JDYZOU2PGPFQ, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.833108591 (519h40m30.759403279s) [noise=true , address=77.90.31.12:4590 , piece_id=W7ZVFJDIJNFOJ2ONAIL5FB6QIDW75FV23MTQYIDISNTI4HMWP72Q, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.833610214 (519h40m30.7599049s) [noise=true , address=77.90.31.14:4663 , piece_id=FTJPZQIFAM7DTUAQPLMWWQKA6ECPZCHYVRM7CVQRYYZWGOZKPE2Q, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.834676903 (519h40m30.760971596s) [noise=true , address=45.91.116.14:4189 , piece_id=Y52DGD267UU5OQKDFDZYYUL3DWIMFDI6TX533SOWBCZCLRMMTE2Q, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.839879741 (519h40m30.766174428s) [noise=true , address=104.36.82.14:4222 , piece_id=JF7XOB4JLIJYZS2KDBVBATWQWRD4OS43IVFTSPJYVCCLAF2OWOYA, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.840357933 (519h40m30.76665261s) [noise=true , address=104.36.82.7:4909 , piece_id=4LWX2WM4TEUJK7NLMIEN45CODLCYMX5QJ75DIIODXYKH5JEWUFJA, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.841075253 (519h40m30.767369934s) [noise=true , address=104.36.82.5:4839 , piece_id=6SLBVGQTJFRJP7QA2CVG2ARGYYFWLRBOCBN33O5WN4NUTRFT6DUA, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.841610950 (519h40m30.767905627s) [noise=true , address=104.36.82.9:4006 , piece_id=BFYMJD5WM7LZRLRDOU42ICLG5GXJIKTHSCFTJZIDV5BXBXTUPWYQ, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.842107860 (519h40m30.768402541s) [noise=true , address=104.36.82.15:4258 , piece_id=LHV6F4ZT6OFPKGOVGBSE3KHNND2SF3WWM4XRRE7QGNLG22ZQ5KGQ, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.842764021 (519h40m30.769058699s) [noise=true , address=77.90.31.15:4697 , piece_id=E3ZD7PJOQTOETE66TKCAMXM33JDTJUVTHUGGZ4EAN25DFFNCAUTA, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.843339192 (519h40m30.769633866s) [noise=true , address=104.36.82.2:4729 , piece_id=AJMUHTXD4T27L6QOQT2MESVEJ2P5S7AITD77NA5D5KHSWLUSLO6Q, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.843944968 (519h40m30.77023966s) [noise=true , address=104.36.82.6:4875 , piece_id=UXCSD7HDZCRJ5WW6235ZQAT2FICARKHZWI7A2YE3JVNWA5W7GSOQ, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.843958762 (519h40m30.770253441s) [seg_pos={0 0}, upload=415871]: Segment reached optimal threshold of 13 pieces.
Jul 15 00:52:03.844132370 (519h40m30.770427052s) [noise=true , address=104.36.82.4:4801 , piece_id=W4ZHXNHCSFTQ4PNBUE44FFD6BML36MXLADIK7W4EMUNWIJW2FZ3A, seg_pos={0 0}, upload=415871]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: expected piece hash; context canceled; EOF
Jul 15 00:52:03.844340698 (519h40m30.770635385s) [noise=true , address=45.91.116.13:4155 , piece_id=7HXRG74ZSSX42CIXL6SVROLXE22DFC4BLPIXAQOOGOIGD6CPRYPA, seg_pos={0 0}, upload=415871]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf): protocol: expected piece hash; context canceled; EOF
Jul 15 00:52:03.844694597 (519h40m30.770989276s) [noise=true , address=104.36.82.10:4078 , piece_id=OZJCT2NLKSVSJLZNTUZKP4FHNJPTZ3WAMKFLWH3YZEOLLDRJ3IOA, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.844732096 (519h40m30.77102683s) [seg_pos={0 0}, upload=415871]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 15 00:52:03.844815170 (519h40m30.771109852s) [upload=415871]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 15 00:52:03.844820406 (519h40m30.771115095s) [upload=415871]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 15 00:52:03.899279901 (519h40m30.825574576s) [upload=415871]: Done uploading segments...
Jul 15 00:52:03.899298008 (519h40m30.825592687s) [upload=415871]: Upload finished. err: <nil>
Duration: 594.997931ms
core
uplink
us-select-1
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
2 out of 60 is failed (threshold=3) [XX..........................................................] duration:
188.639751ms target:
https://us1.storj.io/login
FAIL (2025-07-15 01:34:08 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp: lookup us1.storj.io on 127.0.0.1:53: read udp 127.0.0.1:39250->127.0.0.1:53: i/o timeout
Duration: 188.696131ms
core
satellite
us1
|
checkers.HTTP{"GET" "https://storj.io/", ExpectedStatus: 301, ExpectedRedirect: "https://www.storj.io/"} failure (antiblip window 6/15): (http.storj_io)antiblip:
1 out of 15 is failed (threshold=6) [.X.............] duration:
60.266679ms target:
https://storj.io/
FAIL (2025-07-15 01:34:06 UTC)
orbiter warning: Get "https://storj.io/": dial tcp: lookup storj.io on 127.0.0.1:53: read udp 127.0.0.1:54555->127.0.0.1:53: i/o timeout
Duration: 60.299342ms
core
website
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [................X...........................................] duration:
428.218116ms
FAIL (2025-07-15 01:33:54 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp: lookup auth.us-select-1.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:42004->127.0.0.1:53: i/o timeout
Duration: 428.257423ms
edge
gateway
us-select-1
|
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)antiblip:
2 out of 60 is failed (threshold=3) [..X................X........................................] duration:
56.061105ms
FAIL (2025-07-15 01:33:50 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp: lookup auth.us-select-1.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:58822->127.0.0.1:53: i/o timeout
Duration: 56.088804ms
edge
authservice
http
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [..............XX............................................] duration:
488.657648ms
FAIL (2025-07-15 01:33:36 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp: lookup auth.us-select-1.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:58822->127.0.0.1:53: i/o timeout
Duration: 488.681524ms
edge
gateway
us-select-1
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
2 out of 60 is failed (threshold=3) [.XX.........................................................] duration:
30.61276ms target:
https://us1.storj.io/login
FAIL (2025-07-15 01:33:34 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp: lookup us1.storj.io on 127.0.0.1:53: read udp 127.0.0.1:39250->127.0.0.1:53: i/o timeout
Duration: 30.65516ms
core
satellite
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.........X..................................................] duration:
341.855515ms
FAIL (2025-07-15 01:33:18 UTC)
orbiter warning: uplink: bucket: metaclient: context deadline exceeded
Duration: 341.908596ms
core
uplink
us1
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
2 out of 60 is failed (threshold=3) [..XX........................................................] duration:
32.765373ms target:
https://us1.storj.io/login
FAIL (2025-07-15 01:33:13 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp: lookup us1.storj.io on 127.0.0.1:53: read udp 127.0.0.1:39250->127.0.0.1:53: i/o timeout
Duration: 32.809038ms
core
satellite
us1
|
checkers.HTTP{"GET" "https://storj.io/", ExpectedStatus: 301, ExpectedRedirect: "https://www.storj.io/"} failure (antiblip window 6/15): (http.storj_io)antiblip:
1 out of 15 is failed (threshold=6) [..X............] duration:
55.531615ms target:
https://storj.io/
FAIL (2025-07-15 01:33:09 UTC)
orbiter warning: Get "https://storj.io/": dial tcp: lookup storj.io on 127.0.0.1:53: read udp 127.0.0.1:54555->127.0.0.1:53: i/o timeout
Duration: 55.559911ms
core
website
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........X...................................................] duration:
470.737497ms
FAIL (2025-07-15 01:33:05 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF
Jul 15 00:50:55.285687709 (519h39m22.211982402s) [noise=true , address=104.36.82.6:4875 , piece_id=EI2GYFUMGM4RBVMZ4YX4XBHU2F4DGW7JU3WFO7ZJ3PB5TA5JEPAA, seg_pos={0 0}, upload=415855]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2): protocol: expected piece hash; context canceled; EOF
Jul 15 00:50:55.285704441 (519h39m22.211999129s) [seg_pos={0 0}, upload=415855]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 15 00:50:55.285782799 (519h39m22.212077495s) [upload=415855]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 15 00:50:55.285789426 (519h39m22.212084124s) [upload=415855]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 15 00:50:55.339105588 (519h39m22.265400284s) [upload=415855]: Done uploading segments...
Jul 15 00:50:55.339127659 (519h39m22.265422364s) [upload=415855]: Upload finished. err: <nil>
Jul 15 00:52:03.633999496 (519h40m30.560294172s) [upload=415871]: Starting upload
Jul 15 00:52:03.634082952 (519h40m30.560377645s) [upload=415871]: Done starting upload
Jul 15 00:52:03.634209535 (519h40m30.560504222s) [upload=415871]: Uploading segments...
Jul 15 00:52:03.634480377 (519h40m30.560775053s) [upload=415871]: Got next segment. Inline: false
Jul 15 00:52:03.634530708 (519h40m30.560825395s) [upload=415871]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 15 00:52:03.634536382 (519h40m30.560831069s) [upload=415871]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 15 00:52:03.695685280 (519h40m30.621979973s) [seg_pos={0 0}, upload=415871]: Begin upload segment...
Jul 15 00:52:03.695768486 (519h40m30.622063158s) [seg_pos={0 0}, upload=415871]: Done begin upload segment.
Jul 15 00:52:03.695803916 (519h40m30.622098609s) [noise=true , address=77.90.31.14:4663 , piece_id=FTJPZQIFAM7DTUAQPLMWWQKA6ECPZCHYVRM7CVQRYYZWGOZKPE2Q, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.696411043 (519h40m30.622705739s) [noise=true , address=104.36.82.2:4729 , piece_id=AJMUHTXD4T27L6QOQT2MESVEJ2P5S7AITD77NA5D5KHSWLUSLO6Q, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.696879328 (519h40m30.623174024s) [noise=true , address=104.36.82.7:4909 , piece_id=4LWX2WM4TEUJK7NLMIEN45CODLCYMX5QJ75DIIODXYKH5JEWUFJA, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.697539086 (519h40m30.623833786s) [noise=true , address=104.36.82.10:4078 , piece_id=OZJCT2NLKSVSJLZNTUZKP4FHNJPTZ3WAMKFLWH3YZEOLLDRJ3IOA, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.698495186 (519h40m30.624789876s) [noise=true , address=104.36.82.9:4006 , piece_id=BFYMJD5WM7LZRLRDOU42ICLG5GXJIKTHSCFTJZIDV5BXBXTUPWYQ, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.698964985 (519h40m30.625259674s) [noise=true , address=104.36.82.15:4258 , piece_id=LHV6F4ZT6OFPKGOVGBSE3KHNND2SF3WWM4XRRE7QGNLG22ZQ5KGQ, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.699380429 (519h40m30.625675118s) [noise=true , address=104.36.82.14:4222 , piece_id=JF7XOB4JLIJYZS2KDBVBATWQWRD4OS43IVFTSPJYVCCLAF2OWOYA, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.699832706 (519h40m30.626127395s) [noise=true , address=77.90.31.10:4518 , piece_id=BBO7ZS4WLRV2F3DSPQEG3VDH2BCPL6NHSVHVMIF7JDYZOU2PGPFQ, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.701385016 (519h40m30.627679726s) [noise=true , address=104.36.82.5:4839 , piece_id=6SLBVGQTJFRJP7QA2CVG2ARGYYFWLRBOCBN33O5WN4NUTRFT6DUA, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.701879105 (519h40m30.628173794s) [noise=true , address=104.36.82.6:4875 , piece_id=UXCSD7HDZCRJ5WW6235ZQAT2FICARKHZWI7A2YE3JVNWA5W7GSOQ, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.702867315 (519h40m30.629162012s) [noise=true , address=45.91.116.13:4155 , piece_id=7HXRG74ZSSX42CIXL6SVROLXE22DFC4BLPIXAQOOGOIGD6CPRYPA, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.703883641 (519h40m30.63017834s) [noise=true , address=104.36.82.4:4801 , piece_id=W4ZHXNHCSFTQ4PNBUE44FFD6BML36MXLADIK7W4EMUNWIJW2FZ3A, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.704552977 (519h40m30.630847673s) [noise=true , address=77.90.31.12:4590 , piece_id=W7ZVFJDIJNFOJ2ONAIL5FB6QIDW75FV23MTQYIDISNTI4HMWP72Q, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.705047441 (519h40m30.631342118s) [noise=true , address=45.91.116.10:4083 , piece_id=GBI3BQEAT7HCN3DP775SJ3FERFVDOXOTVCW6RZWKCP7P6T5K7DWQ, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.705519581 (519h40m30.631814267s) [noise=true , address=45.91.116.14:4189 , piece_id=Y52DGD267UU5OQKDFDZYYUL3DWIMFDI6TX533SOWBCZCLRMMTE2Q, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.705848528 (519h40m30.6321432s) [noise=true , address=77.90.31.15:4697 , piece_id=E3ZD7PJOQTOETE66TKCAMXM33JDTJUVTHUGGZ4EAN25DFFNCAUTA, seg_pos={0 0}, upload=415871]: Uploading piece...
Jul 15 00:52:03.827107120 (519h40m30.753401817s) [upload=415871]: Next returned nil segment
Jul 15 00:52:03.827125125 (519h40m30.753419825s) [upload=415871]: Waiting for error group managing segments...
Jul 15 00:52:03.831917682 (519h40m30.758212372s) [noise=true , address=45.91.116.10:4083 , piece_id=GBI3BQEAT7HCN3DP775SJ3FERFVDOXOTVCW6RZWKCP7P6T5K7DWQ, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.832572362 (519h40m30.758867042s) [noise=true , address=77.90.31.10:4518 , piece_id=BBO7ZS4WLRV2F3DSPQEG3VDH2BCPL6NHSVHVMIF7JDYZOU2PGPFQ, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.833108591 (519h40m30.759403279s) [noise=true , address=77.90.31.12:4590 , piece_id=W7ZVFJDIJNFOJ2ONAIL5FB6QIDW75FV23MTQYIDISNTI4HMWP72Q, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.833610214 (519h40m30.7599049s) [noise=true , address=77.90.31.14:4663 , piece_id=FTJPZQIFAM7DTUAQPLMWWQKA6ECPZCHYVRM7CVQRYYZWGOZKPE2Q, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.834676903 (519h40m30.760971596s) [noise=true , address=45.91.116.14:4189 , piece_id=Y52DGD267UU5OQKDFDZYYUL3DWIMFDI6TX533SOWBCZCLRMMTE2Q, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.839879741 (519h40m30.766174428s) [noise=true , address=104.36.82.14:4222 , piece_id=JF7XOB4JLIJYZS2KDBVBATWQWRD4OS43IVFTSPJYVCCLAF2OWOYA, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.840357933 (519h40m30.76665261s) [noise=true , address=104.36.82.7:4909 , piece_id=4LWX2WM4TEUJK7NLMIEN45CODLCYMX5QJ75DIIODXYKH5JEWUFJA, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.841075253 (519h40m30.767369934s) [noise=true , address=104.36.82.5:4839 , piece_id=6SLBVGQTJFRJP7QA2CVG2ARGYYFWLRBOCBN33O5WN4NUTRFT6DUA, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.841610950 (519h40m30.767905627s) [noise=true , address=104.36.82.9:4006 , piece_id=BFYMJD5WM7LZRLRDOU42ICLG5GXJIKTHSCFTJZIDV5BXBXTUPWYQ, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.842107860 (519h40m30.768402541s) [noise=true , address=104.36.82.15:4258 , piece_id=LHV6F4ZT6OFPKGOVGBSE3KHNND2SF3WWM4XRRE7QGNLG22ZQ5KGQ, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.842764021 (519h40m30.769058699s) [noise=true , address=77.90.31.15:4697 , piece_id=E3ZD7PJOQTOETE66TKCAMXM33JDTJUVTHUGGZ4EAN25DFFNCAUTA, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.843339192 (519h40m30.769633866s) [noise=true , address=104.36.82.2:4729 , piece_id=AJMUHTXD4T27L6QOQT2MESVEJ2P5S7AITD77NA5D5KHSWLUSLO6Q, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.843944968 (519h40m30.77023966s) [noise=true , address=104.36.82.6:4875 , piece_id=UXCSD7HDZCRJ5WW6235ZQAT2FICARKHZWI7A2YE3JVNWA5W7GSOQ, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.843958762 (519h40m30.770253441s) [seg_pos={0 0}, upload=415871]: Segment reached optimal threshold of 13 pieces.
Jul 15 00:52:03.844132370 (519h40m30.770427052s) [noise=true , address=104.36.82.4:4801 , piece_id=W4ZHXNHCSFTQ4PNBUE44FFD6BML36MXLADIK7W4EMUNWIJW2FZ3A, seg_pos={0 0}, upload=415871]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: expected piece hash; context canceled; EOF
Jul 15 00:52:03.844340698 (519h40m30.770635385s) [noise=true , address=45.91.116.13:4155 , piece_id=7HXRG74ZSSX42CIXL6SVROLXE22DFC4BLPIXAQOOGOIGD6CPRYPA, seg_pos={0 0}, upload=415871]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf): protocol: expected piece hash; context canceled; EOF
Jul 15 00:52:03.844694597 (519h40m30.770989276s) [noise=true , address=104.36.82.10:4078 , piece_id=OZJCT2NLKSVSJLZNTUZKP4FHNJPTZ3WAMKFLWH3YZEOLLDRJ3IOA, seg_pos={0 0}, upload=415871]: Done uploading piece. err: <nil>
Jul 15 00:52:03.844732096 (519h40m30.77102683s) [seg_pos={0 0}, upload=415871]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 15 00:52:03.844815170 (519h40m30.771109852s) [upload=415871]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 15 00:52:03.844820406 (519h40m30.771115095s) [upload=415871]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 15 00:52:03.899279901 (519h40m30.825574576s) [upload=415871]: Done uploading segments...
Jul 15 00:52:03.899298008 (519h40m30.825592687s) [upload=415871]: Upload finished. err: <nil>
Duration: 470.765269ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
2 out of 60 is failed (threshold=3) [X................X..........................................] duration:
298.009009ms
FAIL (2025-07-15 01:32:40 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp: lookup auth.us-select-1.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:42004->127.0.0.1:53: i/o timeout
Duration: 298.034728ms
edge
gateway
us-select-1
|
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)antiblip:
2 out of 60 is failed (threshold=3) [...X................X.......................................] duration:
63.177333ms
FAIL (2025-07-15 01:32:38 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp: lookup auth.us-select-1.storjshare.io on 127.0.0.1:53: read udp 127.0.0.1:58822->127.0.0.1:53: i/o timeout
Duration: 63.209272ms
edge
authservice
http
us-select-1
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
2 out of 60 is failed (threshold=3) [...XX.......................................................] duration:
182.626013ms target:
https://us1.storj.io/login
FAIL (2025-07-15 01:32:36 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp: lookup us1.storj.io on 127.0.0.1:53: read udp 127.0.0.1:39250->127.0.0.1:53: i/o timeout
Duration: 182.666161ms
core
satellite
us1
|