Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
22 out of 60 is failed (threshold=3) [XXXX.XX.....X..XXXXX.XX..XXX.XX.XX..X.......................] duration:
671.848269ms
FAIL (2025-07-09 03:14:48 UTC)
operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 671.872234ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
13 out of 60 is failed (threshold=3) [X...XXX....XX...X..X..X.XX.X..X.............................] duration:
1.852747481s
FAIL (2025-07-09 03:14:41 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.852776149s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
23 out of 60 is failed (threshold=3) [XXXXX.XX.....X..XXXXX.XX..XXX.XX.XX..X......................] duration:
1.155279854s
FAIL (2025-07-09 03:14:34 UTC)
operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.155314337s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
13 out of 60 is failed (threshold=3) [.X...XXX....XX...X..X..X.XX.X..X............................] duration:
618.121697ms
FAIL (2025-07-09 03:14:04 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 618.142595ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
23 out of 60 is failed (threshold=3) [.XXXXX.XX.....X..XXXXX.XX..XXX.XX.XX..X.....................] duration:
632.056192ms
FAIL (2025-07-09 03:13:50 UTC)
operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 632.072776ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
24 out of 60 is failed (threshold=3) [X.XXXXX.XX.....X..XXXXX.XX..XXX.XX.XX..X....................] duration:
677.927565ms
FAIL (2025-07-09 03:13:34 UTC)
operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 677.970164ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
14 out of 60 is failed (threshold=3) [X.X...XXX....XX...X..X..X.XX.X..X...........................] duration:
665.665317ms
FAIL (2025-07-09 03:13:13 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 665.689718ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
25 out of 60 is failed (threshold=3) [XX.XXXXX.XX.....X..XXXXX.XX..XXX.XX.XX..X...................] duration:
710.3322ms
FAIL (2025-07-09 03:13:10 UTC)
operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 710.358325ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
15 out of 60 is failed (threshold=3) [XX.X...XXX....XX...X..X..X.XX.X..X..........................] duration:
546.60461ms
FAIL (2025-07-09 03:12:49 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 546.633344ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
26 out of 60 is failed (threshold=3) [XXX.XXXXX.XX.....X..XXXXX.XX..XXX.XX.XX..X..................] duration:
727.385096ms
FAIL (2025-07-09 03:12:01 UTC)
operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 727.410948ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
27 out of 60 is failed (threshold=3) [XXXX.XXXXX.XX.....X..XXXXX.XX..XXX.XX.XX..X.................] duration:
31.192886003s
FAIL (2025-07-09 03:11:55 UTC)
operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 31.192914308s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
15 out of 60 is failed (threshold=3) [.XX.X...XXX....XX...X..X..X.XX.X..X.........................] duration:
710.297258ms
FAIL (2025-07-09 03:11:51 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 710.326537ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
16 out of 60 is failed (threshold=3) [X.XX.X...XXX....XX...X..X..X.XX.X..X........................] duration:
518.350722ms
FAIL (2025-07-09 03:10:39 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 518.400042ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
28 out of 60 is failed (threshold=3) [XXXXX.XXXXX.XX.....X..XXXXX.XX..XXX.XX.XX..X................] duration:
597.901072ms
FAIL (2025-07-09 03:10:24 UTC)
operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 597.927389ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
17 out of 60 is failed (threshold=3) [XX.XX.X...XXX....XX...X..X..X.XX.X..X.......................] duration:
613.770972ms
FAIL (2025-07-09 03:10:01 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 613.912413ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
29 out of 60 is failed (threshold=3) [XXXXXX.XXXXX.XX.....X..XXXXX.XX..XXX.XX.XX..X...............] duration:
4.432027987s
FAIL (2025-07-09 03:09:46 UTC)
operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 4.432055703s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
18 out of 60 is failed (threshold=3) [XXX.XX.X...XXX....XX...X..X..X.XX.X..X......................] duration:
538.771084ms
FAIL (2025-07-09 03:09:43 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 538.793847ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
18 out of 60 is failed (threshold=3) [.XXX.XX.X...XXX....XX...X..X..X.XX.X..X.....................] duration:
667.851724ms
FAIL (2025-07-09 03:09:20 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 667.880477ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
29 out of 60 is failed (threshold=3) [.XXXXXX.XXXXX.XX.....X..XXXXX.XX..XXX.XX.XX..X..............] duration:
2.813605964s
FAIL (2025-07-09 03:09:09 UTC)
operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.813637398s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
18 out of 60 is failed (threshold=3) [..XXX.XX.X...XXX....XX...X..X..X.XX.X..X....................] duration:
506.29907ms
FAIL (2025-07-09 03:08:22 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 506.3242ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
18 out of 60 is failed (threshold=3) [...XXX.XX.X...XXX....XX...X..X..X.XX.X..X...................] duration:
770.425512ms
FAIL (2025-07-09 03:08:00 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 770.489826ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
30 out of 60 is failed (threshold=3) [X.XXXXXX.XXXXX.XX.....X..XXXXX.XX..XXX.XX.XX..X.............] duration:
857.274221ms
FAIL (2025-07-09 03:07:57 UTC)
operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 857.302152ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
18 out of 60 is failed (threshold=3) [....XXX.XX.X...XXX....XX...X..X..X.XX.X..X..................] duration:
491.929163ms
FAIL (2025-07-09 03:07:30 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 491.963355ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
18 out of 60 is failed (threshold=3) [.....XXX.XX.X...XXX....XX...X..X..X.XX.X..X.................] duration:
831.164886ms
FAIL (2025-07-09 03:06:54 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 831.189631ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
31 out of 60 is failed (threshold=3) [XX.XXXXXX.XXXXX.XX.....X..XXXXX.XX..XXX.XX.XX..X............] duration:
587.446316ms
FAIL (2025-07-09 03:06:44 UTC)
operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 587.511851ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
457.009775ms
FAIL (2025-07-09 03:06:41 UTC)
orbiter warning:
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
02:25:27.135824454 (377h13m54.062119141s) [noise=true , address=104.36.82.12:4150 , piece_id=DRG5SBA3EE4KVQPEQOJZ6SS537SFWYFUYQXOAQMWYTT6FPVSJIPQ, seg_pos={0 0}, upload=302099]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: context canceled; context canceled
Jul 9 02:25:27.135860079 (377h13m54.062154769s) [seg_pos={0 0}, upload=302099]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 02:25:27.135957283 (377h13m54.062251971s) [upload=302099]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:25:27.135964211 (377h13m54.062258897s) [upload=302099]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:25:27.199410713 (377h13m54.125705389s) [upload=302099]: Done uploading segments...
Jul 9 02:25:27.199427901 (377h13m54.125722577s) [upload=302099]: Upload finished. err: <nil>
Jul 9 02:25:57.289077655 (377h14m24.215372329s) [upload=302109]: Starting upload
Jul 9 02:25:57.289186750 (377h14m24.215481424s) [upload=302109]: Done starting upload
Jul 9 02:25:57.289444851 (377h14m24.215739548s) [upload=302109]: Uploading segments...
Jul 9 02:25:57.289623846 (377h14m24.215918535s) [upload=302109]: Got next segment. Inline: false
Jul 9 02:25:57.289696741 (377h14m24.215991432s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:25:57.289703375 (377h14m24.215998067s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:25:57.318313152 (377h14m24.244607838s) [seg_pos={0 0}, upload=302109]: Begin upload segment...
Jul 9 02:25:57.318363286 (377h14m24.244657964s) [seg_pos={0 0}, upload=302109]: Done begin upload segment.
Jul 9 02:25:57.318386772 (377h14m24.244681463s) [noise=true , address=77.90.31.11:4553 , piece_id=TYW7NBJWACIT7OQS7A4WGXUPR3MB2FKAN2OMGAZJERB4X2DLAO2Q, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318412127 (377h14m24.244706825s) [noise=true , address=77.90.31.10:4518 , piece_id=VJ4HWM2X4VSRSUCQL5QLMBYPFBYV4JYS5ZXCBCB6MTMB76EHZFGQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318822149 (377h14m24.245116841s) [noise=true , address=45.91.116.9:4263 , piece_id=5YILAJ5BZBGOURSZE343ST4RMOTWM6ZOB3EEQQWL5WUNYWOGSAPA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318965862 (377h14m24.24526056s) [noise=true , address=45.91.116.8:4045 , piece_id=KM6ZYTHR6JGTARKZQZYIESPKBEU5H2HQFA7OARIKD2ZXDHPFSMGA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.319187516 (377h14m24.245482203s) [noise=true , address=104.36.82.7:4909 , piece_id=ZRHLWDQ5KIDRWPJ5FXAANQTWERKXCR2PPEGN442IOSYSRFAG7VZA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.319523815 (377h14m24.245818512s) [noise=true , address=45.91.116.14:4191 , piece_id=GTR6T4OTGZTEZ76FWQZ46OCH5WB3O5X27EN4AV33ZJ7D5HX4535A, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.320556632 (377h14m24.24685133s) [noise=true , address=104.36.82.2:4732 , piece_id=DJIGHJMJRAULH2L4YT2WGRXPRQJMNKZG5LICHPSKOCHUL6QDV6BQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.321019141 (377h14m24.247313835s) [noise=true , address=77.90.31.17:4770 , piece_id=V36UBBIYOFJC6KFVVBXEVT4QBUIWN257D3YSDDRAGZVXH7ZXHTBA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.322591797 (377h14m24.248886493s) [noise=true , address=77.90.31.13:4627 , piece_id=HULEKFVZNDNKS4SWWGTJUHYZWWYPDCER5W4GFCVLLDL65PAHULFQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323120451 (377h14m24.249415138s) [noise=true , address=45.91.116.4:4939 , piece_id=ANA3EZU7OZWELNG4QEMTEAYZY5H3C3H4IJFT4XKPUB7WIF44FEVQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323544591 (377h14m24.249839288s) [noise=true , address=45.91.116.7:4011 , piece_id=VDOJRN6ACK4LO44BQTULO6BB3CLAVQI7YIC6XLR32CZV4UJSPE6Q, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323950189 (377h14m24.250244889s) [noise=true , address=104.36.82.10:4078 , piece_id=ZFVUQAEV6DDFLPKCZA2XJ2BZHMMM745LGB4JRZJZQPG72WECGMDQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.326585810 (377h14m24.252880507s) [noise=true , address=45.91.116.5:4297 , piece_id=MSDWDLY5PJHQVI5565EKANJVSTMZFEUD2OTYZ52C65XQUICNMC3A, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.327871214 (377h14m24.254165904s) [noise=true , address=104.36.82.6:4875 , piece_id=C275F2JPVAF52E34CEHT5QXCHUFGIQZ7XGMOAOUOT4RA7YOHRQLQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.328605288 (377h14m24.254899986s) [noise=true , address=104.36.82.5:4838 , piece_id=WK3ALBOEMWWTVEA4FKCS352SZ6YYB35LELURNWXQZK32K4HYDRNA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.335409087 (377h14m24.261703784s) [noise=true , address=45.91.116.6:4973 , piece_id=Z7HQQ2O5QUSJKT4EMIDMEI5PIAEHNVRYX7MALUR2GQULRFIBUOKA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.475344637 (377h14m24.401639326s) [upload=302109]: Next returned nil segment
Jul 9 02:25:57.475376181 (377h14m24.401670879s) [upload=302109]: Waiting for error group managing segments...
Jul 9 02:25:57.477269278 (377h14m24.403563958s) [noise=true , address=45.91.116.7:4011 , piece_id=VDOJRN6ACK4LO44BQTULO6BB3CLAVQI7YIC6XLR32CZV4UJSPE6Q, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.477736179 (377h14m24.40403086s) [noise=true , address=45.91.116.9:4263 , piece_id=5YILAJ5BZBGOURSZE343ST4RMOTWM6ZOB3EEQQWL5WUNYWOGSAPA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.478396652 (377h14m24.404691332s) [noise=true , address=77.90.31.10:4518 , piece_id=VJ4HWM2X4VSRSUCQL5QLMBYPFBYV4JYS5ZXCBCB6MTMB76EHZFGQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.478919538 (377h14m24.405214224s) [noise=true , address=45.91.116.14:4191 , piece_id=GTR6T4OTGZTEZ76FWQZ46OCH5WB3O5X27EN4AV33ZJ7D5HX4535A, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481426716 (377h14m24.407721396s) [noise=true , address=45.91.116.4:4939 , piece_id=ANA3EZU7OZWELNG4QEMTEAYZY5H3C3H4IJFT4XKPUB7WIF44FEVQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481718373 (377h14m24.408013051s) [noise=true , address=45.91.116.5:4297 , piece_id=MSDWDLY5PJHQVI5565EKANJVSTMZFEUD2OTYZ52C65XQUICNMC3A, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481930356 (377h14m24.408225033s) [noise=true , address=77.90.31.17:4770 , piece_id=V36UBBIYOFJC6KFVVBXEVT4QBUIWN257D3YSDDRAGZVXH7ZXHTBA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.482376147 (377h14m24.408670821s) [noise=true , address=45.91.116.8:4045 , piece_id=KM6ZYTHR6JGTARKZQZYIESPKBEU5H2HQFA7OARIKD2ZXDHPFSMGA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.482439192 (377h14m24.408733868s) [noise=true , address=45.91.116.6:4973 , piece_id=Z7HQQ2O5QUSJKT4EMIDMEI5PIAEHNVRYX7MALUR2GQULRFIBUOKA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.483047970 (377h14m24.409342644s) [noise=true , address=77.90.31.13:4627 , piece_id=HULEKFVZNDNKS4SWWGTJUHYZWWYPDCER5W4GFCVLLDL65PAHULFQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.497788632 (377h14m24.424083312s) [noise=true , address=77.90.31.11:4553 , piece_id=TYW7NBJWACIT7OQS7A4WGXUPR3MB2FKAN2OMGAZJERB4X2DLAO2Q, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:16.688921171 (377h14m43.615215851s) [noise=true , address=104.36.82.6:4875 , piece_id=C275F2JPVAF52E34CEHT5QXCHUFGIQZ7XGMOAOUOT4RA7YOHRQLQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:27.177893955 (377h14m54.104188635s) [noise=true , address=104.36.82.2:4732 , piece_id=DJIGHJMJRAULH2L4YT2WGRXPRQJMNKZG5LICHPSKOCHUL6QDV6BQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:27.177985584 (377h14m54.104280281s) [seg_pos={0 0}, upload=302109]: Segment reached optimal threshold of 13 pieces.
Jul 9 02:26:27.178189789 (377h14m54.104484461s) [noise=true , address=104.36.82.5:4838 , piece_id=WK3ALBOEMWWTVEA4FKCS352SZ6YYB35LELURNWXQZK32K4HYDRNA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178252042 (377h14m54.104546733s) [noise=true , address=104.36.82.7:4909 , piece_id=ZRHLWDQ5KIDRWPJ5FXAANQTWERKXCR2PPEGN442IOSYSRFAG7VZA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178273922 (377h14m54.104568608s) [noise=true , address=104.36.82.10:4078 , piece_id=ZFVUQAEV6DDFLPKCZA2XJ2BZHMMM745LGB4JRZJZQPG72WECGMDQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178305983 (377h14m54.104600682s) [seg_pos={0 0}, upload=302109]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 02:26:27.178428768 (377h14m54.104723465s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:26:27.178455810 (377h14m54.104750501s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:26:27.232166184 (377h14m54.158460883s) [upload=302109]: Done uploading segments...
Jul 9 02:26:27.232188444 (377h14m54.158483123s) [upload=302109]: Upload finished. err: <nil>
Duration: 457.041603ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
19 out of 60 is failed (threshold=3) [X.....XXX.XX.X...XXX....XX...X..X..X.XX.X..X................] duration:
498.256047ms
FAIL (2025-07-09 03:06:14 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 498.290732ms
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
545.090366ms
FAIL (2025-07-09 03:06:00 UTC)
orbiter warning:
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
02:25:27.135824454 (377h13m54.062119141s) [noise=true , address=104.36.82.12:4150 , piece_id=DRG5SBA3EE4KVQPEQOJZ6SS537SFWYFUYQXOAQMWYTT6FPVSJIPQ, seg_pos={0 0}, upload=302099]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: context canceled; context canceled
Jul 9 02:25:27.135860079 (377h13m54.062154769s) [seg_pos={0 0}, upload=302099]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 02:25:27.135957283 (377h13m54.062251971s) [upload=302099]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:25:27.135964211 (377h13m54.062258897s) [upload=302099]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:25:27.199410713 (377h13m54.125705389s) [upload=302099]: Done uploading segments...
Jul 9 02:25:27.199427901 (377h13m54.125722577s) [upload=302099]: Upload finished. err: <nil>
Jul 9 02:25:57.289077655 (377h14m24.215372329s) [upload=302109]: Starting upload
Jul 9 02:25:57.289186750 (377h14m24.215481424s) [upload=302109]: Done starting upload
Jul 9 02:25:57.289444851 (377h14m24.215739548s) [upload=302109]: Uploading segments...
Jul 9 02:25:57.289623846 (377h14m24.215918535s) [upload=302109]: Got next segment. Inline: false
Jul 9 02:25:57.289696741 (377h14m24.215991432s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:25:57.289703375 (377h14m24.215998067s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:25:57.318313152 (377h14m24.244607838s) [seg_pos={0 0}, upload=302109]: Begin upload segment...
Jul 9 02:25:57.318363286 (377h14m24.244657964s) [seg_pos={0 0}, upload=302109]: Done begin upload segment.
Jul 9 02:25:57.318386772 (377h14m24.244681463s) [noise=true , address=77.90.31.11:4553 , piece_id=TYW7NBJWACIT7OQS7A4WGXUPR3MB2FKAN2OMGAZJERB4X2DLAO2Q, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318412127 (377h14m24.244706825s) [noise=true , address=77.90.31.10:4518 , piece_id=VJ4HWM2X4VSRSUCQL5QLMBYPFBYV4JYS5ZXCBCB6MTMB76EHZFGQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318822149 (377h14m24.245116841s) [noise=true , address=45.91.116.9:4263 , piece_id=5YILAJ5BZBGOURSZE343ST4RMOTWM6ZOB3EEQQWL5WUNYWOGSAPA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318965862 (377h14m24.24526056s) [noise=true , address=45.91.116.8:4045 , piece_id=KM6ZYTHR6JGTARKZQZYIESPKBEU5H2HQFA7OARIKD2ZXDHPFSMGA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.319187516 (377h14m24.245482203s) [noise=true , address=104.36.82.7:4909 , piece_id=ZRHLWDQ5KIDRWPJ5FXAANQTWERKXCR2PPEGN442IOSYSRFAG7VZA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.319523815 (377h14m24.245818512s) [noise=true , address=45.91.116.14:4191 , piece_id=GTR6T4OTGZTEZ76FWQZ46OCH5WB3O5X27EN4AV33ZJ7D5HX4535A, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.320556632 (377h14m24.24685133s) [noise=true , address=104.36.82.2:4732 , piece_id=DJIGHJMJRAULH2L4YT2WGRXPRQJMNKZG5LICHPSKOCHUL6QDV6BQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.321019141 (377h14m24.247313835s) [noise=true , address=77.90.31.17:4770 , piece_id=V36UBBIYOFJC6KFVVBXEVT4QBUIWN257D3YSDDRAGZVXH7ZXHTBA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.322591797 (377h14m24.248886493s) [noise=true , address=77.90.31.13:4627 , piece_id=HULEKFVZNDNKS4SWWGTJUHYZWWYPDCER5W4GFCVLLDL65PAHULFQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323120451 (377h14m24.249415138s) [noise=true , address=45.91.116.4:4939 , piece_id=ANA3EZU7OZWELNG4QEMTEAYZY5H3C3H4IJFT4XKPUB7WIF44FEVQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323544591 (377h14m24.249839288s) [noise=true , address=45.91.116.7:4011 , piece_id=VDOJRN6ACK4LO44BQTULO6BB3CLAVQI7YIC6XLR32CZV4UJSPE6Q, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323950189 (377h14m24.250244889s) [noise=true , address=104.36.82.10:4078 , piece_id=ZFVUQAEV6DDFLPKCZA2XJ2BZHMMM745LGB4JRZJZQPG72WECGMDQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.326585810 (377h14m24.252880507s) [noise=true , address=45.91.116.5:4297 , piece_id=MSDWDLY5PJHQVI5565EKANJVSTMZFEUD2OTYZ52C65XQUICNMC3A, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.327871214 (377h14m24.254165904s) [noise=true , address=104.36.82.6:4875 , piece_id=C275F2JPVAF52E34CEHT5QXCHUFGIQZ7XGMOAOUOT4RA7YOHRQLQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.328605288 (377h14m24.254899986s) [noise=true , address=104.36.82.5:4838 , piece_id=WK3ALBOEMWWTVEA4FKCS352SZ6YYB35LELURNWXQZK32K4HYDRNA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.335409087 (377h14m24.261703784s) [noise=true , address=45.91.116.6:4973 , piece_id=Z7HQQ2O5QUSJKT4EMIDMEI5PIAEHNVRYX7MALUR2GQULRFIBUOKA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.475344637 (377h14m24.401639326s) [upload=302109]: Next returned nil segment
Jul 9 02:25:57.475376181 (377h14m24.401670879s) [upload=302109]: Waiting for error group managing segments...
Jul 9 02:25:57.477269278 (377h14m24.403563958s) [noise=true , address=45.91.116.7:4011 , piece_id=VDOJRN6ACK4LO44BQTULO6BB3CLAVQI7YIC6XLR32CZV4UJSPE6Q, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.477736179 (377h14m24.40403086s) [noise=true , address=45.91.116.9:4263 , piece_id=5YILAJ5BZBGOURSZE343ST4RMOTWM6ZOB3EEQQWL5WUNYWOGSAPA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.478396652 (377h14m24.404691332s) [noise=true , address=77.90.31.10:4518 , piece_id=VJ4HWM2X4VSRSUCQL5QLMBYPFBYV4JYS5ZXCBCB6MTMB76EHZFGQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.478919538 (377h14m24.405214224s) [noise=true , address=45.91.116.14:4191 , piece_id=GTR6T4OTGZTEZ76FWQZ46OCH5WB3O5X27EN4AV33ZJ7D5HX4535A, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481426716 (377h14m24.407721396s) [noise=true , address=45.91.116.4:4939 , piece_id=ANA3EZU7OZWELNG4QEMTEAYZY5H3C3H4IJFT4XKPUB7WIF44FEVQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481718373 (377h14m24.408013051s) [noise=true , address=45.91.116.5:4297 , piece_id=MSDWDLY5PJHQVI5565EKANJVSTMZFEUD2OTYZ52C65XQUICNMC3A, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481930356 (377h14m24.408225033s) [noise=true , address=77.90.31.17:4770 , piece_id=V36UBBIYOFJC6KFVVBXEVT4QBUIWN257D3YSDDRAGZVXH7ZXHTBA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.482376147 (377h14m24.408670821s) [noise=true , address=45.91.116.8:4045 , piece_id=KM6ZYTHR6JGTARKZQZYIESPKBEU5H2HQFA7OARIKD2ZXDHPFSMGA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.482439192 (377h14m24.408733868s) [noise=true , address=45.91.116.6:4973 , piece_id=Z7HQQ2O5QUSJKT4EMIDMEI5PIAEHNVRYX7MALUR2GQULRFIBUOKA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.483047970 (377h14m24.409342644s) [noise=true , address=77.90.31.13:4627 , piece_id=HULEKFVZNDNKS4SWWGTJUHYZWWYPDCER5W4GFCVLLDL65PAHULFQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.497788632 (377h14m24.424083312s) [noise=true , address=77.90.31.11:4553 , piece_id=TYW7NBJWACIT7OQS7A4WGXUPR3MB2FKAN2OMGAZJERB4X2DLAO2Q, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:16.688921171 (377h14m43.615215851s) [noise=true , address=104.36.82.6:4875 , piece_id=C275F2JPVAF52E34CEHT5QXCHUFGIQZ7XGMOAOUOT4RA7YOHRQLQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:27.177893955 (377h14m54.104188635s) [noise=true , address=104.36.82.2:4732 , piece_id=DJIGHJMJRAULH2L4YT2WGRXPRQJMNKZG5LICHPSKOCHUL6QDV6BQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:27.177985584 (377h14m54.104280281s) [seg_pos={0 0}, upload=302109]: Segment reached optimal threshold of 13 pieces.
Jul 9 02:26:27.178189789 (377h14m54.104484461s) [noise=true , address=104.36.82.5:4838 , piece_id=WK3ALBOEMWWTVEA4FKCS352SZ6YYB35LELURNWXQZK32K4HYDRNA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178252042 (377h14m54.104546733s) [noise=true , address=104.36.82.7:4909 , piece_id=ZRHLWDQ5KIDRWPJ5FXAANQTWERKXCR2PPEGN442IOSYSRFAG7VZA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178273922 (377h14m54.104568608s) [noise=true , address=104.36.82.10:4078 , piece_id=ZFVUQAEV6DDFLPKCZA2XJ2BZHMMM745LGB4JRZJZQPG72WECGMDQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178305983 (377h14m54.104600682s) [seg_pos={0 0}, upload=302109]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 02:26:27.178428768 (377h14m54.104723465s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:26:27.178455810 (377h14m54.104750501s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:26:27.232166184 (377h14m54.158460883s) [upload=302109]: Done uploading segments...
Jul 9 02:26:27.232188444 (377h14m54.158483123s) [upload=302109]: Upload finished. err: <nil>
Duration: 545.112798ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
31 out of 60 is failed (threshold=3) [.XX.XXXXXX.XXXXX.XX.....X..XXXXX.XX..XXX.XX.XX..X...........] duration:
827.660411ms
FAIL (2025-07-09 03:05:38 UTC)
operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 827.691281ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
418.022911ms
FAIL (2025-07-09 03:05:22 UTC)
orbiter warning:
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
02:25:27.135824454 (377h13m54.062119141s) [noise=true , address=104.36.82.12:4150 , piece_id=DRG5SBA3EE4KVQPEQOJZ6SS537SFWYFUYQXOAQMWYTT6FPVSJIPQ, seg_pos={0 0}, upload=302099]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: context canceled; context canceled
Jul 9 02:25:27.135860079 (377h13m54.062154769s) [seg_pos={0 0}, upload=302099]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 02:25:27.135957283 (377h13m54.062251971s) [upload=302099]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:25:27.135964211 (377h13m54.062258897s) [upload=302099]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:25:27.199410713 (377h13m54.125705389s) [upload=302099]: Done uploading segments...
Jul 9 02:25:27.199427901 (377h13m54.125722577s) [upload=302099]: Upload finished. err: <nil>
Jul 9 02:25:57.289077655 (377h14m24.215372329s) [upload=302109]: Starting upload
Jul 9 02:25:57.289186750 (377h14m24.215481424s) [upload=302109]: Done starting upload
Jul 9 02:25:57.289444851 (377h14m24.215739548s) [upload=302109]: Uploading segments...
Jul 9 02:25:57.289623846 (377h14m24.215918535s) [upload=302109]: Got next segment. Inline: false
Jul 9 02:25:57.289696741 (377h14m24.215991432s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:25:57.289703375 (377h14m24.215998067s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:25:57.318313152 (377h14m24.244607838s) [seg_pos={0 0}, upload=302109]: Begin upload segment...
Jul 9 02:25:57.318363286 (377h14m24.244657964s) [seg_pos={0 0}, upload=302109]: Done begin upload segment.
Jul 9 02:25:57.318386772 (377h14m24.244681463s) [noise=true , address=77.90.31.11:4553 , piece_id=TYW7NBJWACIT7OQS7A4WGXUPR3MB2FKAN2OMGAZJERB4X2DLAO2Q, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318412127 (377h14m24.244706825s) [noise=true , address=77.90.31.10:4518 , piece_id=VJ4HWM2X4VSRSUCQL5QLMBYPFBYV4JYS5ZXCBCB6MTMB76EHZFGQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318822149 (377h14m24.245116841s) [noise=true , address=45.91.116.9:4263 , piece_id=5YILAJ5BZBGOURSZE343ST4RMOTWM6ZOB3EEQQWL5WUNYWOGSAPA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318965862 (377h14m24.24526056s) [noise=true , address=45.91.116.8:4045 , piece_id=KM6ZYTHR6JGTARKZQZYIESPKBEU5H2HQFA7OARIKD2ZXDHPFSMGA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.319187516 (377h14m24.245482203s) [noise=true , address=104.36.82.7:4909 , piece_id=ZRHLWDQ5KIDRWPJ5FXAANQTWERKXCR2PPEGN442IOSYSRFAG7VZA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.319523815 (377h14m24.245818512s) [noise=true , address=45.91.116.14:4191 , piece_id=GTR6T4OTGZTEZ76FWQZ46OCH5WB3O5X27EN4AV33ZJ7D5HX4535A, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.320556632 (377h14m24.24685133s) [noise=true , address=104.36.82.2:4732 , piece_id=DJIGHJMJRAULH2L4YT2WGRXPRQJMNKZG5LICHPSKOCHUL6QDV6BQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.321019141 (377h14m24.247313835s) [noise=true , address=77.90.31.17:4770 , piece_id=V36UBBIYOFJC6KFVVBXEVT4QBUIWN257D3YSDDRAGZVXH7ZXHTBA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.322591797 (377h14m24.248886493s) [noise=true , address=77.90.31.13:4627 , piece_id=HULEKFVZNDNKS4SWWGTJUHYZWWYPDCER5W4GFCVLLDL65PAHULFQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323120451 (377h14m24.249415138s) [noise=true , address=45.91.116.4:4939 , piece_id=ANA3EZU7OZWELNG4QEMTEAYZY5H3C3H4IJFT4XKPUB7WIF44FEVQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323544591 (377h14m24.249839288s) [noise=true , address=45.91.116.7:4011 , piece_id=VDOJRN6ACK4LO44BQTULO6BB3CLAVQI7YIC6XLR32CZV4UJSPE6Q, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323950189 (377h14m24.250244889s) [noise=true , address=104.36.82.10:4078 , piece_id=ZFVUQAEV6DDFLPKCZA2XJ2BZHMMM745LGB4JRZJZQPG72WECGMDQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.326585810 (377h14m24.252880507s) [noise=true , address=45.91.116.5:4297 , piece_id=MSDWDLY5PJHQVI5565EKANJVSTMZFEUD2OTYZ52C65XQUICNMC3A, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.327871214 (377h14m24.254165904s) [noise=true , address=104.36.82.6:4875 , piece_id=C275F2JPVAF52E34CEHT5QXCHUFGIQZ7XGMOAOUOT4RA7YOHRQLQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.328605288 (377h14m24.254899986s) [noise=true , address=104.36.82.5:4838 , piece_id=WK3ALBOEMWWTVEA4FKCS352SZ6YYB35LELURNWXQZK32K4HYDRNA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.335409087 (377h14m24.261703784s) [noise=true , address=45.91.116.6:4973 , piece_id=Z7HQQ2O5QUSJKT4EMIDMEI5PIAEHNVRYX7MALUR2GQULRFIBUOKA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.475344637 (377h14m24.401639326s) [upload=302109]: Next returned nil segment
Jul 9 02:25:57.475376181 (377h14m24.401670879s) [upload=302109]: Waiting for error group managing segments...
Jul 9 02:25:57.477269278 (377h14m24.403563958s) [noise=true , address=45.91.116.7:4011 , piece_id=VDOJRN6ACK4LO44BQTULO6BB3CLAVQI7YIC6XLR32CZV4UJSPE6Q, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.477736179 (377h14m24.40403086s) [noise=true , address=45.91.116.9:4263 , piece_id=5YILAJ5BZBGOURSZE343ST4RMOTWM6ZOB3EEQQWL5WUNYWOGSAPA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.478396652 (377h14m24.404691332s) [noise=true , address=77.90.31.10:4518 , piece_id=VJ4HWM2X4VSRSUCQL5QLMBYPFBYV4JYS5ZXCBCB6MTMB76EHZFGQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.478919538 (377h14m24.405214224s) [noise=true , address=45.91.116.14:4191 , piece_id=GTR6T4OTGZTEZ76FWQZ46OCH5WB3O5X27EN4AV33ZJ7D5HX4535A, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481426716 (377h14m24.407721396s) [noise=true , address=45.91.116.4:4939 , piece_id=ANA3EZU7OZWELNG4QEMTEAYZY5H3C3H4IJFT4XKPUB7WIF44FEVQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481718373 (377h14m24.408013051s) [noise=true , address=45.91.116.5:4297 , piece_id=MSDWDLY5PJHQVI5565EKANJVSTMZFEUD2OTYZ52C65XQUICNMC3A, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481930356 (377h14m24.408225033s) [noise=true , address=77.90.31.17:4770 , piece_id=V36UBBIYOFJC6KFVVBXEVT4QBUIWN257D3YSDDRAGZVXH7ZXHTBA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.482376147 (377h14m24.408670821s) [noise=true , address=45.91.116.8:4045 , piece_id=KM6ZYTHR6JGTARKZQZYIESPKBEU5H2HQFA7OARIKD2ZXDHPFSMGA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.482439192 (377h14m24.408733868s) [noise=true , address=45.91.116.6:4973 , piece_id=Z7HQQ2O5QUSJKT4EMIDMEI5PIAEHNVRYX7MALUR2GQULRFIBUOKA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.483047970 (377h14m24.409342644s) [noise=true , address=77.90.31.13:4627 , piece_id=HULEKFVZNDNKS4SWWGTJUHYZWWYPDCER5W4GFCVLLDL65PAHULFQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.497788632 (377h14m24.424083312s) [noise=true , address=77.90.31.11:4553 , piece_id=TYW7NBJWACIT7OQS7A4WGXUPR3MB2FKAN2OMGAZJERB4X2DLAO2Q, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:16.688921171 (377h14m43.615215851s) [noise=true , address=104.36.82.6:4875 , piece_id=C275F2JPVAF52E34CEHT5QXCHUFGIQZ7XGMOAOUOT4RA7YOHRQLQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:27.177893955 (377h14m54.104188635s) [noise=true , address=104.36.82.2:4732 , piece_id=DJIGHJMJRAULH2L4YT2WGRXPRQJMNKZG5LICHPSKOCHUL6QDV6BQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:27.177985584 (377h14m54.104280281s) [seg_pos={0 0}, upload=302109]: Segment reached optimal threshold of 13 pieces.
Jul 9 02:26:27.178189789 (377h14m54.104484461s) [noise=true , address=104.36.82.5:4838 , piece_id=WK3ALBOEMWWTVEA4FKCS352SZ6YYB35LELURNWXQZK32K4HYDRNA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178252042 (377h14m54.104546733s) [noise=true , address=104.36.82.7:4909 , piece_id=ZRHLWDQ5KIDRWPJ5FXAANQTWERKXCR2PPEGN442IOSYSRFAG7VZA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178273922 (377h14m54.104568608s) [noise=true , address=104.36.82.10:4078 , piece_id=ZFVUQAEV6DDFLPKCZA2XJ2BZHMMM745LGB4JRZJZQPG72WECGMDQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178305983 (377h14m54.104600682s) [seg_pos={0 0}, upload=302109]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 02:26:27.178428768 (377h14m54.104723465s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:26:27.178455810 (377h14m54.104750501s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:26:27.232166184 (377h14m54.158460883s) [upload=302109]: Done uploading segments...
Jul 9 02:26:27.232188444 (377h14m54.158483123s) [upload=302109]: Upload finished. err: <nil>
Duration: 418.055142ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
19 out of 60 is failed (threshold=3) [.X.....XXX.XX.X...XXX....XX...X..X..X.XX.X..X...............] duration:
554.463952ms
FAIL (2025-07-09 03:05:14 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 554.487822ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
32 out of 60 is failed (threshold=3) [X.XX.XXXXXX.XXXXX.XX.....X..XXXXX.XX..XXX.XX.XX..X..........] duration:
2.142983424s
FAIL (2025-07-09 03:04:43 UTC)
operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.143014508s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
20 out of 60 is failed (threshold=3) [X.X.....XXX.XX.X...XXX....XX...X..X..X.XX.X..X..............] duration:
524.824654ms
FAIL (2025-07-09 03:04:23 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 524.893842ms
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................] duration:
1.921389006s
FAIL (2025-07-09 03:04:13 UTC)
orbiter warning:
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
02:25:27.135824454 (377h13m54.062119141s) [noise=true , address=104.36.82.12:4150 , piece_id=DRG5SBA3EE4KVQPEQOJZ6SS537SFWYFUYQXOAQMWYTT6FPVSJIPQ, seg_pos={0 0}, upload=302099]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: context canceled; context canceled
Jul 9 02:25:27.135860079 (377h13m54.062154769s) [seg_pos={0 0}, upload=302099]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 02:25:27.135957283 (377h13m54.062251971s) [upload=302099]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:25:27.135964211 (377h13m54.062258897s) [upload=302099]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:25:27.199410713 (377h13m54.125705389s) [upload=302099]: Done uploading segments...
Jul 9 02:25:27.199427901 (377h13m54.125722577s) [upload=302099]: Upload finished. err: <nil>
Jul 9 02:25:57.289077655 (377h14m24.215372329s) [upload=302109]: Starting upload
Jul 9 02:25:57.289186750 (377h14m24.215481424s) [upload=302109]: Done starting upload
Jul 9 02:25:57.289444851 (377h14m24.215739548s) [upload=302109]: Uploading segments...
Jul 9 02:25:57.289623846 (377h14m24.215918535s) [upload=302109]: Got next segment. Inline: false
Jul 9 02:25:57.289696741 (377h14m24.215991432s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:25:57.289703375 (377h14m24.215998067s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:25:57.318313152 (377h14m24.244607838s) [seg_pos={0 0}, upload=302109]: Begin upload segment...
Jul 9 02:25:57.318363286 (377h14m24.244657964s) [seg_pos={0 0}, upload=302109]: Done begin upload segment.
Jul 9 02:25:57.318386772 (377h14m24.244681463s) [noise=true , address=77.90.31.11:4553 , piece_id=TYW7NBJWACIT7OQS7A4WGXUPR3MB2FKAN2OMGAZJERB4X2DLAO2Q, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318412127 (377h14m24.244706825s) [noise=true , address=77.90.31.10:4518 , piece_id=VJ4HWM2X4VSRSUCQL5QLMBYPFBYV4JYS5ZXCBCB6MTMB76EHZFGQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318822149 (377h14m24.245116841s) [noise=true , address=45.91.116.9:4263 , piece_id=5YILAJ5BZBGOURSZE343ST4RMOTWM6ZOB3EEQQWL5WUNYWOGSAPA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318965862 (377h14m24.24526056s) [noise=true , address=45.91.116.8:4045 , piece_id=KM6ZYTHR6JGTARKZQZYIESPKBEU5H2HQFA7OARIKD2ZXDHPFSMGA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.319187516 (377h14m24.245482203s) [noise=true , address=104.36.82.7:4909 , piece_id=ZRHLWDQ5KIDRWPJ5FXAANQTWERKXCR2PPEGN442IOSYSRFAG7VZA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.319523815 (377h14m24.245818512s) [noise=true , address=45.91.116.14:4191 , piece_id=GTR6T4OTGZTEZ76FWQZ46OCH5WB3O5X27EN4AV33ZJ7D5HX4535A, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.320556632 (377h14m24.24685133s) [noise=true , address=104.36.82.2:4732 , piece_id=DJIGHJMJRAULH2L4YT2WGRXPRQJMNKZG5LICHPSKOCHUL6QDV6BQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.321019141 (377h14m24.247313835s) [noise=true , address=77.90.31.17:4770 , piece_id=V36UBBIYOFJC6KFVVBXEVT4QBUIWN257D3YSDDRAGZVXH7ZXHTBA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.322591797 (377h14m24.248886493s) [noise=true , address=77.90.31.13:4627 , piece_id=HULEKFVZNDNKS4SWWGTJUHYZWWYPDCER5W4GFCVLLDL65PAHULFQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323120451 (377h14m24.249415138s) [noise=true , address=45.91.116.4:4939 , piece_id=ANA3EZU7OZWELNG4QEMTEAYZY5H3C3H4IJFT4XKPUB7WIF44FEVQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323544591 (377h14m24.249839288s) [noise=true , address=45.91.116.7:4011 , piece_id=VDOJRN6ACK4LO44BQTULO6BB3CLAVQI7YIC6XLR32CZV4UJSPE6Q, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323950189 (377h14m24.250244889s) [noise=true , address=104.36.82.10:4078 , piece_id=ZFVUQAEV6DDFLPKCZA2XJ2BZHMMM745LGB4JRZJZQPG72WECGMDQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.326585810 (377h14m24.252880507s) [noise=true , address=45.91.116.5:4297 , piece_id=MSDWDLY5PJHQVI5565EKANJVSTMZFEUD2OTYZ52C65XQUICNMC3A, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.327871214 (377h14m24.254165904s) [noise=true , address=104.36.82.6:4875 , piece_id=C275F2JPVAF52E34CEHT5QXCHUFGIQZ7XGMOAOUOT4RA7YOHRQLQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.328605288 (377h14m24.254899986s) [noise=true , address=104.36.82.5:4838 , piece_id=WK3ALBOEMWWTVEA4FKCS352SZ6YYB35LELURNWXQZK32K4HYDRNA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.335409087 (377h14m24.261703784s) [noise=true , address=45.91.116.6:4973 , piece_id=Z7HQQ2O5QUSJKT4EMIDMEI5PIAEHNVRYX7MALUR2GQULRFIBUOKA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.475344637 (377h14m24.401639326s) [upload=302109]: Next returned nil segment
Jul 9 02:25:57.475376181 (377h14m24.401670879s) [upload=302109]: Waiting for error group managing segments...
Jul 9 02:25:57.477269278 (377h14m24.403563958s) [noise=true , address=45.91.116.7:4011 , piece_id=VDOJRN6ACK4LO44BQTULO6BB3CLAVQI7YIC6XLR32CZV4UJSPE6Q, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.477736179 (377h14m24.40403086s) [noise=true , address=45.91.116.9:4263 , piece_id=5YILAJ5BZBGOURSZE343ST4RMOTWM6ZOB3EEQQWL5WUNYWOGSAPA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.478396652 (377h14m24.404691332s) [noise=true , address=77.90.31.10:4518 , piece_id=VJ4HWM2X4VSRSUCQL5QLMBYPFBYV4JYS5ZXCBCB6MTMB76EHZFGQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.478919538 (377h14m24.405214224s) [noise=true , address=45.91.116.14:4191 , piece_id=GTR6T4OTGZTEZ76FWQZ46OCH5WB3O5X27EN4AV33ZJ7D5HX4535A, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481426716 (377h14m24.407721396s) [noise=true , address=45.91.116.4:4939 , piece_id=ANA3EZU7OZWELNG4QEMTEAYZY5H3C3H4IJFT4XKPUB7WIF44FEVQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481718373 (377h14m24.408013051s) [noise=true , address=45.91.116.5:4297 , piece_id=MSDWDLY5PJHQVI5565EKANJVSTMZFEUD2OTYZ52C65XQUICNMC3A, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481930356 (377h14m24.408225033s) [noise=true , address=77.90.31.17:4770 , piece_id=V36UBBIYOFJC6KFVVBXEVT4QBUIWN257D3YSDDRAGZVXH7ZXHTBA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.482376147 (377h14m24.408670821s) [noise=true , address=45.91.116.8:4045 , piece_id=KM6ZYTHR6JGTARKZQZYIESPKBEU5H2HQFA7OARIKD2ZXDHPFSMGA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.482439192 (377h14m24.408733868s) [noise=true , address=45.91.116.6:4973 , piece_id=Z7HQQ2O5QUSJKT4EMIDMEI5PIAEHNVRYX7MALUR2GQULRFIBUOKA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.483047970 (377h14m24.409342644s) [noise=true , address=77.90.31.13:4627 , piece_id=HULEKFVZNDNKS4SWWGTJUHYZWWYPDCER5W4GFCVLLDL65PAHULFQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.497788632 (377h14m24.424083312s) [noise=true , address=77.90.31.11:4553 , piece_id=TYW7NBJWACIT7OQS7A4WGXUPR3MB2FKAN2OMGAZJERB4X2DLAO2Q, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:16.688921171 (377h14m43.615215851s) [noise=true , address=104.36.82.6:4875 , piece_id=C275F2JPVAF52E34CEHT5QXCHUFGIQZ7XGMOAOUOT4RA7YOHRQLQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:27.177893955 (377h14m54.104188635s) [noise=true , address=104.36.82.2:4732 , piece_id=DJIGHJMJRAULH2L4YT2WGRXPRQJMNKZG5LICHPSKOCHUL6QDV6BQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:27.177985584 (377h14m54.104280281s) [seg_pos={0 0}, upload=302109]: Segment reached optimal threshold of 13 pieces.
Jul 9 02:26:27.178189789 (377h14m54.104484461s) [noise=true , address=104.36.82.5:4838 , piece_id=WK3ALBOEMWWTVEA4FKCS352SZ6YYB35LELURNWXQZK32K4HYDRNA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178252042 (377h14m54.104546733s) [noise=true , address=104.36.82.7:4909 , piece_id=ZRHLWDQ5KIDRWPJ5FXAANQTWERKXCR2PPEGN442IOSYSRFAG7VZA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178273922 (377h14m54.104568608s) [noise=true , address=104.36.82.10:4078 , piece_id=ZFVUQAEV6DDFLPKCZA2XJ2BZHMMM745LGB4JRZJZQPG72WECGMDQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178305983 (377h14m54.104600682s) [seg_pos={0 0}, upload=302109]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 02:26:27.178428768 (377h14m54.104723465s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:26:27.178455810 (377h14m54.104750501s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:26:27.232166184 (377h14m54.158460883s) [upload=302109]: Done uploading segments...
Jul 9 02:26:27.232188444 (377h14m54.158483123s) [upload=302109]: Upload finished. err: <nil>
Duration: 1.921423954s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
33 out of 60 is failed (threshold=3) [XX.XX.XXXXXX.XXXXX.XX.....X..XXXXX.XX..XXX.XX.XX..X.........] duration:
671.533917ms
FAIL (2025-07-09 03:04:00 UTC)
operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 671.622604ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
20 out of 60 is failed (threshold=3) [.X.X.....XXX.XX.X...XXX....XX...X..X..X.XX.X..X.............] duration:
963.888704ms
FAIL (2025-07-09 03:03:48 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 963.909165ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
21 out of 60 is failed (threshold=3) [X.X.X.....XXX.XX.X...XXX....XX...X..X..X.XX.X..X............] duration:
517.788517ms
FAIL (2025-07-09 03:03:14 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 517.808833ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
34 out of 60 is failed (threshold=3) [XXX.XX.XXXXXX.XXXXX.XX.....X..XXXXX.XX..XXX.XX.XX..X........] duration:
648.220099ms
FAIL (2025-07-09 03:03:09 UTC)
operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 648.2549ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
438.270474ms
FAIL (2025-07-09 03:03:07 UTC)
orbiter warning:
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
02:25:27.135824454 (377h13m54.062119141s) [noise=true , address=104.36.82.12:4150 , piece_id=DRG5SBA3EE4KVQPEQOJZ6SS537SFWYFUYQXOAQMWYTT6FPVSJIPQ, seg_pos={0 0}, upload=302099]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: context canceled; context canceled
Jul 9 02:25:27.135860079 (377h13m54.062154769s) [seg_pos={0 0}, upload=302099]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 02:25:27.135957283 (377h13m54.062251971s) [upload=302099]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:25:27.135964211 (377h13m54.062258897s) [upload=302099]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:25:27.199410713 (377h13m54.125705389s) [upload=302099]: Done uploading segments...
Jul 9 02:25:27.199427901 (377h13m54.125722577s) [upload=302099]: Upload finished. err: <nil>
Jul 9 02:25:57.289077655 (377h14m24.215372329s) [upload=302109]: Starting upload
Jul 9 02:25:57.289186750 (377h14m24.215481424s) [upload=302109]: Done starting upload
Jul 9 02:25:57.289444851 (377h14m24.215739548s) [upload=302109]: Uploading segments...
Jul 9 02:25:57.289623846 (377h14m24.215918535s) [upload=302109]: Got next segment. Inline: false
Jul 9 02:25:57.289696741 (377h14m24.215991432s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:25:57.289703375 (377h14m24.215998067s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:25:57.318313152 (377h14m24.244607838s) [seg_pos={0 0}, upload=302109]: Begin upload segment...
Jul 9 02:25:57.318363286 (377h14m24.244657964s) [seg_pos={0 0}, upload=302109]: Done begin upload segment.
Jul 9 02:25:57.318386772 (377h14m24.244681463s) [noise=true , address=77.90.31.11:4553 , piece_id=TYW7NBJWACIT7OQS7A4WGXUPR3MB2FKAN2OMGAZJERB4X2DLAO2Q, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318412127 (377h14m24.244706825s) [noise=true , address=77.90.31.10:4518 , piece_id=VJ4HWM2X4VSRSUCQL5QLMBYPFBYV4JYS5ZXCBCB6MTMB76EHZFGQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318822149 (377h14m24.245116841s) [noise=true , address=45.91.116.9:4263 , piece_id=5YILAJ5BZBGOURSZE343ST4RMOTWM6ZOB3EEQQWL5WUNYWOGSAPA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318965862 (377h14m24.24526056s) [noise=true , address=45.91.116.8:4045 , piece_id=KM6ZYTHR6JGTARKZQZYIESPKBEU5H2HQFA7OARIKD2ZXDHPFSMGA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.319187516 (377h14m24.245482203s) [noise=true , address=104.36.82.7:4909 , piece_id=ZRHLWDQ5KIDRWPJ5FXAANQTWERKXCR2PPEGN442IOSYSRFAG7VZA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.319523815 (377h14m24.245818512s) [noise=true , address=45.91.116.14:4191 , piece_id=GTR6T4OTGZTEZ76FWQZ46OCH5WB3O5X27EN4AV33ZJ7D5HX4535A, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.320556632 (377h14m24.24685133s) [noise=true , address=104.36.82.2:4732 , piece_id=DJIGHJMJRAULH2L4YT2WGRXPRQJMNKZG5LICHPSKOCHUL6QDV6BQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.321019141 (377h14m24.247313835s) [noise=true , address=77.90.31.17:4770 , piece_id=V36UBBIYOFJC6KFVVBXEVT4QBUIWN257D3YSDDRAGZVXH7ZXHTBA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.322591797 (377h14m24.248886493s) [noise=true , address=77.90.31.13:4627 , piece_id=HULEKFVZNDNKS4SWWGTJUHYZWWYPDCER5W4GFCVLLDL65PAHULFQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323120451 (377h14m24.249415138s) [noise=true , address=45.91.116.4:4939 , piece_id=ANA3EZU7OZWELNG4QEMTEAYZY5H3C3H4IJFT4XKPUB7WIF44FEVQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323544591 (377h14m24.249839288s) [noise=true , address=45.91.116.7:4011 , piece_id=VDOJRN6ACK4LO44BQTULO6BB3CLAVQI7YIC6XLR32CZV4UJSPE6Q, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323950189 (377h14m24.250244889s) [noise=true , address=104.36.82.10:4078 , piece_id=ZFVUQAEV6DDFLPKCZA2XJ2BZHMMM745LGB4JRZJZQPG72WECGMDQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.326585810 (377h14m24.252880507s) [noise=true , address=45.91.116.5:4297 , piece_id=MSDWDLY5PJHQVI5565EKANJVSTMZFEUD2OTYZ52C65XQUICNMC3A, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.327871214 (377h14m24.254165904s) [noise=true , address=104.36.82.6:4875 , piece_id=C275F2JPVAF52E34CEHT5QXCHUFGIQZ7XGMOAOUOT4RA7YOHRQLQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.328605288 (377h14m24.254899986s) [noise=true , address=104.36.82.5:4838 , piece_id=WK3ALBOEMWWTVEA4FKCS352SZ6YYB35LELURNWXQZK32K4HYDRNA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.335409087 (377h14m24.261703784s) [noise=true , address=45.91.116.6:4973 , piece_id=Z7HQQ2O5QUSJKT4EMIDMEI5PIAEHNVRYX7MALUR2GQULRFIBUOKA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.475344637 (377h14m24.401639326s) [upload=302109]: Next returned nil segment
Jul 9 02:25:57.475376181 (377h14m24.401670879s) [upload=302109]: Waiting for error group managing segments...
Jul 9 02:25:57.477269278 (377h14m24.403563958s) [noise=true , address=45.91.116.7:4011 , piece_id=VDOJRN6ACK4LO44BQTULO6BB3CLAVQI7YIC6XLR32CZV4UJSPE6Q, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.477736179 (377h14m24.40403086s) [noise=true , address=45.91.116.9:4263 , piece_id=5YILAJ5BZBGOURSZE343ST4RMOTWM6ZOB3EEQQWL5WUNYWOGSAPA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.478396652 (377h14m24.404691332s) [noise=true , address=77.90.31.10:4518 , piece_id=VJ4HWM2X4VSRSUCQL5QLMBYPFBYV4JYS5ZXCBCB6MTMB76EHZFGQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.478919538 (377h14m24.405214224s) [noise=true , address=45.91.116.14:4191 , piece_id=GTR6T4OTGZTEZ76FWQZ46OCH5WB3O5X27EN4AV33ZJ7D5HX4535A, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481426716 (377h14m24.407721396s) [noise=true , address=45.91.116.4:4939 , piece_id=ANA3EZU7OZWELNG4QEMTEAYZY5H3C3H4IJFT4XKPUB7WIF44FEVQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481718373 (377h14m24.408013051s) [noise=true , address=45.91.116.5:4297 , piece_id=MSDWDLY5PJHQVI5565EKANJVSTMZFEUD2OTYZ52C65XQUICNMC3A, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481930356 (377h14m24.408225033s) [noise=true , address=77.90.31.17:4770 , piece_id=V36UBBIYOFJC6KFVVBXEVT4QBUIWN257D3YSDDRAGZVXH7ZXHTBA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.482376147 (377h14m24.408670821s) [noise=true , address=45.91.116.8:4045 , piece_id=KM6ZYTHR6JGTARKZQZYIESPKBEU5H2HQFA7OARIKD2ZXDHPFSMGA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.482439192 (377h14m24.408733868s) [noise=true , address=45.91.116.6:4973 , piece_id=Z7HQQ2O5QUSJKT4EMIDMEI5PIAEHNVRYX7MALUR2GQULRFIBUOKA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.483047970 (377h14m24.409342644s) [noise=true , address=77.90.31.13:4627 , piece_id=HULEKFVZNDNKS4SWWGTJUHYZWWYPDCER5W4GFCVLLDL65PAHULFQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.497788632 (377h14m24.424083312s) [noise=true , address=77.90.31.11:4553 , piece_id=TYW7NBJWACIT7OQS7A4WGXUPR3MB2FKAN2OMGAZJERB4X2DLAO2Q, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:16.688921171 (377h14m43.615215851s) [noise=true , address=104.36.82.6:4875 , piece_id=C275F2JPVAF52E34CEHT5QXCHUFGIQZ7XGMOAOUOT4RA7YOHRQLQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:27.177893955 (377h14m54.104188635s) [noise=true , address=104.36.82.2:4732 , piece_id=DJIGHJMJRAULH2L4YT2WGRXPRQJMNKZG5LICHPSKOCHUL6QDV6BQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:27.177985584 (377h14m54.104280281s) [seg_pos={0 0}, upload=302109]: Segment reached optimal threshold of 13 pieces.
Jul 9 02:26:27.178189789 (377h14m54.104484461s) [noise=true , address=104.36.82.5:4838 , piece_id=WK3ALBOEMWWTVEA4FKCS352SZ6YYB35LELURNWXQZK32K4HYDRNA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178252042 (377h14m54.104546733s) [noise=true , address=104.36.82.7:4909 , piece_id=ZRHLWDQ5KIDRWPJ5FXAANQTWERKXCR2PPEGN442IOSYSRFAG7VZA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178273922 (377h14m54.104568608s) [noise=true , address=104.36.82.10:4078 , piece_id=ZFVUQAEV6DDFLPKCZA2XJ2BZHMMM745LGB4JRZJZQPG72WECGMDQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178305983 (377h14m54.104600682s) [seg_pos={0 0}, upload=302109]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 02:26:27.178428768 (377h14m54.104723465s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:26:27.178455810 (377h14m54.104750501s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:26:27.232166184 (377h14m54.158460883s) [upload=302109]: Done uploading segments...
Jul 9 02:26:27.232188444 (377h14m54.158483123s) [upload=302109]: Upload finished. err: <nil>
Duration: 438.300859ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
21 out of 60 is failed (threshold=3) [.X.X.X.....XXX.XX.X...XXX....XX...X..X..X.XX.X..X...........] duration:
463.245937ms
FAIL (2025-07-09 03:02:50 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 463.273664ms
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................] duration:
456.616153ms
FAIL (2025-07-09 03:02:40 UTC)
orbiter warning:
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
02:25:27.135824454 (377h13m54.062119141s) [noise=true , address=104.36.82.12:4150 , piece_id=DRG5SBA3EE4KVQPEQOJZ6SS537SFWYFUYQXOAQMWYTT6FPVSJIPQ, seg_pos={0 0}, upload=302099]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: context canceled; context canceled
Jul 9 02:25:27.135860079 (377h13m54.062154769s) [seg_pos={0 0}, upload=302099]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 02:25:27.135957283 (377h13m54.062251971s) [upload=302099]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:25:27.135964211 (377h13m54.062258897s) [upload=302099]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:25:27.199410713 (377h13m54.125705389s) [upload=302099]: Done uploading segments...
Jul 9 02:25:27.199427901 (377h13m54.125722577s) [upload=302099]: Upload finished. err: <nil>
Jul 9 02:25:57.289077655 (377h14m24.215372329s) [upload=302109]: Starting upload
Jul 9 02:25:57.289186750 (377h14m24.215481424s) [upload=302109]: Done starting upload
Jul 9 02:25:57.289444851 (377h14m24.215739548s) [upload=302109]: Uploading segments...
Jul 9 02:25:57.289623846 (377h14m24.215918535s) [upload=302109]: Got next segment. Inline: false
Jul 9 02:25:57.289696741 (377h14m24.215991432s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:25:57.289703375 (377h14m24.215998067s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:25:57.318313152 (377h14m24.244607838s) [seg_pos={0 0}, upload=302109]: Begin upload segment...
Jul 9 02:25:57.318363286 (377h14m24.244657964s) [seg_pos={0 0}, upload=302109]: Done begin upload segment.
Jul 9 02:25:57.318386772 (377h14m24.244681463s) [noise=true , address=77.90.31.11:4553 , piece_id=TYW7NBJWACIT7OQS7A4WGXUPR3MB2FKAN2OMGAZJERB4X2DLAO2Q, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318412127 (377h14m24.244706825s) [noise=true , address=77.90.31.10:4518 , piece_id=VJ4HWM2X4VSRSUCQL5QLMBYPFBYV4JYS5ZXCBCB6MTMB76EHZFGQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318822149 (377h14m24.245116841s) [noise=true , address=45.91.116.9:4263 , piece_id=5YILAJ5BZBGOURSZE343ST4RMOTWM6ZOB3EEQQWL5WUNYWOGSAPA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318965862 (377h14m24.24526056s) [noise=true , address=45.91.116.8:4045 , piece_id=KM6ZYTHR6JGTARKZQZYIESPKBEU5H2HQFA7OARIKD2ZXDHPFSMGA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.319187516 (377h14m24.245482203s) [noise=true , address=104.36.82.7:4909 , piece_id=ZRHLWDQ5KIDRWPJ5FXAANQTWERKXCR2PPEGN442IOSYSRFAG7VZA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.319523815 (377h14m24.245818512s) [noise=true , address=45.91.116.14:4191 , piece_id=GTR6T4OTGZTEZ76FWQZ46OCH5WB3O5X27EN4AV33ZJ7D5HX4535A, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.320556632 (377h14m24.24685133s) [noise=true , address=104.36.82.2:4732 , piece_id=DJIGHJMJRAULH2L4YT2WGRXPRQJMNKZG5LICHPSKOCHUL6QDV6BQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.321019141 (377h14m24.247313835s) [noise=true , address=77.90.31.17:4770 , piece_id=V36UBBIYOFJC6KFVVBXEVT4QBUIWN257D3YSDDRAGZVXH7ZXHTBA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.322591797 (377h14m24.248886493s) [noise=true , address=77.90.31.13:4627 , piece_id=HULEKFVZNDNKS4SWWGTJUHYZWWYPDCER5W4GFCVLLDL65PAHULFQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323120451 (377h14m24.249415138s) [noise=true , address=45.91.116.4:4939 , piece_id=ANA3EZU7OZWELNG4QEMTEAYZY5H3C3H4IJFT4XKPUB7WIF44FEVQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323544591 (377h14m24.249839288s) [noise=true , address=45.91.116.7:4011 , piece_id=VDOJRN6ACK4LO44BQTULO6BB3CLAVQI7YIC6XLR32CZV4UJSPE6Q, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323950189 (377h14m24.250244889s) [noise=true , address=104.36.82.10:4078 , piece_id=ZFVUQAEV6DDFLPKCZA2XJ2BZHMMM745LGB4JRZJZQPG72WECGMDQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.326585810 (377h14m24.252880507s) [noise=true , address=45.91.116.5:4297 , piece_id=MSDWDLY5PJHQVI5565EKANJVSTMZFEUD2OTYZ52C65XQUICNMC3A, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.327871214 (377h14m24.254165904s) [noise=true , address=104.36.82.6:4875 , piece_id=C275F2JPVAF52E34CEHT5QXCHUFGIQZ7XGMOAOUOT4RA7YOHRQLQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.328605288 (377h14m24.254899986s) [noise=true , address=104.36.82.5:4838 , piece_id=WK3ALBOEMWWTVEA4FKCS352SZ6YYB35LELURNWXQZK32K4HYDRNA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.335409087 (377h14m24.261703784s) [noise=true , address=45.91.116.6:4973 , piece_id=Z7HQQ2O5QUSJKT4EMIDMEI5PIAEHNVRYX7MALUR2GQULRFIBUOKA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.475344637 (377h14m24.401639326s) [upload=302109]: Next returned nil segment
Jul 9 02:25:57.475376181 (377h14m24.401670879s) [upload=302109]: Waiting for error group managing segments...
Jul 9 02:25:57.477269278 (377h14m24.403563958s) [noise=true , address=45.91.116.7:4011 , piece_id=VDOJRN6ACK4LO44BQTULO6BB3CLAVQI7YIC6XLR32CZV4UJSPE6Q, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.477736179 (377h14m24.40403086s) [noise=true , address=45.91.116.9:4263 , piece_id=5YILAJ5BZBGOURSZE343ST4RMOTWM6ZOB3EEQQWL5WUNYWOGSAPA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.478396652 (377h14m24.404691332s) [noise=true , address=77.90.31.10:4518 , piece_id=VJ4HWM2X4VSRSUCQL5QLMBYPFBYV4JYS5ZXCBCB6MTMB76EHZFGQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.478919538 (377h14m24.405214224s) [noise=true , address=45.91.116.14:4191 , piece_id=GTR6T4OTGZTEZ76FWQZ46OCH5WB3O5X27EN4AV33ZJ7D5HX4535A, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481426716 (377h14m24.407721396s) [noise=true , address=45.91.116.4:4939 , piece_id=ANA3EZU7OZWELNG4QEMTEAYZY5H3C3H4IJFT4XKPUB7WIF44FEVQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481718373 (377h14m24.408013051s) [noise=true , address=45.91.116.5:4297 , piece_id=MSDWDLY5PJHQVI5565EKANJVSTMZFEUD2OTYZ52C65XQUICNMC3A, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481930356 (377h14m24.408225033s) [noise=true , address=77.90.31.17:4770 , piece_id=V36UBBIYOFJC6KFVVBXEVT4QBUIWN257D3YSDDRAGZVXH7ZXHTBA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.482376147 (377h14m24.408670821s) [noise=true , address=45.91.116.8:4045 , piece_id=KM6ZYTHR6JGTARKZQZYIESPKBEU5H2HQFA7OARIKD2ZXDHPFSMGA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.482439192 (377h14m24.408733868s) [noise=true , address=45.91.116.6:4973 , piece_id=Z7HQQ2O5QUSJKT4EMIDMEI5PIAEHNVRYX7MALUR2GQULRFIBUOKA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.483047970 (377h14m24.409342644s) [noise=true , address=77.90.31.13:4627 , piece_id=HULEKFVZNDNKS4SWWGTJUHYZWWYPDCER5W4GFCVLLDL65PAHULFQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.497788632 (377h14m24.424083312s) [noise=true , address=77.90.31.11:4553 , piece_id=TYW7NBJWACIT7OQS7A4WGXUPR3MB2FKAN2OMGAZJERB4X2DLAO2Q, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:16.688921171 (377h14m43.615215851s) [noise=true , address=104.36.82.6:4875 , piece_id=C275F2JPVAF52E34CEHT5QXCHUFGIQZ7XGMOAOUOT4RA7YOHRQLQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:27.177893955 (377h14m54.104188635s) [noise=true , address=104.36.82.2:4732 , piece_id=DJIGHJMJRAULH2L4YT2WGRXPRQJMNKZG5LICHPSKOCHUL6QDV6BQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:27.177985584 (377h14m54.104280281s) [seg_pos={0 0}, upload=302109]: Segment reached optimal threshold of 13 pieces.
Jul 9 02:26:27.178189789 (377h14m54.104484461s) [noise=true , address=104.36.82.5:4838 , piece_id=WK3ALBOEMWWTVEA4FKCS352SZ6YYB35LELURNWXQZK32K4HYDRNA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178252042 (377h14m54.104546733s) [noise=true , address=104.36.82.7:4909 , piece_id=ZRHLWDQ5KIDRWPJ5FXAANQTWERKXCR2PPEGN442IOSYSRFAG7VZA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178273922 (377h14m54.104568608s) [noise=true , address=104.36.82.10:4078 , piece_id=ZFVUQAEV6DDFLPKCZA2XJ2BZHMMM745LGB4JRZJZQPG72WECGMDQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178305983 (377h14m54.104600682s) [seg_pos={0 0}, upload=302109]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 02:26:27.178428768 (377h14m54.104723465s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:26:27.178455810 (377h14m54.104750501s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:26:27.232166184 (377h14m54.158460883s) [upload=302109]: Done uploading segments...
Jul 9 02:26:27.232188444 (377h14m54.158483123s) [upload=302109]: Upload finished. err: <nil>
Duration: 456.669399ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......X.....................................................] duration:
416.199325ms
FAIL (2025-07-09 03:02:17 UTC)
orbiter warning:
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
02:25:27.135824454 (377h13m54.062119141s) [noise=true , address=104.36.82.12:4150 , piece_id=DRG5SBA3EE4KVQPEQOJZ6SS537SFWYFUYQXOAQMWYTT6FPVSJIPQ, seg_pos={0 0}, upload=302099]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: context canceled; context canceled
Jul 9 02:25:27.135860079 (377h13m54.062154769s) [seg_pos={0 0}, upload=302099]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 02:25:27.135957283 (377h13m54.062251971s) [upload=302099]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:25:27.135964211 (377h13m54.062258897s) [upload=302099]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:25:27.199410713 (377h13m54.125705389s) [upload=302099]: Done uploading segments...
Jul 9 02:25:27.199427901 (377h13m54.125722577s) [upload=302099]: Upload finished. err: <nil>
Jul 9 02:25:57.289077655 (377h14m24.215372329s) [upload=302109]: Starting upload
Jul 9 02:25:57.289186750 (377h14m24.215481424s) [upload=302109]: Done starting upload
Jul 9 02:25:57.289444851 (377h14m24.215739548s) [upload=302109]: Uploading segments...
Jul 9 02:25:57.289623846 (377h14m24.215918535s) [upload=302109]: Got next segment. Inline: false
Jul 9 02:25:57.289696741 (377h14m24.215991432s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:25:57.289703375 (377h14m24.215998067s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:25:57.318313152 (377h14m24.244607838s) [seg_pos={0 0}, upload=302109]: Begin upload segment...
Jul 9 02:25:57.318363286 (377h14m24.244657964s) [seg_pos={0 0}, upload=302109]: Done begin upload segment.
Jul 9 02:25:57.318386772 (377h14m24.244681463s) [noise=true , address=77.90.31.11:4553 , piece_id=TYW7NBJWACIT7OQS7A4WGXUPR3MB2FKAN2OMGAZJERB4X2DLAO2Q, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318412127 (377h14m24.244706825s) [noise=true , address=77.90.31.10:4518 , piece_id=VJ4HWM2X4VSRSUCQL5QLMBYPFBYV4JYS5ZXCBCB6MTMB76EHZFGQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318822149 (377h14m24.245116841s) [noise=true , address=45.91.116.9:4263 , piece_id=5YILAJ5BZBGOURSZE343ST4RMOTWM6ZOB3EEQQWL5WUNYWOGSAPA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318965862 (377h14m24.24526056s) [noise=true , address=45.91.116.8:4045 , piece_id=KM6ZYTHR6JGTARKZQZYIESPKBEU5H2HQFA7OARIKD2ZXDHPFSMGA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.319187516 (377h14m24.245482203s) [noise=true , address=104.36.82.7:4909 , piece_id=ZRHLWDQ5KIDRWPJ5FXAANQTWERKXCR2PPEGN442IOSYSRFAG7VZA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.319523815 (377h14m24.245818512s) [noise=true , address=45.91.116.14:4191 , piece_id=GTR6T4OTGZTEZ76FWQZ46OCH5WB3O5X27EN4AV33ZJ7D5HX4535A, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.320556632 (377h14m24.24685133s) [noise=true , address=104.36.82.2:4732 , piece_id=DJIGHJMJRAULH2L4YT2WGRXPRQJMNKZG5LICHPSKOCHUL6QDV6BQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.321019141 (377h14m24.247313835s) [noise=true , address=77.90.31.17:4770 , piece_id=V36UBBIYOFJC6KFVVBXEVT4QBUIWN257D3YSDDRAGZVXH7ZXHTBA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.322591797 (377h14m24.248886493s) [noise=true , address=77.90.31.13:4627 , piece_id=HULEKFVZNDNKS4SWWGTJUHYZWWYPDCER5W4GFCVLLDL65PAHULFQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323120451 (377h14m24.249415138s) [noise=true , address=45.91.116.4:4939 , piece_id=ANA3EZU7OZWELNG4QEMTEAYZY5H3C3H4IJFT4XKPUB7WIF44FEVQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323544591 (377h14m24.249839288s) [noise=true , address=45.91.116.7:4011 , piece_id=VDOJRN6ACK4LO44BQTULO6BB3CLAVQI7YIC6XLR32CZV4UJSPE6Q, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323950189 (377h14m24.250244889s) [noise=true , address=104.36.82.10:4078 , piece_id=ZFVUQAEV6DDFLPKCZA2XJ2BZHMMM745LGB4JRZJZQPG72WECGMDQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.326585810 (377h14m24.252880507s) [noise=true , address=45.91.116.5:4297 , piece_id=MSDWDLY5PJHQVI5565EKANJVSTMZFEUD2OTYZ52C65XQUICNMC3A, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.327871214 (377h14m24.254165904s) [noise=true , address=104.36.82.6:4875 , piece_id=C275F2JPVAF52E34CEHT5QXCHUFGIQZ7XGMOAOUOT4RA7YOHRQLQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.328605288 (377h14m24.254899986s) [noise=true , address=104.36.82.5:4838 , piece_id=WK3ALBOEMWWTVEA4FKCS352SZ6YYB35LELURNWXQZK32K4HYDRNA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.335409087 (377h14m24.261703784s) [noise=true , address=45.91.116.6:4973 , piece_id=Z7HQQ2O5QUSJKT4EMIDMEI5PIAEHNVRYX7MALUR2GQULRFIBUOKA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.475344637 (377h14m24.401639326s) [upload=302109]: Next returned nil segment
Jul 9 02:25:57.475376181 (377h14m24.401670879s) [upload=302109]: Waiting for error group managing segments...
Jul 9 02:25:57.477269278 (377h14m24.403563958s) [noise=true , address=45.91.116.7:4011 , piece_id=VDOJRN6ACK4LO44BQTULO6BB3CLAVQI7YIC6XLR32CZV4UJSPE6Q, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.477736179 (377h14m24.40403086s) [noise=true , address=45.91.116.9:4263 , piece_id=5YILAJ5BZBGOURSZE343ST4RMOTWM6ZOB3EEQQWL5WUNYWOGSAPA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.478396652 (377h14m24.404691332s) [noise=true , address=77.90.31.10:4518 , piece_id=VJ4HWM2X4VSRSUCQL5QLMBYPFBYV4JYS5ZXCBCB6MTMB76EHZFGQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.478919538 (377h14m24.405214224s) [noise=true , address=45.91.116.14:4191 , piece_id=GTR6T4OTGZTEZ76FWQZ46OCH5WB3O5X27EN4AV33ZJ7D5HX4535A, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481426716 (377h14m24.407721396s) [noise=true , address=45.91.116.4:4939 , piece_id=ANA3EZU7OZWELNG4QEMTEAYZY5H3C3H4IJFT4XKPUB7WIF44FEVQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481718373 (377h14m24.408013051s) [noise=true , address=45.91.116.5:4297 , piece_id=MSDWDLY5PJHQVI5565EKANJVSTMZFEUD2OTYZ52C65XQUICNMC3A, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481930356 (377h14m24.408225033s) [noise=true , address=77.90.31.17:4770 , piece_id=V36UBBIYOFJC6KFVVBXEVT4QBUIWN257D3YSDDRAGZVXH7ZXHTBA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.482376147 (377h14m24.408670821s) [noise=true , address=45.91.116.8:4045 , piece_id=KM6ZYTHR6JGTARKZQZYIESPKBEU5H2HQFA7OARIKD2ZXDHPFSMGA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.482439192 (377h14m24.408733868s) [noise=true , address=45.91.116.6:4973 , piece_id=Z7HQQ2O5QUSJKT4EMIDMEI5PIAEHNVRYX7MALUR2GQULRFIBUOKA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.483047970 (377h14m24.409342644s) [noise=true , address=77.90.31.13:4627 , piece_id=HULEKFVZNDNKS4SWWGTJUHYZWWYPDCER5W4GFCVLLDL65PAHULFQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.497788632 (377h14m24.424083312s) [noise=true , address=77.90.31.11:4553 , piece_id=TYW7NBJWACIT7OQS7A4WGXUPR3MB2FKAN2OMGAZJERB4X2DLAO2Q, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:16.688921171 (377h14m43.615215851s) [noise=true , address=104.36.82.6:4875 , piece_id=C275F2JPVAF52E34CEHT5QXCHUFGIQZ7XGMOAOUOT4RA7YOHRQLQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:27.177893955 (377h14m54.104188635s) [noise=true , address=104.36.82.2:4732 , piece_id=DJIGHJMJRAULH2L4YT2WGRXPRQJMNKZG5LICHPSKOCHUL6QDV6BQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:27.177985584 (377h14m54.104280281s) [seg_pos={0 0}, upload=302109]: Segment reached optimal threshold of 13 pieces.
Jul 9 02:26:27.178189789 (377h14m54.104484461s) [noise=true , address=104.36.82.5:4838 , piece_id=WK3ALBOEMWWTVEA4FKCS352SZ6YYB35LELURNWXQZK32K4HYDRNA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178252042 (377h14m54.104546733s) [noise=true , address=104.36.82.7:4909 , piece_id=ZRHLWDQ5KIDRWPJ5FXAANQTWERKXCR2PPEGN442IOSYSRFAG7VZA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178273922 (377h14m54.104568608s) [noise=true , address=104.36.82.10:4078 , piece_id=ZFVUQAEV6DDFLPKCZA2XJ2BZHMMM745LGB4JRZJZQPG72WECGMDQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178305983 (377h14m54.104600682s) [seg_pos={0 0}, upload=302109]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 02:26:27.178428768 (377h14m54.104723465s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:26:27.178455810 (377h14m54.104750501s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:26:27.232166184 (377h14m54.158460883s) [upload=302109]: Done uploading segments...
Jul 9 02:26:27.232188444 (377h14m54.158483123s) [upload=302109]: Upload finished. err: <nil>
Duration: 416.231801ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
34 out of 60 is failed (threshold=3) [.XXX.XX.XXXXXX.XXXXX.XX.....X..XXXXX.XX..XXX.XX.XX..X.......] duration:
679.618128ms
FAIL (2025-07-09 03:01:57 UTC)
operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 679.655627ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
21 out of 60 is failed (threshold=3) [..X.X.X.....XXX.XX.X...XXX....XX...X..X..X.XX.X..X..........] duration:
521.485159ms
FAIL (2025-07-09 03:01:49 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 521.512287ms
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......X....................................................] duration:
518.069027ms
FAIL (2025-07-09 03:01:35 UTC)
orbiter warning:
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
02:25:27.135824454 (377h13m54.062119141s) [noise=true , address=104.36.82.12:4150 , piece_id=DRG5SBA3EE4KVQPEQOJZ6SS537SFWYFUYQXOAQMWYTT6FPVSJIPQ, seg_pos={0 0}, upload=302099]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: context canceled; context canceled
Jul 9 02:25:27.135860079 (377h13m54.062154769s) [seg_pos={0 0}, upload=302099]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 02:25:27.135957283 (377h13m54.062251971s) [upload=302099]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:25:27.135964211 (377h13m54.062258897s) [upload=302099]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:25:27.199410713 (377h13m54.125705389s) [upload=302099]: Done uploading segments...
Jul 9 02:25:27.199427901 (377h13m54.125722577s) [upload=302099]: Upload finished. err: <nil>
Jul 9 02:25:57.289077655 (377h14m24.215372329s) [upload=302109]: Starting upload
Jul 9 02:25:57.289186750 (377h14m24.215481424s) [upload=302109]: Done starting upload
Jul 9 02:25:57.289444851 (377h14m24.215739548s) [upload=302109]: Uploading segments...
Jul 9 02:25:57.289623846 (377h14m24.215918535s) [upload=302109]: Got next segment. Inline: false
Jul 9 02:25:57.289696741 (377h14m24.215991432s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:25:57.289703375 (377h14m24.215998067s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:25:57.318313152 (377h14m24.244607838s) [seg_pos={0 0}, upload=302109]: Begin upload segment...
Jul 9 02:25:57.318363286 (377h14m24.244657964s) [seg_pos={0 0}, upload=302109]: Done begin upload segment.
Jul 9 02:25:57.318386772 (377h14m24.244681463s) [noise=true , address=77.90.31.11:4553 , piece_id=TYW7NBJWACIT7OQS7A4WGXUPR3MB2FKAN2OMGAZJERB4X2DLAO2Q, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318412127 (377h14m24.244706825s) [noise=true , address=77.90.31.10:4518 , piece_id=VJ4HWM2X4VSRSUCQL5QLMBYPFBYV4JYS5ZXCBCB6MTMB76EHZFGQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318822149 (377h14m24.245116841s) [noise=true , address=45.91.116.9:4263 , piece_id=5YILAJ5BZBGOURSZE343ST4RMOTWM6ZOB3EEQQWL5WUNYWOGSAPA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.318965862 (377h14m24.24526056s) [noise=true , address=45.91.116.8:4045 , piece_id=KM6ZYTHR6JGTARKZQZYIESPKBEU5H2HQFA7OARIKD2ZXDHPFSMGA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.319187516 (377h14m24.245482203s) [noise=true , address=104.36.82.7:4909 , piece_id=ZRHLWDQ5KIDRWPJ5FXAANQTWERKXCR2PPEGN442IOSYSRFAG7VZA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.319523815 (377h14m24.245818512s) [noise=true , address=45.91.116.14:4191 , piece_id=GTR6T4OTGZTEZ76FWQZ46OCH5WB3O5X27EN4AV33ZJ7D5HX4535A, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.320556632 (377h14m24.24685133s) [noise=true , address=104.36.82.2:4732 , piece_id=DJIGHJMJRAULH2L4YT2WGRXPRQJMNKZG5LICHPSKOCHUL6QDV6BQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.321019141 (377h14m24.247313835s) [noise=true , address=77.90.31.17:4770 , piece_id=V36UBBIYOFJC6KFVVBXEVT4QBUIWN257D3YSDDRAGZVXH7ZXHTBA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.322591797 (377h14m24.248886493s) [noise=true , address=77.90.31.13:4627 , piece_id=HULEKFVZNDNKS4SWWGTJUHYZWWYPDCER5W4GFCVLLDL65PAHULFQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323120451 (377h14m24.249415138s) [noise=true , address=45.91.116.4:4939 , piece_id=ANA3EZU7OZWELNG4QEMTEAYZY5H3C3H4IJFT4XKPUB7WIF44FEVQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323544591 (377h14m24.249839288s) [noise=true , address=45.91.116.7:4011 , piece_id=VDOJRN6ACK4LO44BQTULO6BB3CLAVQI7YIC6XLR32CZV4UJSPE6Q, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.323950189 (377h14m24.250244889s) [noise=true , address=104.36.82.10:4078 , piece_id=ZFVUQAEV6DDFLPKCZA2XJ2BZHMMM745LGB4JRZJZQPG72WECGMDQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.326585810 (377h14m24.252880507s) [noise=true , address=45.91.116.5:4297 , piece_id=MSDWDLY5PJHQVI5565EKANJVSTMZFEUD2OTYZ52C65XQUICNMC3A, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.327871214 (377h14m24.254165904s) [noise=true , address=104.36.82.6:4875 , piece_id=C275F2JPVAF52E34CEHT5QXCHUFGIQZ7XGMOAOUOT4RA7YOHRQLQ, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.328605288 (377h14m24.254899986s) [noise=true , address=104.36.82.5:4838 , piece_id=WK3ALBOEMWWTVEA4FKCS352SZ6YYB35LELURNWXQZK32K4HYDRNA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.335409087 (377h14m24.261703784s) [noise=true , address=45.91.116.6:4973 , piece_id=Z7HQQ2O5QUSJKT4EMIDMEI5PIAEHNVRYX7MALUR2GQULRFIBUOKA, seg_pos={0 0}, upload=302109]: Uploading piece...
Jul 9 02:25:57.475344637 (377h14m24.401639326s) [upload=302109]: Next returned nil segment
Jul 9 02:25:57.475376181 (377h14m24.401670879s) [upload=302109]: Waiting for error group managing segments...
Jul 9 02:25:57.477269278 (377h14m24.403563958s) [noise=true , address=45.91.116.7:4011 , piece_id=VDOJRN6ACK4LO44BQTULO6BB3CLAVQI7YIC6XLR32CZV4UJSPE6Q, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.477736179 (377h14m24.40403086s) [noise=true , address=45.91.116.9:4263 , piece_id=5YILAJ5BZBGOURSZE343ST4RMOTWM6ZOB3EEQQWL5WUNYWOGSAPA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.478396652 (377h14m24.404691332s) [noise=true , address=77.90.31.10:4518 , piece_id=VJ4HWM2X4VSRSUCQL5QLMBYPFBYV4JYS5ZXCBCB6MTMB76EHZFGQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.478919538 (377h14m24.405214224s) [noise=true , address=45.91.116.14:4191 , piece_id=GTR6T4OTGZTEZ76FWQZ46OCH5WB3O5X27EN4AV33ZJ7D5HX4535A, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481426716 (377h14m24.407721396s) [noise=true , address=45.91.116.4:4939 , piece_id=ANA3EZU7OZWELNG4QEMTEAYZY5H3C3H4IJFT4XKPUB7WIF44FEVQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481718373 (377h14m24.408013051s) [noise=true , address=45.91.116.5:4297 , piece_id=MSDWDLY5PJHQVI5565EKANJVSTMZFEUD2OTYZ52C65XQUICNMC3A, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.481930356 (377h14m24.408225033s) [noise=true , address=77.90.31.17:4770 , piece_id=V36UBBIYOFJC6KFVVBXEVT4QBUIWN257D3YSDDRAGZVXH7ZXHTBA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.482376147 (377h14m24.408670821s) [noise=true , address=45.91.116.8:4045 , piece_id=KM6ZYTHR6JGTARKZQZYIESPKBEU5H2HQFA7OARIKD2ZXDHPFSMGA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.482439192 (377h14m24.408733868s) [noise=true , address=45.91.116.6:4973 , piece_id=Z7HQQ2O5QUSJKT4EMIDMEI5PIAEHNVRYX7MALUR2GQULRFIBUOKA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.483047970 (377h14m24.409342644s) [noise=true , address=77.90.31.13:4627 , piece_id=HULEKFVZNDNKS4SWWGTJUHYZWWYPDCER5W4GFCVLLDL65PAHULFQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:25:57.497788632 (377h14m24.424083312s) [noise=true , address=77.90.31.11:4553 , piece_id=TYW7NBJWACIT7OQS7A4WGXUPR3MB2FKAN2OMGAZJERB4X2DLAO2Q, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:16.688921171 (377h14m43.615215851s) [noise=true , address=104.36.82.6:4875 , piece_id=C275F2JPVAF52E34CEHT5QXCHUFGIQZ7XGMOAOUOT4RA7YOHRQLQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:27.177893955 (377h14m54.104188635s) [noise=true , address=104.36.82.2:4732 , piece_id=DJIGHJMJRAULH2L4YT2WGRXPRQJMNKZG5LICHPSKOCHUL6QDV6BQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: <nil>
Jul 9 02:26:27.177985584 (377h14m54.104280281s) [seg_pos={0 0}, upload=302109]: Segment reached optimal threshold of 13 pieces.
Jul 9 02:26:27.178189789 (377h14m54.104484461s) [noise=true , address=104.36.82.5:4838 , piece_id=WK3ALBOEMWWTVEA4FKCS352SZ6YYB35LELURNWXQZK32K4HYDRNA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178252042 (377h14m54.104546733s) [noise=true , address=104.36.82.7:4909 , piece_id=ZRHLWDQ5KIDRWPJ5FXAANQTWERKXCR2PPEGN442IOSYSRFAG7VZA, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178273922 (377h14m54.104568608s) [noise=true , address=104.36.82.10:4078 , piece_id=ZFVUQAEV6DDFLPKCZA2XJ2BZHMMM745LGB4JRZJZQPG72WECGMDQ, seg_pos={0 0}, upload=302109]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 9 02:26:27.178305983 (377h14m54.104600682s) [seg_pos={0 0}, upload=302109]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 9 02:26:27.178428768 (377h14m54.104723465s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:26:27.178455810 (377h14m54.104750501s) [upload=302109]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:26:27.232166184 (377h14m54.158460883s) [upload=302109]: Done uploading segments...
Jul 9 02:26:27.232188444 (377h14m54.158483123s) [upload=302109]: Upload finished. err: <nil>
Duration: 518.100243ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
470.840025ms
FAIL (2025-07-09 03:01:35 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
3:56.263910643 (377h2m23.19020535s) [upload=301954]: Got next segment. Inline: false
Jul 9 02:13:56.263954358 (377h2m23.190249034s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:13:56.263959891 (377h2m23.190254572s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:13:56.302204249 (377h2m23.228498944s) [seg_pos={0 0}, upload=301954]: Begin upload segment...
Jul 9 02:13:56.302291587 (377h2m23.22858628s) [seg_pos={0 0}, upload=301954]: Done begin upload segment.
Jul 9 02:13:56.302338947 (377h2m23.228633682s) [noise=true , address=77.90.31.14:4689 , piece_id=LW7OT74AJCCC3TEKY6OZI553YK6YQFQ6C7PORQB53E7FUS24KQKQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.302352599 (377h2m23.228647272s) [noise=true , address=104.36.82.8:4986 , piece_id=KPY5JVLSAEB5GUKUJJSIF5M6IIHTJISEYYAXCGUSJ7OLI5QCIBNQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.302450837 (377h2m23.228745531s) [noise=true , address=77.90.31.16:4739 , piece_id=H6UCV6RURPXKC2H3YGAT2OXV2VZMW2DWIRG6TFD6CVXKOVMDP2VQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.302958774 (377h2m23.229253667s) [noise=true , address=77.90.31.13:4640 , piece_id=TPPN75DX3UCBM346CSVJN3GSOA47QV4FVPYSSAHWCJYEM42OCVLQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303069086 (377h2m23.229363782s) [noise=true , address=104.36.82.17:4345 , piece_id=YI27O5WLBYO3TPCX4MVUATFORY6U3EQMDKQHGKRHIZMTG7WGFB2A, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303495994 (377h2m23.22979069s) [noise=true , address=104.36.82.36:4644 , piece_id=K2G3MAPD7TWYZPKNHQBMEPJRIQDLE75FYVPAZFX4I3GJ4IUAA7XQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303575840 (377h2m23.229870536s) [noise=true , address=77.90.31.21:4898 , piece_id=75XXVNT4EV4D5UTX7O5XFL44IZOYY5LZSI2VPFQZPRERL6W5M23A, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303949062 (377h2m23.230245641s) [noise=true , address=79.127.201.215:4339 , piece_id=XDCTYE2SVI6RAPCXTRVANFCZYWJDZXLBHHJ47NQY6V7D36DEFPGA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304060410 (377h2m23.230355106s) [noise=true , address=104.36.82.41:4768 , piece_id=BGMF33OKUZSZKGIHJYI3RNXEBDE54TX67DYLJ7JCZMRNK6ZAU22Q, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304407993 (377h2m23.230702685s) [noise=true , address=77.90.31.10:4525 , piece_id=GPVSIZIBZWTUCP5RFYVPFYIYIL2LWU5PU3UASQVRQLY7SSSFMQBA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304548115 (377h2m23.230842812s) [noise=true , address=104.36.82.23:4464 , piece_id=JCJ4MGEZIIZ7YCTM6QXC5ZJ7PI4OEJBOMHZYBU57D2SNGWJWB5CQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304795164 (377h2m23.231089862s) [noise=true , address=77.90.31.18:4838 , piece_id=TGWYXA3AF7LDG4KPVCRNEYFDADAG6TQ5DAX7JJVW5BLVVVNECILA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305012510 (377h2m23.231307211s) [noise=true , address=104.36.82.34:4554 , piece_id=57K5SDV3YEGLVMSJGSLXNKEWOLETXE6CWBM2NPVCNKLFHPVQCJBQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305113737 (377h2m23.231408427s) [noise=true , address=104.36.82.4:4834 , piece_id=YLYIASUPNRTK7KOMRYLN7OQLQBPWC7WFGCPDGUVIOEQJWQQ7A7GA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305383755 (377h2m23.231678434s) [noise=true , address=104.36.82.38:4698 , piece_id=TSOXJNQEWODCWFEITH4IYKZI7A75UZEXRQPIQO7LMP6LT3WLNXUA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305483822 (377h2m23.231778517s) [noise=true , address=104.36.82.42:4830 , piece_id=ENYQTUCIF7OWQPCXTYS24OC5QT63ZJJXOY652GLIG7OQPJEQCGGA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305954019 (377h2m23.232248718s) [noise=true , address=77.90.31.21:4907 , piece_id=2EUWATHUQNZ77BJV65ZFKR5WL4D3AUURLBQ32XR2IVBA7P4TITOQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.306425517 (377h2m23.232720213s) [noise=true , address=104.36.82.22:4541 , piece_id=XN4HDFDNZ4YUJTDPOTMMDFX4FE2YVZDGCIMHEMJGWXRPNHQ6EU7A, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.375470790 (377h2m23.301765481s) [upload=301954]: Next returned nil segment
Jul 9 02:13:56.375489934 (377h2m23.30178463s) [upload=301954]: Waiting for error group managing segments...
Jul 9 02:13:56.381456859 (377h2m23.307751539s) [noise=true , address=77.90.31.10:4525 , piece_id=GPVSIZIBZWTUCP5RFYVPFYIYIL2LWU5PU3UASQVRQLY7SSSFMQBA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.381956907 (377h2m23.308251586s) [noise=true , address=77.90.31.14:4689 , piece_id=LW7OT74AJCCC3TEKY6OZI553YK6YQFQ6C7PORQB53E7FUS24KQKQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.382837088 (377h2m23.309131761s) [noise=true , address=77.90.31.21:4898 , piece_id=75XXVNT4EV4D5UTX7O5XFL44IZOYY5LZSI2VPFQZPRERL6W5M23A, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.393489125 (377h2m23.319783812s) [noise=true , address=77.90.31.21:4907 , piece_id=2EUWATHUQNZ77BJV65ZFKR5WL4D3AUURLBQ32XR2IVBA7P4TITOQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.404692801 (377h2m23.330987481s) [noise=true , address=77.90.31.16:4739 , piece_id=H6UCV6RURPXKC2H3YGAT2OXV2VZMW2DWIRG6TFD6CVXKOVMDP2VQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.404986115 (377h2m23.331280795s) [noise=true , address=77.90.31.18:4838 , piece_id=TGWYXA3AF7LDG4KPVCRNEYFDADAG6TQ5DAX7JJVW5BLVVVNECILA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.433785086 (377h2m23.360079766s) [noise=true , address=77.90.31.13:4640 , piece_id=TPPN75DX3UCBM346CSVJN3GSOA47QV4FVPYSSAHWCJYEM42OCVLQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.598499747 (377h2m23.524794427s) [noise=true , address=79.127.201.215:4339 , piece_id=XDCTYE2SVI6RAPCXTRVANFCZYWJDZXLBHHJ47NQY6V7D36DEFPGA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:11.107785584 (377h2m38.034080273s) [noise=true , address=104.36.82.22:4541 , piece_id=XN4HDFDNZ4YUJTDPOTMMDFX4FE2YVZDGCIMHEMJGWXRPNHQ6EU7A, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:12.515567029 (377h2m39.441861713s) [noise=true , address=104.36.82.17:4345 , piece_id=YI27O5WLBYO3TPCX4MVUATFORY6U3EQMDKQHGKRHIZMTG7WGFB2A, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:12.923090587 (377h2m39.849385267s) [noise=true , address=104.36.82.8:4986 , piece_id=KPY5JVLSAEB5GUKUJJSIF5M6IIHTJISEYYAXCGUSJ7OLI5QCIBNQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:13.453172659 (377h2m40.379467346s) [noise=true , address=104.36.82.38:4698 , piece_id=TSOXJNQEWODCWFEITH4IYKZI7A75UZEXRQPIQO7LMP6LT3WLNXUA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:14.416524856 (377h2m41.342819534s) [noise=true , address=104.36.82.34:4554 , piece_id=57K5SDV3YEGLVMSJGSLXNKEWOLETXE6CWBM2NPVCNKLFHPVQCJBQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:14.846730699 (377h2m41.773025379s) [noise=true , address=104.36.82.23:4464 , piece_id=JCJ4MGEZIIZ7YCTM6QXC5ZJ7PI4OEJBOMHZYBU57D2SNGWJWB5CQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:14.846763777 (377h2m41.773058463s) [seg_pos={0 0}, upload=301954]: Segment reached optimal threshold of 14 pieces.
Jul 9 02:14:14.846894905 (377h2m41.773189582s) [noise=true , address=104.36.82.36:4644 , piece_id=K2G3MAPD7TWYZPKNHQBMEPJRIQDLE75FYVPAZFX4I3GJ4IUAA7XQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1oYHbb73ACitbfkkP5P6cmRyTt6kmaS1y49G4vbsvTvVDwByhT): protocol: expected piece hash; context canceled; EOF
Jul 9 02:14:14.846948435 (377h2m41.773243131s) [noise=true , address=104.36.82.4:4834 , piece_id=YLYIASUPNRTK7KOMRYLN7OQLQBPWC7WFGCPDGUVIOEQJWQQ7A7GA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Z241DCYA5Wig2TdEogxTpY6FzVmFbnKtmHcTwxLxfcZafgooy): protocol: expected piece hash; context canceled; EOF
Jul 9 02:14:14.847094561 (377h2m41.773389249s) [noise=true , address=104.36.82.42:4830 , piece_id=ENYQTUCIF7OWQPCXTYS24OC5QT63ZJJXOY652GLIG7OQPJEQCGGA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:142ocSHHtv8BpSebXFNk5qk1Y23a8pbFAZqE3cq6i3RMX3dPw9): protocol: expected piece hash; context canceled; EOF
Jul 9 02:14:14.847127825 (377h2m41.773422511s) [noise=true , address=104.36.82.41:4768 , piece_id=BGMF33OKUZSZKGIHJYI3RNXEBDE54TX67DYLJ7JCZMRNK6ZAU22Q, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: write tcp 159.223.143.39:43796->104.36.82.41:4768: use of closed network connection; write tcp 159.223.143.39:43796->104.36.82.41:4768: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:43796->104.36.82.41:4768: use of closed network connection
Jul 9 02:14:14.847142999 (377h2m41.773437687s) [seg_pos={0 0}, upload=301954]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jul 9 02:14:14.847204962 (377h2m41.773499652s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:14:14.847209974 (377h2m41.773504671s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:14:14.909581364 (377h2m41.83587606s) [upload=301954]: Done uploading segments...
Jul 9 02:14:14.909605483 (377h2m41.835900172s) [upload=301954]: Upload finished. err: <nil>
Duration: 470.872689ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
22 out of 60 is failed (threshold=3) [X..X.X.X.....XXX.XX.X...XXX....XX...X..X..X.XX.X..X.........] duration:
631.510982ms
FAIL (2025-07-09 03:01:33 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 631.535317ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
35 out of 60 is failed (threshold=3) [X.XXX.XX.XXXXXX.XXXXX.XX.....X..XXXXX.XX..XXX.XX.XX..X......] duration:
724.031223ms
FAIL (2025-07-09 03:01:15 UTC)
operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 724.055005ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
23 out of 60 is failed (threshold=3) [XX..X.X.X.....XXX.XX.X...XXX....XX...X..X..X.XX.X..X........] duration:
531.207698ms
FAIL (2025-07-09 03:01:12 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 531.238386ms
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
638.65368ms
FAIL (2025-07-09 03:00:58 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
3:56.263910643 (377h2m23.19020535s) [upload=301954]: Got next segment. Inline: false
Jul 9 02:13:56.263954358 (377h2m23.190249034s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 9 02:13:56.263959891 (377h2m23.190254572s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 9 02:13:56.302204249 (377h2m23.228498944s) [seg_pos={0 0}, upload=301954]: Begin upload segment...
Jul 9 02:13:56.302291587 (377h2m23.22858628s) [seg_pos={0 0}, upload=301954]: Done begin upload segment.
Jul 9 02:13:56.302338947 (377h2m23.228633682s) [noise=true , address=77.90.31.14:4689 , piece_id=LW7OT74AJCCC3TEKY6OZI553YK6YQFQ6C7PORQB53E7FUS24KQKQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.302352599 (377h2m23.228647272s) [noise=true , address=104.36.82.8:4986 , piece_id=KPY5JVLSAEB5GUKUJJSIF5M6IIHTJISEYYAXCGUSJ7OLI5QCIBNQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.302450837 (377h2m23.228745531s) [noise=true , address=77.90.31.16:4739 , piece_id=H6UCV6RURPXKC2H3YGAT2OXV2VZMW2DWIRG6TFD6CVXKOVMDP2VQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.302958774 (377h2m23.229253667s) [noise=true , address=77.90.31.13:4640 , piece_id=TPPN75DX3UCBM346CSVJN3GSOA47QV4FVPYSSAHWCJYEM42OCVLQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303069086 (377h2m23.229363782s) [noise=true , address=104.36.82.17:4345 , piece_id=YI27O5WLBYO3TPCX4MVUATFORY6U3EQMDKQHGKRHIZMTG7WGFB2A, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303495994 (377h2m23.22979069s) [noise=true , address=104.36.82.36:4644 , piece_id=K2G3MAPD7TWYZPKNHQBMEPJRIQDLE75FYVPAZFX4I3GJ4IUAA7XQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303575840 (377h2m23.229870536s) [noise=true , address=77.90.31.21:4898 , piece_id=75XXVNT4EV4D5UTX7O5XFL44IZOYY5LZSI2VPFQZPRERL6W5M23A, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.303949062 (377h2m23.230245641s) [noise=true , address=79.127.201.215:4339 , piece_id=XDCTYE2SVI6RAPCXTRVANFCZYWJDZXLBHHJ47NQY6V7D36DEFPGA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304060410 (377h2m23.230355106s) [noise=true , address=104.36.82.41:4768 , piece_id=BGMF33OKUZSZKGIHJYI3RNXEBDE54TX67DYLJ7JCZMRNK6ZAU22Q, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304407993 (377h2m23.230702685s) [noise=true , address=77.90.31.10:4525 , piece_id=GPVSIZIBZWTUCP5RFYVPFYIYIL2LWU5PU3UASQVRQLY7SSSFMQBA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304548115 (377h2m23.230842812s) [noise=true , address=104.36.82.23:4464 , piece_id=JCJ4MGEZIIZ7YCTM6QXC5ZJ7PI4OEJBOMHZYBU57D2SNGWJWB5CQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.304795164 (377h2m23.231089862s) [noise=true , address=77.90.31.18:4838 , piece_id=TGWYXA3AF7LDG4KPVCRNEYFDADAG6TQ5DAX7JJVW5BLVVVNECILA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305012510 (377h2m23.231307211s) [noise=true , address=104.36.82.34:4554 , piece_id=57K5SDV3YEGLVMSJGSLXNKEWOLETXE6CWBM2NPVCNKLFHPVQCJBQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305113737 (377h2m23.231408427s) [noise=true , address=104.36.82.4:4834 , piece_id=YLYIASUPNRTK7KOMRYLN7OQLQBPWC7WFGCPDGUVIOEQJWQQ7A7GA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305383755 (377h2m23.231678434s) [noise=true , address=104.36.82.38:4698 , piece_id=TSOXJNQEWODCWFEITH4IYKZI7A75UZEXRQPIQO7LMP6LT3WLNXUA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305483822 (377h2m23.231778517s) [noise=true , address=104.36.82.42:4830 , piece_id=ENYQTUCIF7OWQPCXTYS24OC5QT63ZJJXOY652GLIG7OQPJEQCGGA, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.305954019 (377h2m23.232248718s) [noise=true , address=77.90.31.21:4907 , piece_id=2EUWATHUQNZ77BJV65ZFKR5WL4D3AUURLBQ32XR2IVBA7P4TITOQ, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.306425517 (377h2m23.232720213s) [noise=true , address=104.36.82.22:4541 , piece_id=XN4HDFDNZ4YUJTDPOTMMDFX4FE2YVZDGCIMHEMJGWXRPNHQ6EU7A, seg_pos={0 0}, upload=301954]: Uploading piece...
Jul 9 02:13:56.375470790 (377h2m23.301765481s) [upload=301954]: Next returned nil segment
Jul 9 02:13:56.375489934 (377h2m23.30178463s) [upload=301954]: Waiting for error group managing segments...
Jul 9 02:13:56.381456859 (377h2m23.307751539s) [noise=true , address=77.90.31.10:4525 , piece_id=GPVSIZIBZWTUCP5RFYVPFYIYIL2LWU5PU3UASQVRQLY7SSSFMQBA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.381956907 (377h2m23.308251586s) [noise=true , address=77.90.31.14:4689 , piece_id=LW7OT74AJCCC3TEKY6OZI553YK6YQFQ6C7PORQB53E7FUS24KQKQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.382837088 (377h2m23.309131761s) [noise=true , address=77.90.31.21:4898 , piece_id=75XXVNT4EV4D5UTX7O5XFL44IZOYY5LZSI2VPFQZPRERL6W5M23A, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.393489125 (377h2m23.319783812s) [noise=true , address=77.90.31.21:4907 , piece_id=2EUWATHUQNZ77BJV65ZFKR5WL4D3AUURLBQ32XR2IVBA7P4TITOQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.404692801 (377h2m23.330987481s) [noise=true , address=77.90.31.16:4739 , piece_id=H6UCV6RURPXKC2H3YGAT2OXV2VZMW2DWIRG6TFD6CVXKOVMDP2VQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.404986115 (377h2m23.331280795s) [noise=true , address=77.90.31.18:4838 , piece_id=TGWYXA3AF7LDG4KPVCRNEYFDADAG6TQ5DAX7JJVW5BLVVVNECILA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.433785086 (377h2m23.360079766s) [noise=true , address=77.90.31.13:4640 , piece_id=TPPN75DX3UCBM346CSVJN3GSOA47QV4FVPYSSAHWCJYEM42OCVLQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:13:56.598499747 (377h2m23.524794427s) [noise=true , address=79.127.201.215:4339 , piece_id=XDCTYE2SVI6RAPCXTRVANFCZYWJDZXLBHHJ47NQY6V7D36DEFPGA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:11.107785584 (377h2m38.034080273s) [noise=true , address=104.36.82.22:4541 , piece_id=XN4HDFDNZ4YUJTDPOTMMDFX4FE2YVZDGCIMHEMJGWXRPNHQ6EU7A, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:12.515567029 (377h2m39.441861713s) [noise=true , address=104.36.82.17:4345 , piece_id=YI27O5WLBYO3TPCX4MVUATFORY6U3EQMDKQHGKRHIZMTG7WGFB2A, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:12.923090587 (377h2m39.849385267s) [noise=true , address=104.36.82.8:4986 , piece_id=KPY5JVLSAEB5GUKUJJSIF5M6IIHTJISEYYAXCGUSJ7OLI5QCIBNQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:13.453172659 (377h2m40.379467346s) [noise=true , address=104.36.82.38:4698 , piece_id=TSOXJNQEWODCWFEITH4IYKZI7A75UZEXRQPIQO7LMP6LT3WLNXUA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:14.416524856 (377h2m41.342819534s) [noise=true , address=104.36.82.34:4554 , piece_id=57K5SDV3YEGLVMSJGSLXNKEWOLETXE6CWBM2NPVCNKLFHPVQCJBQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:14.846730699 (377h2m41.773025379s) [noise=true , address=104.36.82.23:4464 , piece_id=JCJ4MGEZIIZ7YCTM6QXC5ZJ7PI4OEJBOMHZYBU57D2SNGWJWB5CQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: <nil>
Jul 9 02:14:14.846763777 (377h2m41.773058463s) [seg_pos={0 0}, upload=301954]: Segment reached optimal threshold of 14 pieces.
Jul 9 02:14:14.846894905 (377h2m41.773189582s) [noise=true , address=104.36.82.36:4644 , piece_id=K2G3MAPD7TWYZPKNHQBMEPJRIQDLE75FYVPAZFX4I3GJ4IUAA7XQ, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1oYHbb73ACitbfkkP5P6cmRyTt6kmaS1y49G4vbsvTvVDwByhT): protocol: expected piece hash; context canceled; EOF
Jul 9 02:14:14.846948435 (377h2m41.773243131s) [noise=true , address=104.36.82.4:4834 , piece_id=YLYIASUPNRTK7KOMRYLN7OQLQBPWC7WFGCPDGUVIOEQJWQQ7A7GA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Z241DCYA5Wig2TdEogxTpY6FzVmFbnKtmHcTwxLxfcZafgooy): protocol: expected piece hash; context canceled; EOF
Jul 9 02:14:14.847094561 (377h2m41.773389249s) [noise=true , address=104.36.82.42:4830 , piece_id=ENYQTUCIF7OWQPCXTYS24OC5QT63ZJJXOY652GLIG7OQPJEQCGGA, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:142ocSHHtv8BpSebXFNk5qk1Y23a8pbFAZqE3cq6i3RMX3dPw9): protocol: expected piece hash; context canceled; EOF
Jul 9 02:14:14.847127825 (377h2m41.773422511s) [noise=true , address=104.36.82.41:4768 , piece_id=BGMF33OKUZSZKGIHJYI3RNXEBDE54TX67DYLJ7JCZMRNK6ZAU22Q, seg_pos={0 0}, upload=301954]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1g7r2boMdct1YvKxHZ5U3RM923N4UDfGNKGtYyAm6VKwfKAJn4): protocol: write tcp 159.223.143.39:43796->104.36.82.41:4768: use of closed network connection; write tcp 159.223.143.39:43796->104.36.82.41:4768: use of closed network connection; piecestore: piecestore close: write tcp 159.223.143.39:43796->104.36.82.41:4768: use of closed network connection
Jul 9 02:14:14.847142999 (377h2m41.773437687s) [seg_pos={0 0}, upload=301954]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jul 9 02:14:14.847204962 (377h2m41.773499652s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 9 02:14:14.847209974 (377h2m41.773504671s) [upload=301954]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 9 02:14:14.909581364 (377h2m41.83587606s) [upload=301954]: Done uploading segments...
Jul 9 02:14:14.909605483 (377h2m41.835900172s) [upload=301954]: Upload finished. err: <nil>
Duration: 638.683925ms
core
uplink
us-select-1
us1-placement-20
|