Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...] duration:
498.323025ms
FAIL (2025-07-22 21:50:56 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
ection (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR): internal: context canceled
Jul 22 21:46:55.346038549 (94h56m37.194645633s) [noise=true , address=104.36.82.15:4258 , piece_id=5PY24JYH7DLQXHDRMLQQ5IKT6FFIN6RUFCREBEPIFONE7VISVNVQ, seg_pos={0 0}, upload=75904]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF): protocol: expected piece hash; context canceled; EOF
Jul 22 21:46:55.346060498 (94h56m37.194667587s) [seg_pos={0 0}, upload=75904]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 22 21:46:55.346144831 (94h56m37.194751911s) [upload=75904]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 22 21:46:55.346149771 (94h56m37.194756842s) [upload=75904]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 22 21:46:55.440538714 (94h56m37.289145783s) [upload=75904]: Done uploading segments...
Jul 22 21:46:55.440561507 (94h56m37.289168574s) [upload=75904]: Upload finished. err: <nil>
Jul 22 21:47:31.519263453 (94h57m13.367870533s) [upload=75912]: Starting upload
Jul 22 21:47:31.519376285 (94h57m13.367983365s) [upload=75912]: Done starting upload
Jul 22 21:47:31.519493413 (94h57m13.368100496s) [upload=75912]: Uploading segments...
Jul 22 21:47:31.519792953 (94h57m13.368400034s) [upload=75912]: Got next segment. Inline: false
Jul 22 21:47:31.519870221 (94h57m13.368477302s) [upload=75912]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 22 21:47:31.519877356 (94h57m13.36848444s) [upload=75912]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 22 21:47:31.550118092 (94h57m13.398725176s) [seg_pos={0 0}, upload=75912]: Begin upload segment...
Jul 22 21:47:31.550179210 (94h57m13.398786279s) [seg_pos={0 0}, upload=75912]: Done begin upload segment.
Jul 22 21:47:31.550216010 (94h57m13.39882309s) [noise=true , address=45.91.116.10:4083 , piece_id=RMMMH2V6YE47GLHC4SH7WSVDTMIQWHURRZVCIICNF4Q2BHKKFXUA, seg_pos={0 0}, upload=75912]: Uploading piece...
Jul 22 21:47:31.550779320 (94h57m13.399386405s) [noise=true , address=104.36.82.13:4186 , piece_id=QIJMECZ3RSIYYFPHHLJUBUOLIBVEFP6MYGB2MCS2ZTW3J74VHGZQ, seg_pos={0 0}, upload=75912]: Uploading piece...
Jul 22 21:47:31.551219677 (94h57m13.399826769s) [noise=true , address=104.36.82.6:4876 , piece_id=RA37PFRDO3PZWDMCJAO3VU5N6YQCKO2OI2HS2QUJKGAPKD4VW7MQ, seg_pos={0 0}, upload=75912]: Uploading piece...
Jul 22 21:47:31.551232646 (94h57m13.399839732s) [noise=true , address=104.36.82.5:4839 , piece_id=JNNQQB2QRHTAPR23I6QQNKYYS44XPSK3CIGYENRPXILMXQPVXUMA, seg_pos={0 0}, upload=75912]: Uploading piece...
Jul 22 21:47:31.551671777 (94h57m13.400278861s) [noise=true , address=45.91.116.12:4226 , piece_id=5EX2VXEZV4UEGDAKUT3GOCMKHCFJTHSGGU766YPDBFKWTVHJ2J3A, seg_pos={0 0}, upload=75912]: Uploading piece...
Jul 22 21:47:31.551714979 (94h57m13.400322072s) [noise=true , address=77.90.31.19:4841 , piece_id=I6QZ7ZBGJPMR3QGLF3YCQYYXB7K23VFZ6GLVROK2PCQM5WKGC3PQ, seg_pos={0 0}, upload=75912]: Uploading piece...
Jul 22 21:47:31.552134271 (94h57m13.400741357s) [noise=true , address=77.90.31.14:4662 , piece_id=CF6KE6TKTAMWIQCWO7KUKYVVZFCW2P7NMWZXRLFIG7T5FUNX73CQ, seg_pos={0 0}, upload=75912]: Uploading piece...
Jul 22 21:47:31.552517707 (94h57m13.401124789s) [noise=true , address=77.90.31.21:4879 , piece_id=J3SC36EE4ZZADUQK27O3LLWWUA5TCXW4EHNELOUA3FQOQTRSYMIA, seg_pos={0 0}, upload=75912]: Uploading piece...
Jul 22 21:47:31.553106449 (94h57m13.40171353s) [noise=true , address=77.90.31.13:4628 , piece_id=6KOM7XH2YRYYH62V7ID5UDIMOTYK5T7ZB7IEZL5C3RTIPR3MQPPQ, seg_pos={0 0}, upload=75912]: Uploading piece...
Jul 22 21:47:31.553489978 (94h57m13.402097067s) [noise=true , address=77.90.31.14:4663 , piece_id=CQHSN3IW5ZLTAXUISH4KOOGPIYSKTR2B4ZGAW5JRZLZJ6DAZRZGA, seg_pos={0 0}, upload=75912]: Uploading piece...
Jul 22 21:47:31.553968118 (94h57m13.402575218s) [noise=true , address=104.36.82.16:4293 , piece_id=ED352K6WPIXI3VLWJ4KDRPECABYDDOWOARMB2AIM5IF7HRRS32AA, seg_pos={0 0}, upload=75912]: Uploading piece...
Jul 22 21:47:31.554350077 (94h57m13.402957164s) [noise=true , address=45.91.116.9:4261 , piece_id=YF7AV6FJRI4EL2R5B4TBIPINHISWY2HMJHOQS3UALNC4IDJ7ZH6A, seg_pos={0 0}, upload=75912]: Uploading piece...
Jul 22 21:47:31.554690623 (94h57m13.403297703s) [noise=true , address=104.36.82.14:4223 , piece_id=YGXRBQT6PXNCYAMXODTZ23FOSFVLWEJFGHTB4RVZ6MBR7WORFPPQ, seg_pos={0 0}, upload=75912]: Uploading piece...
Jul 22 21:47:31.554743323 (94h57m13.403350408s) [noise=true , address=77.90.31.10:4519 , piece_id=UFDX2ALGOW3E3OM4O25I7RJ5KD2OZXMQDK5N6GMZ5PP2O5KBJ2SQ, seg_pos={0 0}, upload=75912]: Uploading piece...
Jul 22 21:47:31.555062939 (94h57m13.403670021s) [noise=true , address=45.91.116.4:4940 , piece_id=3BQX3GVAEFQMCOSSQZTYNU63PNCXPEJFK5RAPGWT5DJKAG22VHZA, seg_pos={0 0}, upload=75912]: Uploading piece...
Jul 22 21:47:31.555980475 (94h57m13.404587565s) [noise=true , address=104.36.82.12:4150 , piece_id=BGBTMVBUQ2ZKBHHNHVQMNMVXGZBR2HUVXPEVUAGHBS732HF6IF4Q, seg_pos={0 0}, upload=75912]: Uploading piece...
Jul 22 21:47:31.670490620 (94h57m13.5190977s) [upload=75912]: Next returned nil segment
Jul 22 21:47:31.670506462 (94h57m13.519113542s) [upload=75912]: Waiting for error group managing segments...
Jul 22 21:47:31.679811603 (94h57m13.528418687s) [noise=true , address=45.91.116.9:4261 , piece_id=YF7AV6FJRI4EL2R5B4TBIPINHISWY2HMJHOQS3UALNC4IDJ7ZH6A, seg_pos={0 0}, upload=75912]: Done uploading piece. err: <nil>
Jul 22 21:47:31.680585357 (94h57m13.529192453s) [noise=true , address=104.36.82.6:4876 , piece_id=RA37PFRDO3PZWDMCJAO3VU5N6YQCKO2OI2HS2QUJKGAPKD4VW7MQ, seg_pos={0 0}, upload=75912]: Done uploading piece. err: <nil>
Jul 22 21:47:31.681123786 (94h57m13.529730877s) [noise=true , address=45.91.116.10:4083 , piece_id=RMMMH2V6YE47GLHC4SH7WSVDTMIQWHURRZVCIICNF4Q2BHKKFXUA, seg_pos={0 0}, upload=75912]: Done uploading piece. err: <nil>
Jul 22 21:47:31.681658707 (94h57m13.530265792s) [noise=true , address=77.90.31.14:4662 , piece_id=CF6KE6TKTAMWIQCWO7KUKYVVZFCW2P7NMWZXRLFIG7T5FUNX73CQ, seg_pos={0 0}, upload=75912]: Done uploading piece. err: <nil>
Jul 22 21:47:31.682124760 (94h57m13.530731842s) [noise=true , address=77.90.31.21:4879 , piece_id=J3SC36EE4ZZADUQK27O3LLWWUA5TCXW4EHNELOUA3FQOQTRSYMIA, seg_pos={0 0}, upload=75912]: Done uploading piece. err: <nil>
Jul 22 21:47:31.682215414 (94h57m13.530822487s) [noise=true , address=104.36.82.16:4293 , piece_id=ED352K6WPIXI3VLWJ4KDRPECABYDDOWOARMB2AIM5IF7HRRS32AA, seg_pos={0 0}, upload=75912]: Done uploading piece. err: <nil>
Jul 22 21:47:31.682734686 (94h57m13.531341768s) [noise=true , address=45.91.116.4:4940 , piece_id=3BQX3GVAEFQMCOSSQZTYNU63PNCXPEJFK5RAPGWT5DJKAG22VHZA, seg_pos={0 0}, upload=75912]: Done uploading piece. err: <nil>
Jul 22 21:47:31.683680993 (94h57m13.532288081s) [noise=true , address=77.90.31.10:4519 , piece_id=UFDX2ALGOW3E3OM4O25I7RJ5KD2OZXMQDK5N6GMZ5PP2O5KBJ2SQ, seg_pos={0 0}, upload=75912]: Done uploading piece. err: <nil>
Jul 22 21:47:31.684034593 (94h57m13.532641678s) [noise=true , address=104.36.82.13:4186 , piece_id=QIJMECZ3RSIYYFPHHLJUBUOLIBVEFP6MYGB2MCS2ZTW3J74VHGZQ, seg_pos={0 0}, upload=75912]: Done uploading piece. err: <nil>
Jul 22 21:47:31.684705248 (94h57m13.533312331s) [noise=true , address=104.36.82.5:4839 , piece_id=JNNQQB2QRHTAPR23I6QQNKYYS44XPSK3CIGYENRPXILMXQPVXUMA, seg_pos={0 0}, upload=75912]: Done uploading piece. err: <nil>
Jul 22 21:47:31.685807652 (94h57m13.534414744s) [noise=true , address=77.90.31.14:4663 , piece_id=CQHSN3IW5ZLTAXUISH4KOOGPIYSKTR2B4ZGAW5JRZLZJ6DAZRZGA, seg_pos={0 0}, upload=75912]: Done uploading piece. err: <nil>
Jul 22 21:47:31.686950724 (94h57m13.535557812s) [noise=true , address=77.90.31.19:4841 , piece_id=I6QZ7ZBGJPMR3QGLF3YCQYYXB7K23VFZ6GLVROK2PCQM5WKGC3PQ, seg_pos={0 0}, upload=75912]: Done uploading piece. err: <nil>
Jul 22 21:47:31.688656634 (94h57m13.537263722s) [noise=true , address=77.90.31.13:4628 , piece_id=6KOM7XH2YRYYH62V7ID5UDIMOTYK5T7ZB7IEZL5C3RTIPR3MQPPQ, seg_pos={0 0}, upload=75912]: Done uploading piece. err: <nil>
Jul 22 21:47:31.688684903 (94h57m13.537291992s) [seg_pos={0 0}, upload=75912]: Segment reached optimal threshold of 13 pieces.
Jul 22 21:47:31.688995099 (94h57m13.537602189s) [noise=true , address=104.36.82.14:4223 , piece_id=YGXRBQT6PXNCYAMXODTZ23FOSFVLWEJFGHTB4RVZ6MBR7WORFPPQ, seg_pos={0 0}, upload=75912]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: expected piece hash; context canceled; EOF
Jul 22 21:47:31.689108554 (94h57m13.537715637s) [noise=true , address=104.36.82.12:4150 , piece_id=BGBTMVBUQ2ZKBHHNHVQMNMVXGZBR2HUVXPEVUAGHBS732HF6IF4Q, seg_pos={0 0}, upload=75912]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: expected piece hash; context canceled; EOF
Jul 22 21:47:31.689206179 (94h57m13.537813263s) [noise=true , address=45.91.116.12:4226 , piece_id=5EX2VXEZV4UEGDAKUT3GOCMKHCFJTHSGGU766YPDBFKWTVHJ2J3A, seg_pos={0 0}, upload=75912]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF
Jul 22 21:47:31.689248152 (94h57m13.537855236s) [seg_pos={0 0}, upload=75912]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 22 21:47:31.689423619 (94h57m13.538030706s) [upload=75912]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 22 21:47:31.689430297 (94h57m13.538037378s) [upload=75912]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 22 21:47:31.742297333 (94h57m13.590904424s) [upload=75912]: Done uploading segments...
Jul 22 21:47:31.742319834 (94h57m13.590926916s) [upload=75912]: Upload finished. err: <nil>
Duration: 498.355323ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......] duration:
247.028332ms
FAIL (2025-07-22 21:50:38 UTC)
orbiter warning: uplink: bucket: metaclient: context deadline exceeded
Duration: 247.071932ms
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....] duration:
323.708716ms
FAIL (2025-07-22 21:49:46 UTC)
orbiter warning: uplink: bucket: metaclient: context deadline exceeded
Duration: 323.744609ms
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..] duration:
561.211423ms
FAIL (2025-07-22 21:49:45 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
ection (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR): internal: context canceled
Jul 22 21:46:55.346038549 (94h56m37.194645633s) [noise=true , address=104.36.82.15:4258 , piece_id=5PY24JYH7DLQXHDRMLQQ5IKT6FFIN6RUFCREBEPIFONE7VISVNVQ, seg_pos={0 0}, upload=75904]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF): protocol: expected piece hash; context canceled; EOF
Jul 22 21:46:55.346060498 (94h56m37.194667587s) [seg_pos={0 0}, upload=75904]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 22 21:46:55.346144831 (94h56m37.194751911s) [upload=75904]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 22 21:46:55.346149771 (94h56m37.194756842s) [upload=75904]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 22 21:46:55.440538714 (94h56m37.289145783s) [upload=75904]: Done uploading segments...
Jul 22 21:46:55.440561507 (94h56m37.289168574s) [upload=75904]: Upload finished. err: <nil>
Jul 22 21:47:31.519263453 (94h57m13.367870533s) [upload=75912]: Starting upload
Jul 22 21:47:31.519376285 (94h57m13.367983365s) [upload=75912]: Done starting upload
Jul 22 21:47:31.519493413 (94h57m13.368100496s) [upload=75912]: Uploading segments...
Jul 22 21:47:31.519792953 (94h57m13.368400034s) [upload=75912]: Got next segment. Inline: false
Jul 22 21:47:31.519870221 (94h57m13.368477302s) [upload=75912]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 22 21:47:31.519877356 (94h57m13.36848444s) [upload=75912]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 22 21:47:31.550118092 (94h57m13.398725176s) [seg_pos={0 0}, upload=75912]: Begin upload segment...
Jul 22 21:47:31.550179210 (94h57m13.398786279s) [seg_pos={0 0}, upload=75912]: Done begin upload segment.
Jul 22 21:47:31.550216010 (94h57m13.39882309s) [noise=true , address=45.91.116.10:4083 , piece_id=RMMMH2V6YE47GLHC4SH7WSVDTMIQWHURRZVCIICNF4Q2BHKKFXUA, seg_pos={0 0}, upload=75912]: Uploading piece...
Jul 22 21:47:31.550779320 (94h57m13.399386405s) [noise=true , address=104.36.82.13:4186 , piece_id=QIJMECZ3RSIYYFPHHLJUBUOLIBVEFP6MYGB2MCS2ZTW3J74VHGZQ, seg_pos={0 0}, upload=75912]: Uploading piece...
Jul 22 21:47:31.551219677 (94h57m13.399826769s) [noise=true , address=104.36.82.6:4876 , piece_id=RA37PFRDO3PZWDMCJAO3VU5N6YQCKO2OI2HS2QUJKGAPKD4VW7MQ, seg_pos={0 0}, upload=75912]: Uploading piece...
Jul 22 21:47:31.551232646 (94h57m13.399839732s) [noise=true , address=104.36.82.5:4839 , piece_id=JNNQQB2QRHTAPR23I6QQNKYYS44XPSK3CIGYENRPXILMXQPVXUMA, seg_pos={0 0}, upload=75912]: Uploading piece...
Jul 22 21:47:31.551671777 (94h57m13.400278861s) [noise=true , address=45.91.116.12:4226 , piece_id=5EX2VXEZV4UEGDAKUT3GOCMKHCFJTHSGGU766YPDBFKWTVHJ2J3A, seg_pos={0 0}, upload=75912]: Uploading piece...
Jul 22 21:47:31.551714979 (94h57m13.400322072s) [noise=true , address=77.90.31.19:4841 , piece_id=I6QZ7ZBGJPMR3QGLF3YCQYYXB7K23VFZ6GLVROK2PCQM5WKGC3PQ, seg_pos={0 0}, upload=75912]: Uploading piece...
Jul 22 21:47:31.552134271 (94h57m13.400741357s) [noise=true , address=77.90.31.14:4662 , piece_id=CF6KE6TKTAMWIQCWO7KUKYVVZFCW2P7NMWZXRLFIG7T5FUNX73CQ, seg_pos={0 0}, upload=75912]: Uploading piece...
Jul 22 21:47:31.552517707 (94h57m13.401124789s) [noise=true , address=77.90.31.21:4879 , piece_id=J3SC36EE4ZZADUQK27O3LLWWUA5TCXW4EHNELOUA3FQOQTRSYMIA, seg_pos={0 0}, upload=75912]: Uploading piece...
Jul 22 21:47:31.553106449 (94h57m13.40171353s) [noise=true , address=77.90.31.13:4628 , piece_id=6KOM7XH2YRYYH62V7ID5UDIMOTYK5T7ZB7IEZL5C3RTIPR3MQPPQ, seg_pos={0 0}, upload=75912]: Uploading piece...
Jul 22 21:47:31.553489978 (94h57m13.402097067s) [noise=true , address=77.90.31.14:4663 , piece_id=CQHSN3IW5ZLTAXUISH4KOOGPIYSKTR2B4ZGAW5JRZLZJ6DAZRZGA, seg_pos={0 0}, upload=75912]: Uploading piece...
Jul 22 21:47:31.553968118 (94h57m13.402575218s) [noise=true , address=104.36.82.16:4293 , piece_id=ED352K6WPIXI3VLWJ4KDRPECABYDDOWOARMB2AIM5IF7HRRS32AA, seg_pos={0 0}, upload=75912]: Uploading piece...
Jul 22 21:47:31.554350077 (94h57m13.402957164s) [noise=true , address=45.91.116.9:4261 , piece_id=YF7AV6FJRI4EL2R5B4TBIPINHISWY2HMJHOQS3UALNC4IDJ7ZH6A, seg_pos={0 0}, upload=75912]: Uploading piece...
Jul 22 21:47:31.554690623 (94h57m13.403297703s) [noise=true , address=104.36.82.14:4223 , piece_id=YGXRBQT6PXNCYAMXODTZ23FOSFVLWEJFGHTB4RVZ6MBR7WORFPPQ, seg_pos={0 0}, upload=75912]: Uploading piece...
Jul 22 21:47:31.554743323 (94h57m13.403350408s) [noise=true , address=77.90.31.10:4519 , piece_id=UFDX2ALGOW3E3OM4O25I7RJ5KD2OZXMQDK5N6GMZ5PP2O5KBJ2SQ, seg_pos={0 0}, upload=75912]: Uploading piece...
Jul 22 21:47:31.555062939 (94h57m13.403670021s) [noise=true , address=45.91.116.4:4940 , piece_id=3BQX3GVAEFQMCOSSQZTYNU63PNCXPEJFK5RAPGWT5DJKAG22VHZA, seg_pos={0 0}, upload=75912]: Uploading piece...
Jul 22 21:47:31.555980475 (94h57m13.404587565s) [noise=true , address=104.36.82.12:4150 , piece_id=BGBTMVBUQ2ZKBHHNHVQMNMVXGZBR2HUVXPEVUAGHBS732HF6IF4Q, seg_pos={0 0}, upload=75912]: Uploading piece...
Jul 22 21:47:31.670490620 (94h57m13.5190977s) [upload=75912]: Next returned nil segment
Jul 22 21:47:31.670506462 (94h57m13.519113542s) [upload=75912]: Waiting for error group managing segments...
Jul 22 21:47:31.679811603 (94h57m13.528418687s) [noise=true , address=45.91.116.9:4261 , piece_id=YF7AV6FJRI4EL2R5B4TBIPINHISWY2HMJHOQS3UALNC4IDJ7ZH6A, seg_pos={0 0}, upload=75912]: Done uploading piece. err: <nil>
Jul 22 21:47:31.680585357 (94h57m13.529192453s) [noise=true , address=104.36.82.6:4876 , piece_id=RA37PFRDO3PZWDMCJAO3VU5N6YQCKO2OI2HS2QUJKGAPKD4VW7MQ, seg_pos={0 0}, upload=75912]: Done uploading piece. err: <nil>
Jul 22 21:47:31.681123786 (94h57m13.529730877s) [noise=true , address=45.91.116.10:4083 , piece_id=RMMMH2V6YE47GLHC4SH7WSVDTMIQWHURRZVCIICNF4Q2BHKKFXUA, seg_pos={0 0}, upload=75912]: Done uploading piece. err: <nil>
Jul 22 21:47:31.681658707 (94h57m13.530265792s) [noise=true , address=77.90.31.14:4662 , piece_id=CF6KE6TKTAMWIQCWO7KUKYVVZFCW2P7NMWZXRLFIG7T5FUNX73CQ, seg_pos={0 0}, upload=75912]: Done uploading piece. err: <nil>
Jul 22 21:47:31.682124760 (94h57m13.530731842s) [noise=true , address=77.90.31.21:4879 , piece_id=J3SC36EE4ZZADUQK27O3LLWWUA5TCXW4EHNELOUA3FQOQTRSYMIA, seg_pos={0 0}, upload=75912]: Done uploading piece. err: <nil>
Jul 22 21:47:31.682215414 (94h57m13.530822487s) [noise=true , address=104.36.82.16:4293 , piece_id=ED352K6WPIXI3VLWJ4KDRPECABYDDOWOARMB2AIM5IF7HRRS32AA, seg_pos={0 0}, upload=75912]: Done uploading piece. err: <nil>
Jul 22 21:47:31.682734686 (94h57m13.531341768s) [noise=true , address=45.91.116.4:4940 , piece_id=3BQX3GVAEFQMCOSSQZTYNU63PNCXPEJFK5RAPGWT5DJKAG22VHZA, seg_pos={0 0}, upload=75912]: Done uploading piece. err: <nil>
Jul 22 21:47:31.683680993 (94h57m13.532288081s) [noise=true , address=77.90.31.10:4519 , piece_id=UFDX2ALGOW3E3OM4O25I7RJ5KD2OZXMQDK5N6GMZ5PP2O5KBJ2SQ, seg_pos={0 0}, upload=75912]: Done uploading piece. err: <nil>
Jul 22 21:47:31.684034593 (94h57m13.532641678s) [noise=true , address=104.36.82.13:4186 , piece_id=QIJMECZ3RSIYYFPHHLJUBUOLIBVEFP6MYGB2MCS2ZTW3J74VHGZQ, seg_pos={0 0}, upload=75912]: Done uploading piece. err: <nil>
Jul 22 21:47:31.684705248 (94h57m13.533312331s) [noise=true , address=104.36.82.5:4839 , piece_id=JNNQQB2QRHTAPR23I6QQNKYYS44XPSK3CIGYENRPXILMXQPVXUMA, seg_pos={0 0}, upload=75912]: Done uploading piece. err: <nil>
Jul 22 21:47:31.685807652 (94h57m13.534414744s) [noise=true , address=77.90.31.14:4663 , piece_id=CQHSN3IW5ZLTAXUISH4KOOGPIYSKTR2B4ZGAW5JRZLZJ6DAZRZGA, seg_pos={0 0}, upload=75912]: Done uploading piece. err: <nil>
Jul 22 21:47:31.686950724 (94h57m13.535557812s) [noise=true , address=77.90.31.19:4841 , piece_id=I6QZ7ZBGJPMR3QGLF3YCQYYXB7K23VFZ6GLVROK2PCQM5WKGC3PQ, seg_pos={0 0}, upload=75912]: Done uploading piece. err: <nil>
Jul 22 21:47:31.688656634 (94h57m13.537263722s) [noise=true , address=77.90.31.13:4628 , piece_id=6KOM7XH2YRYYH62V7ID5UDIMOTYK5T7ZB7IEZL5C3RTIPR3MQPPQ, seg_pos={0 0}, upload=75912]: Done uploading piece. err: <nil>
Jul 22 21:47:31.688684903 (94h57m13.537291992s) [seg_pos={0 0}, upload=75912]: Segment reached optimal threshold of 13 pieces.
Jul 22 21:47:31.688995099 (94h57m13.537602189s) [noise=true , address=104.36.82.14:4223 , piece_id=YGXRBQT6PXNCYAMXODTZ23FOSFVLWEJFGHTB4RVZ6MBR7WORFPPQ, seg_pos={0 0}, upload=75912]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: expected piece hash; context canceled; EOF
Jul 22 21:47:31.689108554 (94h57m13.537715637s) [noise=true , address=104.36.82.12:4150 , piece_id=BGBTMVBUQ2ZKBHHNHVQMNMVXGZBR2HUVXPEVUAGHBS732HF6IF4Q, seg_pos={0 0}, upload=75912]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: expected piece hash; context canceled; EOF
Jul 22 21:47:31.689206179 (94h57m13.537813263s) [noise=true , address=45.91.116.12:4226 , piece_id=5EX2VXEZV4UEGDAKUT3GOCMKHCFJTHSGGU766YPDBFKWTVHJ2J3A, seg_pos={0 0}, upload=75912]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF
Jul 22 21:47:31.689248152 (94h57m13.537855236s) [seg_pos={0 0}, upload=75912]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 22 21:47:31.689423619 (94h57m13.538030706s) [upload=75912]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 22 21:47:31.689430297 (94h57m13.538037378s) [upload=75912]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 22 21:47:31.742297333 (94h57m13.590904424s) [upload=75912]: Done uploading segments...
Jul 22 21:47:31.742319834 (94h57m13.590926916s) [upload=75912]: Upload finished. err: <nil>
Duration: 561.244245ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....] duration:
412.085602ms
FAIL (2025-07-22 21:49:17 UTC)
orbiter warning: uplink: bucket: metaclient: context deadline exceeded
Duration: 412.119835ms
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.] duration:
457.197011ms
FAIL (2025-07-22 21:48:43 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
ection (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR): internal: context canceled
Jul 22 21:46:55.346038549 (94h56m37.194645633s) [noise=true , address=104.36.82.15:4258 , piece_id=5PY24JYH7DLQXHDRMLQQ5IKT6FFIN6RUFCREBEPIFONE7VISVNVQ, seg_pos={0 0}, upload=75904]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF): protocol: expected piece hash; context canceled; EOF
Jul 22 21:46:55.346060498 (94h56m37.194667587s) [seg_pos={0 0}, upload=75904]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 22 21:46:55.346144831 (94h56m37.194751911s) [upload=75904]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 22 21:46:55.346149771 (94h56m37.194756842s) [upload=75904]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 22 21:46:55.440538714 (94h56m37.289145783s) [upload=75904]: Done uploading segments...
Jul 22 21:46:55.440561507 (94h56m37.289168574s) [upload=75904]: Upload finished. err: <nil>
Jul 22 21:47:31.519263453 (94h57m13.367870533s) [upload=75912]: Starting upload
Jul 22 21:47:31.519376285 (94h57m13.367983365s) [upload=75912]: Done starting upload
Jul 22 21:47:31.519493413 (94h57m13.368100496s) [upload=75912]: Uploading segments...
Jul 22 21:47:31.519792953 (94h57m13.368400034s) [upload=75912]: Got next segment. Inline: false
Jul 22 21:47:31.519870221 (94h57m13.368477302s) [upload=75912]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 22 21:47:31.519877356 (94h57m13.36848444s) [upload=75912]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 22 21:47:31.550118092 (94h57m13.398725176s) [seg_pos={0 0}, upload=75912]: Begin upload segment...
Jul 22 21:47:31.550179210 (94h57m13.398786279s) [seg_pos={0 0}, upload=75912]: Done begin upload segment.
Jul 22 21:47:31.550216010 (94h57m13.39882309s) [noise=true , address=45.91.116.10:4083 , piece_id=RMMMH2V6YE47GLHC4SH7WSVDTMIQWHURRZVCIICNF4Q2BHKKFXUA, seg_pos={0 0}, upload=75912]: Uploading piece...
Jul 22 21:47:31.550779320 (94h57m13.399386405s) [noise=true , address=104.36.82.13:4186 , piece_id=QIJMECZ3RSIYYFPHHLJUBUOLIBVEFP6MYGB2MCS2ZTW3J74VHGZQ, seg_pos={0 0}, upload=75912]: Uploading piece...
Jul 22 21:47:31.551219677 (94h57m13.399826769s) [noise=true , address=104.36.82.6:4876 , piece_id=RA37PFRDO3PZWDMCJAO3VU5N6YQCKO2OI2HS2QUJKGAPKD4VW7MQ, seg_pos={0 0}, upload=75912]: Uploading piece...
Jul 22 21:47:31.551232646 (94h57m13.399839732s) [noise=true , address=104.36.82.5:4839 , piece_id=JNNQQB2QRHTAPR23I6QQNKYYS44XPSK3CIGYENRPXILMXQPVXUMA, seg_pos={0 0}, upload=75912]: Uploading piece...
Jul 22 21:47:31.551671777 (94h57m13.400278861s) [noise=true , address=45.91.116.12:4226 , piece_id=5EX2VXEZV4UEGDAKUT3GOCMKHCFJTHSGGU766YPDBFKWTVHJ2J3A, seg_pos={0 0}, upload=75912]: Uploading piece...
Jul 22 21:47:31.551714979 (94h57m13.400322072s) [noise=true , address=77.90.31.19:4841 , piece_id=I6QZ7ZBGJPMR3QGLF3YCQYYXB7K23VFZ6GLVROK2PCQM5WKGC3PQ, seg_pos={0 0}, upload=75912]: Uploading piece...
Jul 22 21:47:31.552134271 (94h57m13.400741357s) [noise=true , address=77.90.31.14:4662 , piece_id=CF6KE6TKTAMWIQCWO7KUKYVVZFCW2P7NMWZXRLFIG7T5FUNX73CQ, seg_pos={0 0}, upload=75912]: Uploading piece...
Jul 22 21:47:31.552517707 (94h57m13.401124789s) [noise=true , address=77.90.31.21:4879 , piece_id=J3SC36EE4ZZADUQK27O3LLWWUA5TCXW4EHNELOUA3FQOQTRSYMIA, seg_pos={0 0}, upload=75912]: Uploading piece...
Jul 22 21:47:31.553106449 (94h57m13.40171353s) [noise=true , address=77.90.31.13:4628 , piece_id=6KOM7XH2YRYYH62V7ID5UDIMOTYK5T7ZB7IEZL5C3RTIPR3MQPPQ, seg_pos={0 0}, upload=75912]: Uploading piece...
Jul 22 21:47:31.553489978 (94h57m13.402097067s) [noise=true , address=77.90.31.14:4663 , piece_id=CQHSN3IW5ZLTAXUISH4KOOGPIYSKTR2B4ZGAW5JRZLZJ6DAZRZGA, seg_pos={0 0}, upload=75912]: Uploading piece...
Jul 22 21:47:31.553968118 (94h57m13.402575218s) [noise=true , address=104.36.82.16:4293 , piece_id=ED352K6WPIXI3VLWJ4KDRPECABYDDOWOARMB2AIM5IF7HRRS32AA, seg_pos={0 0}, upload=75912]: Uploading piece...
Jul 22 21:47:31.554350077 (94h57m13.402957164s) [noise=true , address=45.91.116.9:4261 , piece_id=YF7AV6FJRI4EL2R5B4TBIPINHISWY2HMJHOQS3UALNC4IDJ7ZH6A, seg_pos={0 0}, upload=75912]: Uploading piece...
Jul 22 21:47:31.554690623 (94h57m13.403297703s) [noise=true , address=104.36.82.14:4223 , piece_id=YGXRBQT6PXNCYAMXODTZ23FOSFVLWEJFGHTB4RVZ6MBR7WORFPPQ, seg_pos={0 0}, upload=75912]: Uploading piece...
Jul 22 21:47:31.554743323 (94h57m13.403350408s) [noise=true , address=77.90.31.10:4519 , piece_id=UFDX2ALGOW3E3OM4O25I7RJ5KD2OZXMQDK5N6GMZ5PP2O5KBJ2SQ, seg_pos={0 0}, upload=75912]: Uploading piece...
Jul 22 21:47:31.555062939 (94h57m13.403670021s) [noise=true , address=45.91.116.4:4940 , piece_id=3BQX3GVAEFQMCOSSQZTYNU63PNCXPEJFK5RAPGWT5DJKAG22VHZA, seg_pos={0 0}, upload=75912]: Uploading piece...
Jul 22 21:47:31.555980475 (94h57m13.404587565s) [noise=true , address=104.36.82.12:4150 , piece_id=BGBTMVBUQ2ZKBHHNHVQMNMVXGZBR2HUVXPEVUAGHBS732HF6IF4Q, seg_pos={0 0}, upload=75912]: Uploading piece...
Jul 22 21:47:31.670490620 (94h57m13.5190977s) [upload=75912]: Next returned nil segment
Jul 22 21:47:31.670506462 (94h57m13.519113542s) [upload=75912]: Waiting for error group managing segments...
Jul 22 21:47:31.679811603 (94h57m13.528418687s) [noise=true , address=45.91.116.9:4261 , piece_id=YF7AV6FJRI4EL2R5B4TBIPINHISWY2HMJHOQS3UALNC4IDJ7ZH6A, seg_pos={0 0}, upload=75912]: Done uploading piece. err: <nil>
Jul 22 21:47:31.680585357 (94h57m13.529192453s) [noise=true , address=104.36.82.6:4876 , piece_id=RA37PFRDO3PZWDMCJAO3VU5N6YQCKO2OI2HS2QUJKGAPKD4VW7MQ, seg_pos={0 0}, upload=75912]: Done uploading piece. err: <nil>
Jul 22 21:47:31.681123786 (94h57m13.529730877s) [noise=true , address=45.91.116.10:4083 , piece_id=RMMMH2V6YE47GLHC4SH7WSVDTMIQWHURRZVCIICNF4Q2BHKKFXUA, seg_pos={0 0}, upload=75912]: Done uploading piece. err: <nil>
Jul 22 21:47:31.681658707 (94h57m13.530265792s) [noise=true , address=77.90.31.14:4662 , piece_id=CF6KE6TKTAMWIQCWO7KUKYVVZFCW2P7NMWZXRLFIG7T5FUNX73CQ, seg_pos={0 0}, upload=75912]: Done uploading piece. err: <nil>
Jul 22 21:47:31.682124760 (94h57m13.530731842s) [noise=true , address=77.90.31.21:4879 , piece_id=J3SC36EE4ZZADUQK27O3LLWWUA5TCXW4EHNELOUA3FQOQTRSYMIA, seg_pos={0 0}, upload=75912]: Done uploading piece. err: <nil>
Jul 22 21:47:31.682215414 (94h57m13.530822487s) [noise=true , address=104.36.82.16:4293 , piece_id=ED352K6WPIXI3VLWJ4KDRPECABYDDOWOARMB2AIM5IF7HRRS32AA, seg_pos={0 0}, upload=75912]: Done uploading piece. err: <nil>
Jul 22 21:47:31.682734686 (94h57m13.531341768s) [noise=true , address=45.91.116.4:4940 , piece_id=3BQX3GVAEFQMCOSSQZTYNU63PNCXPEJFK5RAPGWT5DJKAG22VHZA, seg_pos={0 0}, upload=75912]: Done uploading piece. err: <nil>
Jul 22 21:47:31.683680993 (94h57m13.532288081s) [noise=true , address=77.90.31.10:4519 , piece_id=UFDX2ALGOW3E3OM4O25I7RJ5KD2OZXMQDK5N6GMZ5PP2O5KBJ2SQ, seg_pos={0 0}, upload=75912]: Done uploading piece. err: <nil>
Jul 22 21:47:31.684034593 (94h57m13.532641678s) [noise=true , address=104.36.82.13:4186 , piece_id=QIJMECZ3RSIYYFPHHLJUBUOLIBVEFP6MYGB2MCS2ZTW3J74VHGZQ, seg_pos={0 0}, upload=75912]: Done uploading piece. err: <nil>
Jul 22 21:47:31.684705248 (94h57m13.533312331s) [noise=true , address=104.36.82.5:4839 , piece_id=JNNQQB2QRHTAPR23I6QQNKYYS44XPSK3CIGYENRPXILMXQPVXUMA, seg_pos={0 0}, upload=75912]: Done uploading piece. err: <nil>
Jul 22 21:47:31.685807652 (94h57m13.534414744s) [noise=true , address=77.90.31.14:4663 , piece_id=CQHSN3IW5ZLTAXUISH4KOOGPIYSKTR2B4ZGAW5JRZLZJ6DAZRZGA, seg_pos={0 0}, upload=75912]: Done uploading piece. err: <nil>
Jul 22 21:47:31.686950724 (94h57m13.535557812s) [noise=true , address=77.90.31.19:4841 , piece_id=I6QZ7ZBGJPMR3QGLF3YCQYYXB7K23VFZ6GLVROK2PCQM5WKGC3PQ, seg_pos={0 0}, upload=75912]: Done uploading piece. err: <nil>
Jul 22 21:47:31.688656634 (94h57m13.537263722s) [noise=true , address=77.90.31.13:4628 , piece_id=6KOM7XH2YRYYH62V7ID5UDIMOTYK5T7ZB7IEZL5C3RTIPR3MQPPQ, seg_pos={0 0}, upload=75912]: Done uploading piece. err: <nil>
Jul 22 21:47:31.688684903 (94h57m13.537291992s) [seg_pos={0 0}, upload=75912]: Segment reached optimal threshold of 13 pieces.
Jul 22 21:47:31.688995099 (94h57m13.537602189s) [noise=true , address=104.36.82.14:4223 , piece_id=YGXRBQT6PXNCYAMXODTZ23FOSFVLWEJFGHTB4RVZ6MBR7WORFPPQ, seg_pos={0 0}, upload=75912]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: expected piece hash; context canceled; EOF
Jul 22 21:47:31.689108554 (94h57m13.537715637s) [noise=true , address=104.36.82.12:4150 , piece_id=BGBTMVBUQ2ZKBHHNHVQMNMVXGZBR2HUVXPEVUAGHBS732HF6IF4Q, seg_pos={0 0}, upload=75912]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: expected piece hash; context canceled; EOF
Jul 22 21:47:31.689206179 (94h57m13.537813263s) [noise=true , address=45.91.116.12:4226 , piece_id=5EX2VXEZV4UEGDAKUT3GOCMKHCFJTHSGGU766YPDBFKWTVHJ2J3A, seg_pos={0 0}, upload=75912]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF
Jul 22 21:47:31.689248152 (94h57m13.537855236s) [seg_pos={0 0}, upload=75912]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 22 21:47:31.689423619 (94h57m13.538030706s) [upload=75912]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 22 21:47:31.689430297 (94h57m13.538037378s) [upload=75912]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 22 21:47:31.742297333 (94h57m13.590904424s) [upload=75912]: Done uploading segments...
Jul 22 21:47:31.742319834 (94h57m13.590926916s) [upload=75912]: Upload finished. err: <nil>
Duration: 457.232165ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
30.000941949s
FAIL (2025-07-22 21:48:31 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
ection (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR): internal: context canceled
Jul 22 21:46:55.346038549 (94h56m37.194645633s) [noise=true , address=104.36.82.15:4258 , piece_id=5PY24JYH7DLQXHDRMLQQ5IKT6FFIN6RUFCREBEPIFONE7VISVNVQ, seg_pos={0 0}, upload=75904]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF): protocol: expected piece hash; context canceled; EOF
Jul 22 21:46:55.346060498 (94h56m37.194667587s) [seg_pos={0 0}, upload=75904]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 22 21:46:55.346144831 (94h56m37.194751911s) [upload=75904]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 22 21:46:55.346149771 (94h56m37.194756842s) [upload=75904]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 22 21:46:55.440538714 (94h56m37.289145783s) [upload=75904]: Done uploading segments...
Jul 22 21:46:55.440561507 (94h56m37.289168574s) [upload=75904]: Upload finished. err: <nil>
Jul 22 21:47:31.519263453 (94h57m13.367870533s) [upload=75912]: Starting upload
Jul 22 21:47:31.519376285 (94h57m13.367983365s) [upload=75912]: Done starting upload
Jul 22 21:47:31.519493413 (94h57m13.368100496s) [upload=75912]: Uploading segments...
Jul 22 21:47:31.519792953 (94h57m13.368400034s) [upload=75912]: Got next segment. Inline: false
Jul 22 21:47:31.519870221 (94h57m13.368477302s) [upload=75912]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 22 21:47:31.519877356 (94h57m13.36848444s) [upload=75912]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 22 21:47:31.550118092 (94h57m13.398725176s) [seg_pos={0 0}, upload=75912]: Begin upload segment...
Jul 22 21:47:31.550179210 (94h57m13.398786279s) [seg_pos={0 0}, upload=75912]: Done begin upload segment.
Jul 22 21:47:31.550216010 (94h57m13.39882309s) [noise=true , address=45.91.116.10:4083 , piece_id=RMMMH2V6YE47GLHC4SH7WSVDTMIQWHURRZVCIICNF4Q2BHKKFXUA, seg_pos={0 0}, upload=75912]: Uploading piece...
Jul 22 21:47:31.550779320 (94h57m13.399386405s) [noise=true , address=104.36.82.13:4186 , piece_id=QIJMECZ3RSIYYFPHHLJUBUOLIBVEFP6MYGB2MCS2ZTW3J74VHGZQ, seg_pos={0 0}, upload=75912]: Uploading piece...
Jul 22 21:47:31.551219677 (94h57m13.399826769s) [noise=true , address=104.36.82.6:4876 , piece_id=RA37PFRDO3PZWDMCJAO3VU5N6YQCKO2OI2HS2QUJKGAPKD4VW7MQ, seg_pos={0 0}, upload=75912]: Uploading piece...
Jul 22 21:47:31.551232646 (94h57m13.399839732s) [noise=true , address=104.36.82.5:4839 , piece_id=JNNQQB2QRHTAPR23I6QQNKYYS44XPSK3CIGYENRPXILMXQPVXUMA, seg_pos={0 0}, upload=75912]: Uploading piece...
Jul 22 21:47:31.551671777 (94h57m13.400278861s) [noise=true , address=45.91.116.12:4226 , piece_id=5EX2VXEZV4UEGDAKUT3GOCMKHCFJTHSGGU766YPDBFKWTVHJ2J3A, seg_pos={0 0}, upload=75912]: Uploading piece...
Jul 22 21:47:31.551714979 (94h57m13.400322072s) [noise=true , address=77.90.31.19:4841 , piece_id=I6QZ7ZBGJPMR3QGLF3YCQYYXB7K23VFZ6GLVROK2PCQM5WKGC3PQ, seg_pos={0 0}, upload=75912]: Uploading piece...
Jul 22 21:47:31.552134271 (94h57m13.400741357s) [noise=true , address=77.90.31.14:4662 , piece_id=CF6KE6TKTAMWIQCWO7KUKYVVZFCW2P7NMWZXRLFIG7T5FUNX73CQ, seg_pos={0 0}, upload=75912]: Uploading piece...
Jul 22 21:47:31.552517707 (94h57m13.401124789s) [noise=true , address=77.90.31.21:4879 , piece_id=J3SC36EE4ZZADUQK27O3LLWWUA5TCXW4EHNELOUA3FQOQTRSYMIA, seg_pos={0 0}, upload=75912]: Uploading piece...
Jul 22 21:47:31.553106449 (94h57m13.40171353s) [noise=true , address=77.90.31.13:4628 , piece_id=6KOM7XH2YRYYH62V7ID5UDIMOTYK5T7ZB7IEZL5C3RTIPR3MQPPQ, seg_pos={0 0}, upload=75912]: Uploading piece...
Jul 22 21:47:31.553489978 (94h57m13.402097067s) [noise=true , address=77.90.31.14:4663 , piece_id=CQHSN3IW5ZLTAXUISH4KOOGPIYSKTR2B4ZGAW5JRZLZJ6DAZRZGA, seg_pos={0 0}, upload=75912]: Uploading piece...
Jul 22 21:47:31.553968118 (94h57m13.402575218s) [noise=true , address=104.36.82.16:4293 , piece_id=ED352K6WPIXI3VLWJ4KDRPECABYDDOWOARMB2AIM5IF7HRRS32AA, seg_pos={0 0}, upload=75912]: Uploading piece...
Jul 22 21:47:31.554350077 (94h57m13.402957164s) [noise=true , address=45.91.116.9:4261 , piece_id=YF7AV6FJRI4EL2R5B4TBIPINHISWY2HMJHOQS3UALNC4IDJ7ZH6A, seg_pos={0 0}, upload=75912]: Uploading piece...
Jul 22 21:47:31.554690623 (94h57m13.403297703s) [noise=true , address=104.36.82.14:4223 , piece_id=YGXRBQT6PXNCYAMXODTZ23FOSFVLWEJFGHTB4RVZ6MBR7WORFPPQ, seg_pos={0 0}, upload=75912]: Uploading piece...
Jul 22 21:47:31.554743323 (94h57m13.403350408s) [noise=true , address=77.90.31.10:4519 , piece_id=UFDX2ALGOW3E3OM4O25I7RJ5KD2OZXMQDK5N6GMZ5PP2O5KBJ2SQ, seg_pos={0 0}, upload=75912]: Uploading piece...
Jul 22 21:47:31.555062939 (94h57m13.403670021s) [noise=true , address=45.91.116.4:4940 , piece_id=3BQX3GVAEFQMCOSSQZTYNU63PNCXPEJFK5RAPGWT5DJKAG22VHZA, seg_pos={0 0}, upload=75912]: Uploading piece...
Jul 22 21:47:31.555980475 (94h57m13.404587565s) [noise=true , address=104.36.82.12:4150 , piece_id=BGBTMVBUQ2ZKBHHNHVQMNMVXGZBR2HUVXPEVUAGHBS732HF6IF4Q, seg_pos={0 0}, upload=75912]: Uploading piece...
Jul 22 21:47:31.670490620 (94h57m13.5190977s) [upload=75912]: Next returned nil segment
Jul 22 21:47:31.670506462 (94h57m13.519113542s) [upload=75912]: Waiting for error group managing segments...
Jul 22 21:47:31.679811603 (94h57m13.528418687s) [noise=true , address=45.91.116.9:4261 , piece_id=YF7AV6FJRI4EL2R5B4TBIPINHISWY2HMJHOQS3UALNC4IDJ7ZH6A, seg_pos={0 0}, upload=75912]: Done uploading piece. err: <nil>
Jul 22 21:47:31.680585357 (94h57m13.529192453s) [noise=true , address=104.36.82.6:4876 , piece_id=RA37PFRDO3PZWDMCJAO3VU5N6YQCKO2OI2HS2QUJKGAPKD4VW7MQ, seg_pos={0 0}, upload=75912]: Done uploading piece. err: <nil>
Jul 22 21:47:31.681123786 (94h57m13.529730877s) [noise=true , address=45.91.116.10:4083 , piece_id=RMMMH2V6YE47GLHC4SH7WSVDTMIQWHURRZVCIICNF4Q2BHKKFXUA, seg_pos={0 0}, upload=75912]: Done uploading piece. err: <nil>
Jul 22 21:47:31.681658707 (94h57m13.530265792s) [noise=true , address=77.90.31.14:4662 , piece_id=CF6KE6TKTAMWIQCWO7KUKYVVZFCW2P7NMWZXRLFIG7T5FUNX73CQ, seg_pos={0 0}, upload=75912]: Done uploading piece. err: <nil>
Jul 22 21:47:31.682124760 (94h57m13.530731842s) [noise=true , address=77.90.31.21:4879 , piece_id=J3SC36EE4ZZADUQK27O3LLWWUA5TCXW4EHNELOUA3FQOQTRSYMIA, seg_pos={0 0}, upload=75912]: Done uploading piece. err: <nil>
Jul 22 21:47:31.682215414 (94h57m13.530822487s) [noise=true , address=104.36.82.16:4293 , piece_id=ED352K6WPIXI3VLWJ4KDRPECABYDDOWOARMB2AIM5IF7HRRS32AA, seg_pos={0 0}, upload=75912]: Done uploading piece. err: <nil>
Jul 22 21:47:31.682734686 (94h57m13.531341768s) [noise=true , address=45.91.116.4:4940 , piece_id=3BQX3GVAEFQMCOSSQZTYNU63PNCXPEJFK5RAPGWT5DJKAG22VHZA, seg_pos={0 0}, upload=75912]: Done uploading piece. err: <nil>
Jul 22 21:47:31.683680993 (94h57m13.532288081s) [noise=true , address=77.90.31.10:4519 , piece_id=UFDX2ALGOW3E3OM4O25I7RJ5KD2OZXMQDK5N6GMZ5PP2O5KBJ2SQ, seg_pos={0 0}, upload=75912]: Done uploading piece. err: <nil>
Jul 22 21:47:31.684034593 (94h57m13.532641678s) [noise=true , address=104.36.82.13:4186 , piece_id=QIJMECZ3RSIYYFPHHLJUBUOLIBVEFP6MYGB2MCS2ZTW3J74VHGZQ, seg_pos={0 0}, upload=75912]: Done uploading piece. err: <nil>
Jul 22 21:47:31.684705248 (94h57m13.533312331s) [noise=true , address=104.36.82.5:4839 , piece_id=JNNQQB2QRHTAPR23I6QQNKYYS44XPSK3CIGYENRPXILMXQPVXUMA, seg_pos={0 0}, upload=75912]: Done uploading piece. err: <nil>
Jul 22 21:47:31.685807652 (94h57m13.534414744s) [noise=true , address=77.90.31.14:4663 , piece_id=CQHSN3IW5ZLTAXUISH4KOOGPIYSKTR2B4ZGAW5JRZLZJ6DAZRZGA, seg_pos={0 0}, upload=75912]: Done uploading piece. err: <nil>
Jul 22 21:47:31.686950724 (94h57m13.535557812s) [noise=true , address=77.90.31.19:4841 , piece_id=I6QZ7ZBGJPMR3QGLF3YCQYYXB7K23VFZ6GLVROK2PCQM5WKGC3PQ, seg_pos={0 0}, upload=75912]: Done uploading piece. err: <nil>
Jul 22 21:47:31.688656634 (94h57m13.537263722s) [noise=true , address=77.90.31.13:4628 , piece_id=6KOM7XH2YRYYH62V7ID5UDIMOTYK5T7ZB7IEZL5C3RTIPR3MQPPQ, seg_pos={0 0}, upload=75912]: Done uploading piece. err: <nil>
Jul 22 21:47:31.688684903 (94h57m13.537291992s) [seg_pos={0 0}, upload=75912]: Segment reached optimal threshold of 13 pieces.
Jul 22 21:47:31.688995099 (94h57m13.537602189s) [noise=true , address=104.36.82.14:4223 , piece_id=YGXRBQT6PXNCYAMXODTZ23FOSFVLWEJFGHTB4RVZ6MBR7WORFPPQ, seg_pos={0 0}, upload=75912]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: expected piece hash; context canceled; EOF
Jul 22 21:47:31.689108554 (94h57m13.537715637s) [noise=true , address=104.36.82.12:4150 , piece_id=BGBTMVBUQ2ZKBHHNHVQMNMVXGZBR2HUVXPEVUAGHBS732HF6IF4Q, seg_pos={0 0}, upload=75912]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: expected piece hash; context canceled; EOF
Jul 22 21:47:31.689206179 (94h57m13.537813263s) [noise=true , address=45.91.116.12:4226 , piece_id=5EX2VXEZV4UEGDAKUT3GOCMKHCFJTHSGGU766YPDBFKWTVHJ2J3A, seg_pos={0 0}, upload=75912]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF
Jul 22 21:47:31.689248152 (94h57m13.537855236s) [seg_pos={0 0}, upload=75912]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 22 21:47:31.689423619 (94h57m13.538030706s) [upload=75912]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 22 21:47:31.689430297 (94h57m13.538037378s) [upload=75912]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 22 21:47:31.742297333 (94h57m13.590904424s) [upload=75912]: Done uploading segments...
Jul 22 21:47:31.742319834 (94h57m13.590926916s) [upload=75912]: Upload finished. err: <nil>
Duration: 30.000967047s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...] duration:
307.258229ms
FAIL (2025-07-22 21:48:24 UTC)
orbiter warning: uplink: bucket: metaclient: context deadline exceeded
Duration: 307.295501ms
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..] duration:
233.640998ms
FAIL (2025-07-22 21:47:29 UTC)
orbiter warning: uplink: bucket: metaclient: context deadline exceeded
Duration: 233.696144ms
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.] duration:
339.725412ms
FAIL (2025-07-22 21:46:25 UTC)
orbiter warning: uplink: bucket: metaclient: context deadline exceeded
Duration: 339.755243ms
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
30.001022645s
FAIL (2025-07-22 21:45:49 UTC)
orbiter warning: uplink: bucket: metaclient: context deadline exceeded
Duration: 30.001046498s
core
uplink
us1
|
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)antiblip:
1 out of 15 is failed (threshold=6) [X..............] duration:
211.167434ms remote:
"tcp:34.202.203.47:443" target:
https://www.storj.io/dcs-satellites
FAIL (2025-07-22 17:05:52 UTC)
orbiter warning: Get "https://www.storj.io/dcs-satellites": EOF
Duration: 211.193483ms
core
website
|
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)antiblip:
1 out of 15 is failed (threshold=6) [.X.............] duration:
107.23048ms remote:
"tcp:34.202.203.47:443" target:
https://www.storj.io/dcs-satellites
FAIL (2025-07-22 17:05:34 UTC)
orbiter warning: Get "https://www.storj.io/dcs-satellites": EOF
Duration: 107.261583ms
core
website
|
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)antiblip:
1 out of 15 is failed (threshold=6) [..X............] duration:
141.908127ms remote:
"tcp:34.202.203.47:443" target:
https://www.storj.io/dcs-satellites
FAIL (2025-07-22 17:04:42 UTC)
orbiter warning: Get "https://www.storj.io/dcs-satellites": EOF
Duration: 141.935324ms
core
website
|
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)antiblip:
1 out of 15 is failed (threshold=6) [...X...........] duration:
109.897435ms remote:
"tcp:34.202.203.47:443" target:
https://www.storj.io/dcs-satellites
FAIL (2025-07-22 17:03:47 UTC)
orbiter warning: Get "https://www.storj.io/dcs-satellites": EOF
Duration: 109.928778ms
core
website
|
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)antiblip:
1 out of 15 is failed (threshold=6) [....X..........] duration:
150.286889ms remote:
"tcp:34.202.203.47:443" target:
https://www.storj.io/dcs-satellites
FAIL (2025-07-22 17:03:18 UTC)
orbiter warning: Get "https://www.storj.io/dcs-satellites": EOF
Duration: 150.328938ms
core
website
|
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)antiblip:
1 out of 15 is failed (threshold=6) [.....X.........] duration:
107.527697ms remote:
"tcp:34.202.203.47:443" target:
https://www.storj.io/dcs-satellites
FAIL (2025-07-22 17:02:36 UTC)
orbiter warning: Get "https://www.storj.io/dcs-satellites": EOF
Duration: 107.557767ms
core
website
|
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)antiblip:
1 out of 15 is failed (threshold=6) [......X........] duration:
120.246139ms remote:
"tcp:34.202.203.47:443" target:
https://www.storj.io/dcs-satellites
FAIL (2025-07-22 17:01:38 UTC)
orbiter warning: Get "https://www.storj.io/dcs-satellites": EOF
Duration: 120.383531ms
core
website
|
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)antiblip:
1 out of 15 is failed (threshold=6) [.......X.......] duration:
99.55228ms remote:
"tcp:34.202.203.47:443" target:
https://www.storj.io/dcs-satellites
FAIL (2025-07-22 17:01:07 UTC)
orbiter warning: Get "https://www.storj.io/dcs-satellites": EOF
Duration: 99.58232ms
core
website
|
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)antiblip:
1 out of 15 is failed (threshold=6) [........X......] duration:
87.164131ms remote:
"tcp:34.202.203.47:443" target:
https://www.storj.io/dcs-satellites
FAIL (2025-07-22 17:00:46 UTC)
orbiter warning: Get "https://www.storj.io/dcs-satellites": EOF
Duration: 87.191568ms
core
website
|
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)antiblip:
1 out of 15 is failed (threshold=6) [.........X.....] duration:
249.999585ms remote:
"tcp:34.202.203.47:443" target:
https://www.storj.io/dcs-satellites
FAIL (2025-07-22 17:00:15 UTC)
orbiter warning: Get "https://www.storj.io/dcs-satellites": EOF
Duration: 250.026741ms
core
website
|
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)antiblip:
1 out of 15 is failed (threshold=6) [..........X....] duration:
111.609989ms remote:
"tcp:34.202.203.47:443" target:
https://www.storj.io/dcs-satellites
FAIL (2025-07-22 16:59:01 UTC)
orbiter warning: Get "https://www.storj.io/dcs-satellites": EOF
Duration: 111.671538ms
core
website
|
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)antiblip:
1 out of 15 is failed (threshold=6) [...........X...] duration:
81.750729ms remote:
"tcp:34.202.203.47:443" target:
https://www.storj.io/dcs-satellites
FAIL (2025-07-22 16:58:36 UTC)
orbiter warning: Get "https://www.storj.io/dcs-satellites": EOF
Duration: 81.777785ms
core
website
|
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)antiblip:
1 out of 15 is failed (threshold=6) [............X..] duration:
129.045391ms remote:
"tcp:34.202.203.47:443" target:
https://www.storj.io/dcs-satellites
FAIL (2025-07-22 16:57:58 UTC)
orbiter warning: Get "https://www.storj.io/dcs-satellites": EOF
Duration: 129.078847ms
core
website
|
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)antiblip:
1 out of 15 is failed (threshold=6) [.............X.] duration:
4.312048715s remote:
"tcp:34.202.203.47:443" target:
https://www.storj.io/dcs-satellites
FAIL (2025-07-22 16:57:42 UTC)
orbiter warning: Get "https://www.storj.io/dcs-satellites": EOF
Duration: 4.312065805s
core
website
|
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)antiblip:
1 out of 15 is failed (threshold=6) [..............X] duration:
1m4.799936611s remote:
"tcp:34.202.203.47:443" target:
https://www.storj.io/dcs-satellites
FAIL (2025-07-22 16:57:37 UTC)
orbiter warning: Get "https://www.storj.io/dcs-satellites": EOF
Duration: 1m4.799962749s
core
website
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
585.097811ms
FAIL (2025-07-21 23:58:17 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: write tcp 159.223.143.39:45270->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 21 23:17:24.165004211 (72h27m6.013611298s) [seg_pos={0 0}, upload=57949]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 21 23:17:24.165120348 (72h27m6.013727437s) [upload=57949]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 21 23:17:24.165126833 (72h27m6.013733917s) [upload=57949]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 21 23:17:24.212200709 (72h27m6.060807788s) [upload=57949]: Done uploading segments...
Jul 21 23:17:24.212223698 (72h27m6.060830779s) [upload=57949]: Upload finished. err: <nil>
Jul 21 23:18:17.656177722 (72h27m59.504784801s) [upload=57962]: Starting upload
Jul 21 23:18:17.656257051 (72h27m59.504864134s) [upload=57962]: Done starting upload
Jul 21 23:18:17.656517180 (72h27m59.505124252s) [upload=57962]: Uploading segments...
Jul 21 23:18:17.656648005 (72h27m59.505255091s) [upload=57962]: Got next segment. Inline: false
Jul 21 23:18:17.656730310 (72h27m59.505337381s) [upload=57962]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 21 23:18:17.656755643 (72h27m59.505362732s) [upload=57962]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 21 23:18:17.688469593 (72h27m59.537076677s) [seg_pos={0 0}, upload=57962]: Begin upload segment...
Jul 21 23:18:17.688525948 (72h27m59.537133018s) [seg_pos={0 0}, upload=57962]: Done begin upload segment.
Jul 21 23:18:17.688554683 (72h27m59.53716178s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.688567851 (72h27m59.537174942s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689009443 (72h27m59.537616529s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689160244 (72h27m59.537767326s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689433427 (72h27m59.538040502s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689720722 (72h27m59.538327803s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689914229 (72h27m59.538521313s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690190143 (72h27m59.538797225s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690301325 (72h27m59.538908411s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690627802 (72h27m59.5392349s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690668175 (72h27m59.53927526s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690952592 (72h27m59.539559677s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.691126121 (72h27m59.539733203s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.691329514 (72h27m59.539936601s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.691589223 (72h27m59.540196305s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.691703336 (72h27m59.54031042s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.774337466 (72h27m59.622944538s) [upload=57962]: Next returned nil segment
Jul 21 23:18:17.774357849 (72h27m59.622964934s) [upload=57962]: Waiting for error group managing segments...
Jul 21 23:18:17.782297334 (72h27m59.630904403s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:17.782803757 (72h27m59.631410828s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:17.807443161 (72h27m59.656050239s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:18.754331863 (72h28m0.602938929s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:19.875439965 (72h28m1.724047036s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:22.232185161 (72h28m4.080792241s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:25.127197806 (72h28m6.975804875s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:25.352395099 (72h28m7.201002177s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:29.802237912 (72h28m11.650845006s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:32.623147661 (72h28m14.471754735s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:38.504797891 (72h28m20.353404969s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:40.260235005 (72h28m22.108842072s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:43.979948314 (72h28m25.828555395s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:43.979982630 (72h28m25.828589711s) [seg_pos={0 0}, upload=57962]: Segment reached optimal threshold of 13 pieces.
Jul 21 23:18:43.980515951 (72h28m25.829123035s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: rpc: dial tcp 104.36.82.5:4840: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 21 23:18:43.980555334 (72h28m25.829162419s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV): protocol: write tcp 159.223.143.39:35166->104.36.82.5:4837: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 21 23:18:43.980603676 (72h28m25.829210767s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: write tcp 159.223.143.39:59214->104.36.82.12:4149: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 21 23:18:43.980622868 (72h28m25.82922994s) [seg_pos={0 0}, upload=57962]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 21 23:18:43.980694561 (72h28m25.829301646s) [upload=57962]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 21 23:18:43.980699451 (72h28m25.829306525s) [upload=57962]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 21 23:18:44.029370168 (72h28m25.877977237s) [upload=57962]: Done uploading segments...
Jul 21 23:18:44.029388033 (72h28m25.877995113s) [upload=57962]: Upload finished. err: <nil>
Duration: 585.143412ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
410.645653ms
FAIL (2025-07-21 23:57:43 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: write tcp 159.223.143.39:45270->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 21 23:17:24.165004211 (72h27m6.013611298s) [seg_pos={0 0}, upload=57949]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 21 23:17:24.165120348 (72h27m6.013727437s) [upload=57949]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 21 23:17:24.165126833 (72h27m6.013733917s) [upload=57949]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 21 23:17:24.212200709 (72h27m6.060807788s) [upload=57949]: Done uploading segments...
Jul 21 23:17:24.212223698 (72h27m6.060830779s) [upload=57949]: Upload finished. err: <nil>
Jul 21 23:18:17.656177722 (72h27m59.504784801s) [upload=57962]: Starting upload
Jul 21 23:18:17.656257051 (72h27m59.504864134s) [upload=57962]: Done starting upload
Jul 21 23:18:17.656517180 (72h27m59.505124252s) [upload=57962]: Uploading segments...
Jul 21 23:18:17.656648005 (72h27m59.505255091s) [upload=57962]: Got next segment. Inline: false
Jul 21 23:18:17.656730310 (72h27m59.505337381s) [upload=57962]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 21 23:18:17.656755643 (72h27m59.505362732s) [upload=57962]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 21 23:18:17.688469593 (72h27m59.537076677s) [seg_pos={0 0}, upload=57962]: Begin upload segment...
Jul 21 23:18:17.688525948 (72h27m59.537133018s) [seg_pos={0 0}, upload=57962]: Done begin upload segment.
Jul 21 23:18:17.688554683 (72h27m59.53716178s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.688567851 (72h27m59.537174942s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689009443 (72h27m59.537616529s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689160244 (72h27m59.537767326s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689433427 (72h27m59.538040502s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689720722 (72h27m59.538327803s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689914229 (72h27m59.538521313s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690190143 (72h27m59.538797225s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690301325 (72h27m59.538908411s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690627802 (72h27m59.5392349s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690668175 (72h27m59.53927526s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690952592 (72h27m59.539559677s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.691126121 (72h27m59.539733203s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.691329514 (72h27m59.539936601s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.691589223 (72h27m59.540196305s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.691703336 (72h27m59.54031042s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.774337466 (72h27m59.622944538s) [upload=57962]: Next returned nil segment
Jul 21 23:18:17.774357849 (72h27m59.622964934s) [upload=57962]: Waiting for error group managing segments...
Jul 21 23:18:17.782297334 (72h27m59.630904403s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:17.782803757 (72h27m59.631410828s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:17.807443161 (72h27m59.656050239s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:18.754331863 (72h28m0.602938929s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:19.875439965 (72h28m1.724047036s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:22.232185161 (72h28m4.080792241s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:25.127197806 (72h28m6.975804875s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:25.352395099 (72h28m7.201002177s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:29.802237912 (72h28m11.650845006s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:32.623147661 (72h28m14.471754735s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:38.504797891 (72h28m20.353404969s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:40.260235005 (72h28m22.108842072s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:43.979948314 (72h28m25.828555395s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:43.979982630 (72h28m25.828589711s) [seg_pos={0 0}, upload=57962]: Segment reached optimal threshold of 13 pieces.
Jul 21 23:18:43.980515951 (72h28m25.829123035s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: rpc: dial tcp 104.36.82.5:4840: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 21 23:18:43.980555334 (72h28m25.829162419s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV): protocol: write tcp 159.223.143.39:35166->104.36.82.5:4837: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 21 23:18:43.980603676 (72h28m25.829210767s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: write tcp 159.223.143.39:59214->104.36.82.12:4149: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 21 23:18:43.980622868 (72h28m25.82922994s) [seg_pos={0 0}, upload=57962]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 21 23:18:43.980694561 (72h28m25.829301646s) [upload=57962]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 21 23:18:43.980699451 (72h28m25.829306525s) [upload=57962]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 21 23:18:44.029370168 (72h28m25.877977237s) [upload=57962]: Done uploading segments...
Jul 21 23:18:44.029388033 (72h28m25.877995113s) [upload=57962]: Upload finished. err: <nil>
Duration: 410.694459ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
434.745333ms
FAIL (2025-07-21 23:56:49 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: write tcp 159.223.143.39:45270->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 21 23:17:24.165004211 (72h27m6.013611298s) [seg_pos={0 0}, upload=57949]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 21 23:17:24.165120348 (72h27m6.013727437s) [upload=57949]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 21 23:17:24.165126833 (72h27m6.013733917s) [upload=57949]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 21 23:17:24.212200709 (72h27m6.060807788s) [upload=57949]: Done uploading segments...
Jul 21 23:17:24.212223698 (72h27m6.060830779s) [upload=57949]: Upload finished. err: <nil>
Jul 21 23:18:17.656177722 (72h27m59.504784801s) [upload=57962]: Starting upload
Jul 21 23:18:17.656257051 (72h27m59.504864134s) [upload=57962]: Done starting upload
Jul 21 23:18:17.656517180 (72h27m59.505124252s) [upload=57962]: Uploading segments...
Jul 21 23:18:17.656648005 (72h27m59.505255091s) [upload=57962]: Got next segment. Inline: false
Jul 21 23:18:17.656730310 (72h27m59.505337381s) [upload=57962]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 21 23:18:17.656755643 (72h27m59.505362732s) [upload=57962]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 21 23:18:17.688469593 (72h27m59.537076677s) [seg_pos={0 0}, upload=57962]: Begin upload segment...
Jul 21 23:18:17.688525948 (72h27m59.537133018s) [seg_pos={0 0}, upload=57962]: Done begin upload segment.
Jul 21 23:18:17.688554683 (72h27m59.53716178s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.688567851 (72h27m59.537174942s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689009443 (72h27m59.537616529s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689160244 (72h27m59.537767326s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689433427 (72h27m59.538040502s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689720722 (72h27m59.538327803s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689914229 (72h27m59.538521313s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690190143 (72h27m59.538797225s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690301325 (72h27m59.538908411s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690627802 (72h27m59.5392349s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690668175 (72h27m59.53927526s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690952592 (72h27m59.539559677s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.691126121 (72h27m59.539733203s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.691329514 (72h27m59.539936601s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.691589223 (72h27m59.540196305s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.691703336 (72h27m59.54031042s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.774337466 (72h27m59.622944538s) [upload=57962]: Next returned nil segment
Jul 21 23:18:17.774357849 (72h27m59.622964934s) [upload=57962]: Waiting for error group managing segments...
Jul 21 23:18:17.782297334 (72h27m59.630904403s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:17.782803757 (72h27m59.631410828s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:17.807443161 (72h27m59.656050239s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:18.754331863 (72h28m0.602938929s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:19.875439965 (72h28m1.724047036s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:22.232185161 (72h28m4.080792241s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:25.127197806 (72h28m6.975804875s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:25.352395099 (72h28m7.201002177s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:29.802237912 (72h28m11.650845006s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:32.623147661 (72h28m14.471754735s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:38.504797891 (72h28m20.353404969s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:40.260235005 (72h28m22.108842072s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:43.979948314 (72h28m25.828555395s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:43.979982630 (72h28m25.828589711s) [seg_pos={0 0}, upload=57962]: Segment reached optimal threshold of 13 pieces.
Jul 21 23:18:43.980515951 (72h28m25.829123035s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: rpc: dial tcp 104.36.82.5:4840: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 21 23:18:43.980555334 (72h28m25.829162419s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV): protocol: write tcp 159.223.143.39:35166->104.36.82.5:4837: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 21 23:18:43.980603676 (72h28m25.829210767s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: write tcp 159.223.143.39:59214->104.36.82.12:4149: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 21 23:18:43.980622868 (72h28m25.82922994s) [seg_pos={0 0}, upload=57962]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 21 23:18:43.980694561 (72h28m25.829301646s) [upload=57962]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 21 23:18:43.980699451 (72h28m25.829306525s) [upload=57962]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 21 23:18:44.029370168 (72h28m25.877977237s) [upload=57962]: Done uploading segments...
Jul 21 23:18:44.029388033 (72h28m25.877995113s) [upload=57962]: Upload finished. err: <nil>
Duration: 434.773677ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................] duration:
679.532583ms
FAIL (2025-07-21 23:55:48 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: write tcp 159.223.143.39:45270->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 21 23:17:24.165004211 (72h27m6.013611298s) [seg_pos={0 0}, upload=57949]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 21 23:17:24.165120348 (72h27m6.013727437s) [upload=57949]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 21 23:17:24.165126833 (72h27m6.013733917s) [upload=57949]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 21 23:17:24.212200709 (72h27m6.060807788s) [upload=57949]: Done uploading segments...
Jul 21 23:17:24.212223698 (72h27m6.060830779s) [upload=57949]: Upload finished. err: <nil>
Jul 21 23:18:17.656177722 (72h27m59.504784801s) [upload=57962]: Starting upload
Jul 21 23:18:17.656257051 (72h27m59.504864134s) [upload=57962]: Done starting upload
Jul 21 23:18:17.656517180 (72h27m59.505124252s) [upload=57962]: Uploading segments...
Jul 21 23:18:17.656648005 (72h27m59.505255091s) [upload=57962]: Got next segment. Inline: false
Jul 21 23:18:17.656730310 (72h27m59.505337381s) [upload=57962]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 21 23:18:17.656755643 (72h27m59.505362732s) [upload=57962]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 21 23:18:17.688469593 (72h27m59.537076677s) [seg_pos={0 0}, upload=57962]: Begin upload segment...
Jul 21 23:18:17.688525948 (72h27m59.537133018s) [seg_pos={0 0}, upload=57962]: Done begin upload segment.
Jul 21 23:18:17.688554683 (72h27m59.53716178s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.688567851 (72h27m59.537174942s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689009443 (72h27m59.537616529s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689160244 (72h27m59.537767326s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689433427 (72h27m59.538040502s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689720722 (72h27m59.538327803s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689914229 (72h27m59.538521313s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690190143 (72h27m59.538797225s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690301325 (72h27m59.538908411s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690627802 (72h27m59.5392349s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690668175 (72h27m59.53927526s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690952592 (72h27m59.539559677s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.691126121 (72h27m59.539733203s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.691329514 (72h27m59.539936601s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.691589223 (72h27m59.540196305s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.691703336 (72h27m59.54031042s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.774337466 (72h27m59.622944538s) [upload=57962]: Next returned nil segment
Jul 21 23:18:17.774357849 (72h27m59.622964934s) [upload=57962]: Waiting for error group managing segments...
Jul 21 23:18:17.782297334 (72h27m59.630904403s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:17.782803757 (72h27m59.631410828s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:17.807443161 (72h27m59.656050239s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:18.754331863 (72h28m0.602938929s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:19.875439965 (72h28m1.724047036s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:22.232185161 (72h28m4.080792241s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:25.127197806 (72h28m6.975804875s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:25.352395099 (72h28m7.201002177s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:29.802237912 (72h28m11.650845006s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:32.623147661 (72h28m14.471754735s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:38.504797891 (72h28m20.353404969s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:40.260235005 (72h28m22.108842072s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:43.979948314 (72h28m25.828555395s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:43.979982630 (72h28m25.828589711s) [seg_pos={0 0}, upload=57962]: Segment reached optimal threshold of 13 pieces.
Jul 21 23:18:43.980515951 (72h28m25.829123035s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: rpc: dial tcp 104.36.82.5:4840: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 21 23:18:43.980555334 (72h28m25.829162419s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV): protocol: write tcp 159.223.143.39:35166->104.36.82.5:4837: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 21 23:18:43.980603676 (72h28m25.829210767s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: write tcp 159.223.143.39:59214->104.36.82.12:4149: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 21 23:18:43.980622868 (72h28m25.82922994s) [seg_pos={0 0}, upload=57962]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 21 23:18:43.980694561 (72h28m25.829301646s) [upload=57962]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 21 23:18:43.980699451 (72h28m25.829306525s) [upload=57962]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 21 23:18:44.029370168 (72h28m25.877977237s) [upload=57962]: Done uploading segments...
Jul 21 23:18:44.029388033 (72h28m25.877995113s) [upload=57962]: Upload finished. err: <nil>
Duration: 679.560363ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
638.354255ms
FAIL (2025-07-21 23:54:46 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: write tcp 159.223.143.39:45270->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 21 23:17:24.165004211 (72h27m6.013611298s) [seg_pos={0 0}, upload=57949]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 21 23:17:24.165120348 (72h27m6.013727437s) [upload=57949]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 21 23:17:24.165126833 (72h27m6.013733917s) [upload=57949]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 21 23:17:24.212200709 (72h27m6.060807788s) [upload=57949]: Done uploading segments...
Jul 21 23:17:24.212223698 (72h27m6.060830779s) [upload=57949]: Upload finished. err: <nil>
Jul 21 23:18:17.656177722 (72h27m59.504784801s) [upload=57962]: Starting upload
Jul 21 23:18:17.656257051 (72h27m59.504864134s) [upload=57962]: Done starting upload
Jul 21 23:18:17.656517180 (72h27m59.505124252s) [upload=57962]: Uploading segments...
Jul 21 23:18:17.656648005 (72h27m59.505255091s) [upload=57962]: Got next segment. Inline: false
Jul 21 23:18:17.656730310 (72h27m59.505337381s) [upload=57962]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 21 23:18:17.656755643 (72h27m59.505362732s) [upload=57962]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 21 23:18:17.688469593 (72h27m59.537076677s) [seg_pos={0 0}, upload=57962]: Begin upload segment...
Jul 21 23:18:17.688525948 (72h27m59.537133018s) [seg_pos={0 0}, upload=57962]: Done begin upload segment.
Jul 21 23:18:17.688554683 (72h27m59.53716178s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.688567851 (72h27m59.537174942s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689009443 (72h27m59.537616529s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689160244 (72h27m59.537767326s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689433427 (72h27m59.538040502s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689720722 (72h27m59.538327803s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689914229 (72h27m59.538521313s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690190143 (72h27m59.538797225s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690301325 (72h27m59.538908411s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690627802 (72h27m59.5392349s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690668175 (72h27m59.53927526s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690952592 (72h27m59.539559677s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.691126121 (72h27m59.539733203s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.691329514 (72h27m59.539936601s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.691589223 (72h27m59.540196305s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.691703336 (72h27m59.54031042s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.774337466 (72h27m59.622944538s) [upload=57962]: Next returned nil segment
Jul 21 23:18:17.774357849 (72h27m59.622964934s) [upload=57962]: Waiting for error group managing segments...
Jul 21 23:18:17.782297334 (72h27m59.630904403s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:17.782803757 (72h27m59.631410828s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:17.807443161 (72h27m59.656050239s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:18.754331863 (72h28m0.602938929s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:19.875439965 (72h28m1.724047036s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:22.232185161 (72h28m4.080792241s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:25.127197806 (72h28m6.975804875s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:25.352395099 (72h28m7.201002177s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:29.802237912 (72h28m11.650845006s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:32.623147661 (72h28m14.471754735s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:38.504797891 (72h28m20.353404969s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:40.260235005 (72h28m22.108842072s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:43.979948314 (72h28m25.828555395s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:43.979982630 (72h28m25.828589711s) [seg_pos={0 0}, upload=57962]: Segment reached optimal threshold of 13 pieces.
Jul 21 23:18:43.980515951 (72h28m25.829123035s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: rpc: dial tcp 104.36.82.5:4840: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 21 23:18:43.980555334 (72h28m25.829162419s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV): protocol: write tcp 159.223.143.39:35166->104.36.82.5:4837: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 21 23:18:43.980603676 (72h28m25.829210767s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: write tcp 159.223.143.39:59214->104.36.82.12:4149: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 21 23:18:43.980622868 (72h28m25.82922994s) [seg_pos={0 0}, upload=57962]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 21 23:18:43.980694561 (72h28m25.829301646s) [upload=57962]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 21 23:18:43.980699451 (72h28m25.829306525s) [upload=57962]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 21 23:18:44.029370168 (72h28m25.877977237s) [upload=57962]: Done uploading segments...
Jul 21 23:18:44.029388033 (72h28m25.877995113s) [upload=57962]: Upload finished. err: <nil>
Duration: 638.381384ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................] duration:
560.375827ms
FAIL (2025-07-21 23:53:49 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: write tcp 159.223.143.39:45270->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 21 23:17:24.165004211 (72h27m6.013611298s) [seg_pos={0 0}, upload=57949]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 21 23:17:24.165120348 (72h27m6.013727437s) [upload=57949]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 21 23:17:24.165126833 (72h27m6.013733917s) [upload=57949]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 21 23:17:24.212200709 (72h27m6.060807788s) [upload=57949]: Done uploading segments...
Jul 21 23:17:24.212223698 (72h27m6.060830779s) [upload=57949]: Upload finished. err: <nil>
Jul 21 23:18:17.656177722 (72h27m59.504784801s) [upload=57962]: Starting upload
Jul 21 23:18:17.656257051 (72h27m59.504864134s) [upload=57962]: Done starting upload
Jul 21 23:18:17.656517180 (72h27m59.505124252s) [upload=57962]: Uploading segments...
Jul 21 23:18:17.656648005 (72h27m59.505255091s) [upload=57962]: Got next segment. Inline: false
Jul 21 23:18:17.656730310 (72h27m59.505337381s) [upload=57962]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 21 23:18:17.656755643 (72h27m59.505362732s) [upload=57962]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 21 23:18:17.688469593 (72h27m59.537076677s) [seg_pos={0 0}, upload=57962]: Begin upload segment...
Jul 21 23:18:17.688525948 (72h27m59.537133018s) [seg_pos={0 0}, upload=57962]: Done begin upload segment.
Jul 21 23:18:17.688554683 (72h27m59.53716178s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.688567851 (72h27m59.537174942s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689009443 (72h27m59.537616529s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689160244 (72h27m59.537767326s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689433427 (72h27m59.538040502s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689720722 (72h27m59.538327803s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689914229 (72h27m59.538521313s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690190143 (72h27m59.538797225s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690301325 (72h27m59.538908411s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690627802 (72h27m59.5392349s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690668175 (72h27m59.53927526s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690952592 (72h27m59.539559677s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.691126121 (72h27m59.539733203s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.691329514 (72h27m59.539936601s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.691589223 (72h27m59.540196305s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.691703336 (72h27m59.54031042s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.774337466 (72h27m59.622944538s) [upload=57962]: Next returned nil segment
Jul 21 23:18:17.774357849 (72h27m59.622964934s) [upload=57962]: Waiting for error group managing segments...
Jul 21 23:18:17.782297334 (72h27m59.630904403s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:17.782803757 (72h27m59.631410828s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:17.807443161 (72h27m59.656050239s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:18.754331863 (72h28m0.602938929s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:19.875439965 (72h28m1.724047036s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:22.232185161 (72h28m4.080792241s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:25.127197806 (72h28m6.975804875s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:25.352395099 (72h28m7.201002177s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:29.802237912 (72h28m11.650845006s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:32.623147661 (72h28m14.471754735s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:38.504797891 (72h28m20.353404969s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:40.260235005 (72h28m22.108842072s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:43.979948314 (72h28m25.828555395s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:43.979982630 (72h28m25.828589711s) [seg_pos={0 0}, upload=57962]: Segment reached optimal threshold of 13 pieces.
Jul 21 23:18:43.980515951 (72h28m25.829123035s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: rpc: dial tcp 104.36.82.5:4840: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 21 23:18:43.980555334 (72h28m25.829162419s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV): protocol: write tcp 159.223.143.39:35166->104.36.82.5:4837: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 21 23:18:43.980603676 (72h28m25.829210767s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: write tcp 159.223.143.39:59214->104.36.82.12:4149: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 21 23:18:43.980622868 (72h28m25.82922994s) [seg_pos={0 0}, upload=57962]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 21 23:18:43.980694561 (72h28m25.829301646s) [upload=57962]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 21 23:18:43.980699451 (72h28m25.829306525s) [upload=57962]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 21 23:18:44.029370168 (72h28m25.877977237s) [upload=57962]: Done uploading segments...
Jul 21 23:18:44.029388033 (72h28m25.877995113s) [upload=57962]: Upload finished. err: <nil>
Duration: 560.403891ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......X.....................................................] duration:
703.062812ms
FAIL (2025-07-21 23:52:47 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: write tcp 159.223.143.39:45270->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 21 23:17:24.165004211 (72h27m6.013611298s) [seg_pos={0 0}, upload=57949]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 21 23:17:24.165120348 (72h27m6.013727437s) [upload=57949]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 21 23:17:24.165126833 (72h27m6.013733917s) [upload=57949]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 21 23:17:24.212200709 (72h27m6.060807788s) [upload=57949]: Done uploading segments...
Jul 21 23:17:24.212223698 (72h27m6.060830779s) [upload=57949]: Upload finished. err: <nil>
Jul 21 23:18:17.656177722 (72h27m59.504784801s) [upload=57962]: Starting upload
Jul 21 23:18:17.656257051 (72h27m59.504864134s) [upload=57962]: Done starting upload
Jul 21 23:18:17.656517180 (72h27m59.505124252s) [upload=57962]: Uploading segments...
Jul 21 23:18:17.656648005 (72h27m59.505255091s) [upload=57962]: Got next segment. Inline: false
Jul 21 23:18:17.656730310 (72h27m59.505337381s) [upload=57962]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 21 23:18:17.656755643 (72h27m59.505362732s) [upload=57962]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 21 23:18:17.688469593 (72h27m59.537076677s) [seg_pos={0 0}, upload=57962]: Begin upload segment...
Jul 21 23:18:17.688525948 (72h27m59.537133018s) [seg_pos={0 0}, upload=57962]: Done begin upload segment.
Jul 21 23:18:17.688554683 (72h27m59.53716178s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.688567851 (72h27m59.537174942s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689009443 (72h27m59.537616529s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689160244 (72h27m59.537767326s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689433427 (72h27m59.538040502s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689720722 (72h27m59.538327803s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689914229 (72h27m59.538521313s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690190143 (72h27m59.538797225s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690301325 (72h27m59.538908411s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690627802 (72h27m59.5392349s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690668175 (72h27m59.53927526s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690952592 (72h27m59.539559677s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.691126121 (72h27m59.539733203s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.691329514 (72h27m59.539936601s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.691589223 (72h27m59.540196305s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.691703336 (72h27m59.54031042s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.774337466 (72h27m59.622944538s) [upload=57962]: Next returned nil segment
Jul 21 23:18:17.774357849 (72h27m59.622964934s) [upload=57962]: Waiting for error group managing segments...
Jul 21 23:18:17.782297334 (72h27m59.630904403s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:17.782803757 (72h27m59.631410828s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:17.807443161 (72h27m59.656050239s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:18.754331863 (72h28m0.602938929s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:19.875439965 (72h28m1.724047036s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:22.232185161 (72h28m4.080792241s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:25.127197806 (72h28m6.975804875s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:25.352395099 (72h28m7.201002177s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:29.802237912 (72h28m11.650845006s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:32.623147661 (72h28m14.471754735s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:38.504797891 (72h28m20.353404969s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:40.260235005 (72h28m22.108842072s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:43.979948314 (72h28m25.828555395s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:43.979982630 (72h28m25.828589711s) [seg_pos={0 0}, upload=57962]: Segment reached optimal threshold of 13 pieces.
Jul 21 23:18:43.980515951 (72h28m25.829123035s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: rpc: dial tcp 104.36.82.5:4840: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 21 23:18:43.980555334 (72h28m25.829162419s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV): protocol: write tcp 159.223.143.39:35166->104.36.82.5:4837: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 21 23:18:43.980603676 (72h28m25.829210767s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: write tcp 159.223.143.39:59214->104.36.82.12:4149: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 21 23:18:43.980622868 (72h28m25.82922994s) [seg_pos={0 0}, upload=57962]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 21 23:18:43.980694561 (72h28m25.829301646s) [upload=57962]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 21 23:18:43.980699451 (72h28m25.829306525s) [upload=57962]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 21 23:18:44.029370168 (72h28m25.877977237s) [upload=57962]: Done uploading segments...
Jul 21 23:18:44.029388033 (72h28m25.877995113s) [upload=57962]: Upload finished. err: <nil>
Duration: 703.087556ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......X....................................................] duration:
430.580105ms
FAIL (2025-07-21 23:51:49 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: write tcp 159.223.143.39:45270->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 21 23:17:24.165004211 (72h27m6.013611298s) [seg_pos={0 0}, upload=57949]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 21 23:17:24.165120348 (72h27m6.013727437s) [upload=57949]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 21 23:17:24.165126833 (72h27m6.013733917s) [upload=57949]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 21 23:17:24.212200709 (72h27m6.060807788s) [upload=57949]: Done uploading segments...
Jul 21 23:17:24.212223698 (72h27m6.060830779s) [upload=57949]: Upload finished. err: <nil>
Jul 21 23:18:17.656177722 (72h27m59.504784801s) [upload=57962]: Starting upload
Jul 21 23:18:17.656257051 (72h27m59.504864134s) [upload=57962]: Done starting upload
Jul 21 23:18:17.656517180 (72h27m59.505124252s) [upload=57962]: Uploading segments...
Jul 21 23:18:17.656648005 (72h27m59.505255091s) [upload=57962]: Got next segment. Inline: false
Jul 21 23:18:17.656730310 (72h27m59.505337381s) [upload=57962]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 21 23:18:17.656755643 (72h27m59.505362732s) [upload=57962]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 21 23:18:17.688469593 (72h27m59.537076677s) [seg_pos={0 0}, upload=57962]: Begin upload segment...
Jul 21 23:18:17.688525948 (72h27m59.537133018s) [seg_pos={0 0}, upload=57962]: Done begin upload segment.
Jul 21 23:18:17.688554683 (72h27m59.53716178s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.688567851 (72h27m59.537174942s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689009443 (72h27m59.537616529s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689160244 (72h27m59.537767326s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689433427 (72h27m59.538040502s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689720722 (72h27m59.538327803s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689914229 (72h27m59.538521313s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690190143 (72h27m59.538797225s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690301325 (72h27m59.538908411s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690627802 (72h27m59.5392349s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690668175 (72h27m59.53927526s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690952592 (72h27m59.539559677s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.691126121 (72h27m59.539733203s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.691329514 (72h27m59.539936601s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.691589223 (72h27m59.540196305s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.691703336 (72h27m59.54031042s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.774337466 (72h27m59.622944538s) [upload=57962]: Next returned nil segment
Jul 21 23:18:17.774357849 (72h27m59.622964934s) [upload=57962]: Waiting for error group managing segments...
Jul 21 23:18:17.782297334 (72h27m59.630904403s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:17.782803757 (72h27m59.631410828s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:17.807443161 (72h27m59.656050239s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:18.754331863 (72h28m0.602938929s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:19.875439965 (72h28m1.724047036s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:22.232185161 (72h28m4.080792241s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:25.127197806 (72h28m6.975804875s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:25.352395099 (72h28m7.201002177s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:29.802237912 (72h28m11.650845006s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:32.623147661 (72h28m14.471754735s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:38.504797891 (72h28m20.353404969s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:40.260235005 (72h28m22.108842072s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:43.979948314 (72h28m25.828555395s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:43.979982630 (72h28m25.828589711s) [seg_pos={0 0}, upload=57962]: Segment reached optimal threshold of 13 pieces.
Jul 21 23:18:43.980515951 (72h28m25.829123035s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: rpc: dial tcp 104.36.82.5:4840: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 21 23:18:43.980555334 (72h28m25.829162419s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV): protocol: write tcp 159.223.143.39:35166->104.36.82.5:4837: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 21 23:18:43.980603676 (72h28m25.829210767s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: write tcp 159.223.143.39:59214->104.36.82.12:4149: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 21 23:18:43.980622868 (72h28m25.82922994s) [seg_pos={0 0}, upload=57962]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 21 23:18:43.980694561 (72h28m25.829301646s) [upload=57962]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 21 23:18:43.980699451 (72h28m25.829306525s) [upload=57962]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 21 23:18:44.029370168 (72h28m25.877977237s) [upload=57962]: Done uploading segments...
Jul 21 23:18:44.029388033 (72h28m25.877995113s) [upload=57962]: Upload finished. err: <nil>
Duration: 430.611196ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
21.848172787s
FAIL (2025-07-21 23:51:27 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 21.848233406s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........X...................................................] duration:
941.419976ms
FAIL (2025-07-21 23:51:21 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: write tcp 159.223.143.39:45270->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 21 23:17:24.165004211 (72h27m6.013611298s) [seg_pos={0 0}, upload=57949]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 21 23:17:24.165120348 (72h27m6.013727437s) [upload=57949]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 21 23:17:24.165126833 (72h27m6.013733917s) [upload=57949]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 21 23:17:24.212200709 (72h27m6.060807788s) [upload=57949]: Done uploading segments...
Jul 21 23:17:24.212223698 (72h27m6.060830779s) [upload=57949]: Upload finished. err: <nil>
Jul 21 23:18:17.656177722 (72h27m59.504784801s) [upload=57962]: Starting upload
Jul 21 23:18:17.656257051 (72h27m59.504864134s) [upload=57962]: Done starting upload
Jul 21 23:18:17.656517180 (72h27m59.505124252s) [upload=57962]: Uploading segments...
Jul 21 23:18:17.656648005 (72h27m59.505255091s) [upload=57962]: Got next segment. Inline: false
Jul 21 23:18:17.656730310 (72h27m59.505337381s) [upload=57962]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 21 23:18:17.656755643 (72h27m59.505362732s) [upload=57962]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 21 23:18:17.688469593 (72h27m59.537076677s) [seg_pos={0 0}, upload=57962]: Begin upload segment...
Jul 21 23:18:17.688525948 (72h27m59.537133018s) [seg_pos={0 0}, upload=57962]: Done begin upload segment.
Jul 21 23:18:17.688554683 (72h27m59.53716178s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.688567851 (72h27m59.537174942s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689009443 (72h27m59.537616529s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689160244 (72h27m59.537767326s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689433427 (72h27m59.538040502s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689720722 (72h27m59.538327803s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689914229 (72h27m59.538521313s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690190143 (72h27m59.538797225s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690301325 (72h27m59.538908411s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690627802 (72h27m59.5392349s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690668175 (72h27m59.53927526s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690952592 (72h27m59.539559677s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.691126121 (72h27m59.539733203s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.691329514 (72h27m59.539936601s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.691589223 (72h27m59.540196305s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.691703336 (72h27m59.54031042s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.774337466 (72h27m59.622944538s) [upload=57962]: Next returned nil segment
Jul 21 23:18:17.774357849 (72h27m59.622964934s) [upload=57962]: Waiting for error group managing segments...
Jul 21 23:18:17.782297334 (72h27m59.630904403s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:17.782803757 (72h27m59.631410828s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:17.807443161 (72h27m59.656050239s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:18.754331863 (72h28m0.602938929s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:19.875439965 (72h28m1.724047036s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:22.232185161 (72h28m4.080792241s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:25.127197806 (72h28m6.975804875s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:25.352395099 (72h28m7.201002177s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:29.802237912 (72h28m11.650845006s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:32.623147661 (72h28m14.471754735s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:38.504797891 (72h28m20.353404969s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:40.260235005 (72h28m22.108842072s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:43.979948314 (72h28m25.828555395s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:43.979982630 (72h28m25.828589711s) [seg_pos={0 0}, upload=57962]: Segment reached optimal threshold of 13 pieces.
Jul 21 23:18:43.980515951 (72h28m25.829123035s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: rpc: dial tcp 104.36.82.5:4840: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 21 23:18:43.980555334 (72h28m25.829162419s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV): protocol: write tcp 159.223.143.39:35166->104.36.82.5:4837: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 21 23:18:43.980603676 (72h28m25.829210767s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: write tcp 159.223.143.39:59214->104.36.82.12:4149: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 21 23:18:43.980622868 (72h28m25.82922994s) [seg_pos={0 0}, upload=57962]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 21 23:18:43.980694561 (72h28m25.829301646s) [upload=57962]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 21 23:18:43.980699451 (72h28m25.829306525s) [upload=57962]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 21 23:18:44.029370168 (72h28m25.877977237s) [upload=57962]: Done uploading segments...
Jul 21 23:18:44.029388033 (72h28m25.877995113s) [upload=57962]: Upload finished. err: <nil>
Duration: 941.451146ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........X..................................................] duration:
595.694235ms
FAIL (2025-07-21 23:50:31 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: write tcp 159.223.143.39:45270->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 21 23:17:24.165004211 (72h27m6.013611298s) [seg_pos={0 0}, upload=57949]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 21 23:17:24.165120348 (72h27m6.013727437s) [upload=57949]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 21 23:17:24.165126833 (72h27m6.013733917s) [upload=57949]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 21 23:17:24.212200709 (72h27m6.060807788s) [upload=57949]: Done uploading segments...
Jul 21 23:17:24.212223698 (72h27m6.060830779s) [upload=57949]: Upload finished. err: <nil>
Jul 21 23:18:17.656177722 (72h27m59.504784801s) [upload=57962]: Starting upload
Jul 21 23:18:17.656257051 (72h27m59.504864134s) [upload=57962]: Done starting upload
Jul 21 23:18:17.656517180 (72h27m59.505124252s) [upload=57962]: Uploading segments...
Jul 21 23:18:17.656648005 (72h27m59.505255091s) [upload=57962]: Got next segment. Inline: false
Jul 21 23:18:17.656730310 (72h27m59.505337381s) [upload=57962]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 21 23:18:17.656755643 (72h27m59.505362732s) [upload=57962]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 21 23:18:17.688469593 (72h27m59.537076677s) [seg_pos={0 0}, upload=57962]: Begin upload segment...
Jul 21 23:18:17.688525948 (72h27m59.537133018s) [seg_pos={0 0}, upload=57962]: Done begin upload segment.
Jul 21 23:18:17.688554683 (72h27m59.53716178s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.688567851 (72h27m59.537174942s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689009443 (72h27m59.537616529s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689160244 (72h27m59.537767326s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689433427 (72h27m59.538040502s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689720722 (72h27m59.538327803s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689914229 (72h27m59.538521313s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690190143 (72h27m59.538797225s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690301325 (72h27m59.538908411s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690627802 (72h27m59.5392349s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690668175 (72h27m59.53927526s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690952592 (72h27m59.539559677s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.691126121 (72h27m59.539733203s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.691329514 (72h27m59.539936601s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.691589223 (72h27m59.540196305s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.691703336 (72h27m59.54031042s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.774337466 (72h27m59.622944538s) [upload=57962]: Next returned nil segment
Jul 21 23:18:17.774357849 (72h27m59.622964934s) [upload=57962]: Waiting for error group managing segments...
Jul 21 23:18:17.782297334 (72h27m59.630904403s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:17.782803757 (72h27m59.631410828s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:17.807443161 (72h27m59.656050239s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:18.754331863 (72h28m0.602938929s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:19.875439965 (72h28m1.724047036s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:22.232185161 (72h28m4.080792241s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:25.127197806 (72h28m6.975804875s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:25.352395099 (72h28m7.201002177s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:29.802237912 (72h28m11.650845006s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:32.623147661 (72h28m14.471754735s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:38.504797891 (72h28m20.353404969s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:40.260235005 (72h28m22.108842072s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:43.979948314 (72h28m25.828555395s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:43.979982630 (72h28m25.828589711s) [seg_pos={0 0}, upload=57962]: Segment reached optimal threshold of 13 pieces.
Jul 21 23:18:43.980515951 (72h28m25.829123035s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: rpc: dial tcp 104.36.82.5:4840: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 21 23:18:43.980555334 (72h28m25.829162419s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV): protocol: write tcp 159.223.143.39:35166->104.36.82.5:4837: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 21 23:18:43.980603676 (72h28m25.829210767s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: write tcp 159.223.143.39:59214->104.36.82.12:4149: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 21 23:18:43.980622868 (72h28m25.82922994s) [seg_pos={0 0}, upload=57962]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 21 23:18:43.980694561 (72h28m25.829301646s) [upload=57962]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 21 23:18:43.980699451 (72h28m25.829306525s) [upload=57962]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 21 23:18:44.029370168 (72h28m25.877977237s) [upload=57962]: Done uploading segments...
Jul 21 23:18:44.029388033 (72h28m25.877995113s) [upload=57962]: Upload finished. err: <nil>
Duration: 595.729958ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
825.562954ms
FAIL (2025-07-21 23:50:11 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 825.652292ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........X.................................................] duration:
1.319676968s
FAIL (2025-07-21 23:49:37 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: write tcp 159.223.143.39:45270->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 21 23:17:24.165004211 (72h27m6.013611298s) [seg_pos={0 0}, upload=57949]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 21 23:17:24.165120348 (72h27m6.013727437s) [upload=57949]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 21 23:17:24.165126833 (72h27m6.013733917s) [upload=57949]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 21 23:17:24.212200709 (72h27m6.060807788s) [upload=57949]: Done uploading segments...
Jul 21 23:17:24.212223698 (72h27m6.060830779s) [upload=57949]: Upload finished. err: <nil>
Jul 21 23:18:17.656177722 (72h27m59.504784801s) [upload=57962]: Starting upload
Jul 21 23:18:17.656257051 (72h27m59.504864134s) [upload=57962]: Done starting upload
Jul 21 23:18:17.656517180 (72h27m59.505124252s) [upload=57962]: Uploading segments...
Jul 21 23:18:17.656648005 (72h27m59.505255091s) [upload=57962]: Got next segment. Inline: false
Jul 21 23:18:17.656730310 (72h27m59.505337381s) [upload=57962]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 21 23:18:17.656755643 (72h27m59.505362732s) [upload=57962]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 21 23:18:17.688469593 (72h27m59.537076677s) [seg_pos={0 0}, upload=57962]: Begin upload segment...
Jul 21 23:18:17.688525948 (72h27m59.537133018s) [seg_pos={0 0}, upload=57962]: Done begin upload segment.
Jul 21 23:18:17.688554683 (72h27m59.53716178s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.688567851 (72h27m59.537174942s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689009443 (72h27m59.537616529s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689160244 (72h27m59.537767326s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689433427 (72h27m59.538040502s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689720722 (72h27m59.538327803s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689914229 (72h27m59.538521313s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690190143 (72h27m59.538797225s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690301325 (72h27m59.538908411s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690627802 (72h27m59.5392349s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690668175 (72h27m59.53927526s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690952592 (72h27m59.539559677s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.691126121 (72h27m59.539733203s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.691329514 (72h27m59.539936601s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.691589223 (72h27m59.540196305s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.691703336 (72h27m59.54031042s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.774337466 (72h27m59.622944538s) [upload=57962]: Next returned nil segment
Jul 21 23:18:17.774357849 (72h27m59.622964934s) [upload=57962]: Waiting for error group managing segments...
Jul 21 23:18:17.782297334 (72h27m59.630904403s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:17.782803757 (72h27m59.631410828s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:17.807443161 (72h27m59.656050239s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:18.754331863 (72h28m0.602938929s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:19.875439965 (72h28m1.724047036s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:22.232185161 (72h28m4.080792241s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:25.127197806 (72h28m6.975804875s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:25.352395099 (72h28m7.201002177s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:29.802237912 (72h28m11.650845006s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:32.623147661 (72h28m14.471754735s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:38.504797891 (72h28m20.353404969s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:40.260235005 (72h28m22.108842072s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:43.979948314 (72h28m25.828555395s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:43.979982630 (72h28m25.828589711s) [seg_pos={0 0}, upload=57962]: Segment reached optimal threshold of 13 pieces.
Jul 21 23:18:43.980515951 (72h28m25.829123035s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: rpc: dial tcp 104.36.82.5:4840: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 21 23:18:43.980555334 (72h28m25.829162419s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV): protocol: write tcp 159.223.143.39:35166->104.36.82.5:4837: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 21 23:18:43.980603676 (72h28m25.829210767s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: write tcp 159.223.143.39:59214->104.36.82.12:4149: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 21 23:18:43.980622868 (72h28m25.82922994s) [seg_pos={0 0}, upload=57962]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 21 23:18:43.980694561 (72h28m25.829301646s) [upload=57962]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 21 23:18:43.980699451 (72h28m25.829306525s) [upload=57962]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 21 23:18:44.029370168 (72h28m25.877977237s) [upload=57962]: Done uploading segments...
Jul 21 23:18:44.029388033 (72h28m25.877995113s) [upload=57962]: Upload finished. err: <nil>
Duration: 1.319711039s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
4.389324481s
FAIL (2025-07-21 23:49:18 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 4.389399721s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [X..X........................................................] duration:
10.541937607s
FAIL (2025-07-21 23:49:04 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 10.541958958s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........X................................................] duration:
1.400073074s
FAIL (2025-07-21 23:48:50 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: write tcp 159.223.143.39:45270->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 21 23:17:24.165004211 (72h27m6.013611298s) [seg_pos={0 0}, upload=57949]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 21 23:17:24.165120348 (72h27m6.013727437s) [upload=57949]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 21 23:17:24.165126833 (72h27m6.013733917s) [upload=57949]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 21 23:17:24.212200709 (72h27m6.060807788s) [upload=57949]: Done uploading segments...
Jul 21 23:17:24.212223698 (72h27m6.060830779s) [upload=57949]: Upload finished. err: <nil>
Jul 21 23:18:17.656177722 (72h27m59.504784801s) [upload=57962]: Starting upload
Jul 21 23:18:17.656257051 (72h27m59.504864134s) [upload=57962]: Done starting upload
Jul 21 23:18:17.656517180 (72h27m59.505124252s) [upload=57962]: Uploading segments...
Jul 21 23:18:17.656648005 (72h27m59.505255091s) [upload=57962]: Got next segment. Inline: false
Jul 21 23:18:17.656730310 (72h27m59.505337381s) [upload=57962]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 21 23:18:17.656755643 (72h27m59.505362732s) [upload=57962]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 21 23:18:17.688469593 (72h27m59.537076677s) [seg_pos={0 0}, upload=57962]: Begin upload segment...
Jul 21 23:18:17.688525948 (72h27m59.537133018s) [seg_pos={0 0}, upload=57962]: Done begin upload segment.
Jul 21 23:18:17.688554683 (72h27m59.53716178s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.688567851 (72h27m59.537174942s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689009443 (72h27m59.537616529s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689160244 (72h27m59.537767326s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689433427 (72h27m59.538040502s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689720722 (72h27m59.538327803s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689914229 (72h27m59.538521313s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690190143 (72h27m59.538797225s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690301325 (72h27m59.538908411s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690627802 (72h27m59.5392349s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690668175 (72h27m59.53927526s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690952592 (72h27m59.539559677s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.691126121 (72h27m59.539733203s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.691329514 (72h27m59.539936601s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.691589223 (72h27m59.540196305s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.691703336 (72h27m59.54031042s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.774337466 (72h27m59.622944538s) [upload=57962]: Next returned nil segment
Jul 21 23:18:17.774357849 (72h27m59.622964934s) [upload=57962]: Waiting for error group managing segments...
Jul 21 23:18:17.782297334 (72h27m59.630904403s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:17.782803757 (72h27m59.631410828s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:17.807443161 (72h27m59.656050239s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:18.754331863 (72h28m0.602938929s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:19.875439965 (72h28m1.724047036s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:22.232185161 (72h28m4.080792241s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:25.127197806 (72h28m6.975804875s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:25.352395099 (72h28m7.201002177s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:29.802237912 (72h28m11.650845006s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:32.623147661 (72h28m14.471754735s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:38.504797891 (72h28m20.353404969s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:40.260235005 (72h28m22.108842072s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:43.979948314 (72h28m25.828555395s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:43.979982630 (72h28m25.828589711s) [seg_pos={0 0}, upload=57962]: Segment reached optimal threshold of 13 pieces.
Jul 21 23:18:43.980515951 (72h28m25.829123035s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: rpc: dial tcp 104.36.82.5:4840: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 21 23:18:43.980555334 (72h28m25.829162419s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV): protocol: write tcp 159.223.143.39:35166->104.36.82.5:4837: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 21 23:18:43.980603676 (72h28m25.829210767s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: write tcp 159.223.143.39:59214->104.36.82.12:4149: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 21 23:18:43.980622868 (72h28m25.82922994s) [seg_pos={0 0}, upload=57962]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 21 23:18:43.980694561 (72h28m25.829301646s) [upload=57962]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 21 23:18:43.980699451 (72h28m25.829306525s) [upload=57962]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 21 23:18:44.029370168 (72h28m25.877977237s) [upload=57962]: Done uploading segments...
Jul 21 23:18:44.029388033 (72h28m25.877995113s) [upload=57962]: Upload finished. err: <nil>
Duration: 1.400100776s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [.X..X.......................................................] duration:
16.687000691s
FAIL (2025-07-21 23:48:10 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 16.68702536s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [............X...............................................] duration:
954.179927ms
FAIL (2025-07-21 23:48:02 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: write tcp 159.223.143.39:45270->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 21 23:17:24.165004211 (72h27m6.013611298s) [seg_pos={0 0}, upload=57949]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 21 23:17:24.165120348 (72h27m6.013727437s) [upload=57949]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 21 23:17:24.165126833 (72h27m6.013733917s) [upload=57949]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 21 23:17:24.212200709 (72h27m6.060807788s) [upload=57949]: Done uploading segments...
Jul 21 23:17:24.212223698 (72h27m6.060830779s) [upload=57949]: Upload finished. err: <nil>
Jul 21 23:18:17.656177722 (72h27m59.504784801s) [upload=57962]: Starting upload
Jul 21 23:18:17.656257051 (72h27m59.504864134s) [upload=57962]: Done starting upload
Jul 21 23:18:17.656517180 (72h27m59.505124252s) [upload=57962]: Uploading segments...
Jul 21 23:18:17.656648005 (72h27m59.505255091s) [upload=57962]: Got next segment. Inline: false
Jul 21 23:18:17.656730310 (72h27m59.505337381s) [upload=57962]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 21 23:18:17.656755643 (72h27m59.505362732s) [upload=57962]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 21 23:18:17.688469593 (72h27m59.537076677s) [seg_pos={0 0}, upload=57962]: Begin upload segment...
Jul 21 23:18:17.688525948 (72h27m59.537133018s) [seg_pos={0 0}, upload=57962]: Done begin upload segment.
Jul 21 23:18:17.688554683 (72h27m59.53716178s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.688567851 (72h27m59.537174942s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689009443 (72h27m59.537616529s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689160244 (72h27m59.537767326s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689433427 (72h27m59.538040502s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689720722 (72h27m59.538327803s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689914229 (72h27m59.538521313s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690190143 (72h27m59.538797225s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690301325 (72h27m59.538908411s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690627802 (72h27m59.5392349s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690668175 (72h27m59.53927526s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690952592 (72h27m59.539559677s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.691126121 (72h27m59.539733203s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.691329514 (72h27m59.539936601s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.691589223 (72h27m59.540196305s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.691703336 (72h27m59.54031042s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.774337466 (72h27m59.622944538s) [upload=57962]: Next returned nil segment
Jul 21 23:18:17.774357849 (72h27m59.622964934s) [upload=57962]: Waiting for error group managing segments...
Jul 21 23:18:17.782297334 (72h27m59.630904403s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:17.782803757 (72h27m59.631410828s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:17.807443161 (72h27m59.656050239s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:18.754331863 (72h28m0.602938929s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:19.875439965 (72h28m1.724047036s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:22.232185161 (72h28m4.080792241s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:25.127197806 (72h28m6.975804875s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:25.352395099 (72h28m7.201002177s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:29.802237912 (72h28m11.650845006s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:32.623147661 (72h28m14.471754735s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:38.504797891 (72h28m20.353404969s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:40.260235005 (72h28m22.108842072s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:43.979948314 (72h28m25.828555395s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:43.979982630 (72h28m25.828589711s) [seg_pos={0 0}, upload=57962]: Segment reached optimal threshold of 13 pieces.
Jul 21 23:18:43.980515951 (72h28m25.829123035s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: rpc: dial tcp 104.36.82.5:4840: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 21 23:18:43.980555334 (72h28m25.829162419s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV): protocol: write tcp 159.223.143.39:35166->104.36.82.5:4837: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 21 23:18:43.980603676 (72h28m25.829210767s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: write tcp 159.223.143.39:59214->104.36.82.12:4149: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 21 23:18:43.980622868 (72h28m25.82922994s) [seg_pos={0 0}, upload=57962]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 21 23:18:43.980694561 (72h28m25.829301646s) [upload=57962]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 21 23:18:43.980699451 (72h28m25.829306525s) [upload=57962]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 21 23:18:44.029370168 (72h28m25.877977237s) [upload=57962]: Done uploading segments...
Jul 21 23:18:44.029388033 (72h28m25.877995113s) [upload=57962]: Upload finished. err: <nil>
Duration: 954.210797ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............X..............................................] duration:
672.834589ms
FAIL (2025-07-21 23:47:42 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: write tcp 159.223.143.39:45270->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 21 23:17:24.165004211 (72h27m6.013611298s) [seg_pos={0 0}, upload=57949]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 21 23:17:24.165120348 (72h27m6.013727437s) [upload=57949]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 21 23:17:24.165126833 (72h27m6.013733917s) [upload=57949]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 21 23:17:24.212200709 (72h27m6.060807788s) [upload=57949]: Done uploading segments...
Jul 21 23:17:24.212223698 (72h27m6.060830779s) [upload=57949]: Upload finished. err: <nil>
Jul 21 23:18:17.656177722 (72h27m59.504784801s) [upload=57962]: Starting upload
Jul 21 23:18:17.656257051 (72h27m59.504864134s) [upload=57962]: Done starting upload
Jul 21 23:18:17.656517180 (72h27m59.505124252s) [upload=57962]: Uploading segments...
Jul 21 23:18:17.656648005 (72h27m59.505255091s) [upload=57962]: Got next segment. Inline: false
Jul 21 23:18:17.656730310 (72h27m59.505337381s) [upload=57962]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 21 23:18:17.656755643 (72h27m59.505362732s) [upload=57962]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 21 23:18:17.688469593 (72h27m59.537076677s) [seg_pos={0 0}, upload=57962]: Begin upload segment...
Jul 21 23:18:17.688525948 (72h27m59.537133018s) [seg_pos={0 0}, upload=57962]: Done begin upload segment.
Jul 21 23:18:17.688554683 (72h27m59.53716178s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.688567851 (72h27m59.537174942s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689009443 (72h27m59.537616529s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689160244 (72h27m59.537767326s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689433427 (72h27m59.538040502s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689720722 (72h27m59.538327803s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689914229 (72h27m59.538521313s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690190143 (72h27m59.538797225s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690301325 (72h27m59.538908411s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690627802 (72h27m59.5392349s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690668175 (72h27m59.53927526s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690952592 (72h27m59.539559677s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.691126121 (72h27m59.539733203s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.691329514 (72h27m59.539936601s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.691589223 (72h27m59.540196305s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.691703336 (72h27m59.54031042s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.774337466 (72h27m59.622944538s) [upload=57962]: Next returned nil segment
Jul 21 23:18:17.774357849 (72h27m59.622964934s) [upload=57962]: Waiting for error group managing segments...
Jul 21 23:18:17.782297334 (72h27m59.630904403s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:17.782803757 (72h27m59.631410828s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:17.807443161 (72h27m59.656050239s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:18.754331863 (72h28m0.602938929s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:19.875439965 (72h28m1.724047036s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:22.232185161 (72h28m4.080792241s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:25.127197806 (72h28m6.975804875s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:25.352395099 (72h28m7.201002177s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:29.802237912 (72h28m11.650845006s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:32.623147661 (72h28m14.471754735s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:38.504797891 (72h28m20.353404969s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:40.260235005 (72h28m22.108842072s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:43.979948314 (72h28m25.828555395s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:43.979982630 (72h28m25.828589711s) [seg_pos={0 0}, upload=57962]: Segment reached optimal threshold of 13 pieces.
Jul 21 23:18:43.980515951 (72h28m25.829123035s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: rpc: dial tcp 104.36.82.5:4840: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 21 23:18:43.980555334 (72h28m25.829162419s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV): protocol: write tcp 159.223.143.39:35166->104.36.82.5:4837: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 21 23:18:43.980603676 (72h28m25.829210767s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: write tcp 159.223.143.39:59214->104.36.82.12:4149: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 21 23:18:43.980622868 (72h28m25.82922994s) [seg_pos={0 0}, upload=57962]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 21 23:18:43.980694561 (72h28m25.829301646s) [upload=57962]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 21 23:18:43.980699451 (72h28m25.829306525s) [upload=57962]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 21 23:18:44.029370168 (72h28m25.877977237s) [upload=57962]: Done uploading segments...
Jul 21 23:18:44.029388033 (72h28m25.877995113s) [upload=57962]: Upload finished. err: <nil>
Duration: 672.864906ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [..X..X......................................................] duration:
9.165484325s
FAIL (2025-07-21 23:47:11 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 9.165533268s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............X.............................................] duration:
710.495897ms
FAIL (2025-07-21 23:46:54 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: write tcp 159.223.143.39:45270->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 21 23:17:24.165004211 (72h27m6.013611298s) [seg_pos={0 0}, upload=57949]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 21 23:17:24.165120348 (72h27m6.013727437s) [upload=57949]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 21 23:17:24.165126833 (72h27m6.013733917s) [upload=57949]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 21 23:17:24.212200709 (72h27m6.060807788s) [upload=57949]: Done uploading segments...
Jul 21 23:17:24.212223698 (72h27m6.060830779s) [upload=57949]: Upload finished. err: <nil>
Jul 21 23:18:17.656177722 (72h27m59.504784801s) [upload=57962]: Starting upload
Jul 21 23:18:17.656257051 (72h27m59.504864134s) [upload=57962]: Done starting upload
Jul 21 23:18:17.656517180 (72h27m59.505124252s) [upload=57962]: Uploading segments...
Jul 21 23:18:17.656648005 (72h27m59.505255091s) [upload=57962]: Got next segment. Inline: false
Jul 21 23:18:17.656730310 (72h27m59.505337381s) [upload=57962]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 21 23:18:17.656755643 (72h27m59.505362732s) [upload=57962]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 21 23:18:17.688469593 (72h27m59.537076677s) [seg_pos={0 0}, upload=57962]: Begin upload segment...
Jul 21 23:18:17.688525948 (72h27m59.537133018s) [seg_pos={0 0}, upload=57962]: Done begin upload segment.
Jul 21 23:18:17.688554683 (72h27m59.53716178s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.688567851 (72h27m59.537174942s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689009443 (72h27m59.537616529s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689160244 (72h27m59.537767326s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689433427 (72h27m59.538040502s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689720722 (72h27m59.538327803s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689914229 (72h27m59.538521313s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690190143 (72h27m59.538797225s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690301325 (72h27m59.538908411s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690627802 (72h27m59.5392349s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690668175 (72h27m59.53927526s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690952592 (72h27m59.539559677s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.691126121 (72h27m59.539733203s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.691329514 (72h27m59.539936601s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.691589223 (72h27m59.540196305s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.691703336 (72h27m59.54031042s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.774337466 (72h27m59.622944538s) [upload=57962]: Next returned nil segment
Jul 21 23:18:17.774357849 (72h27m59.622964934s) [upload=57962]: Waiting for error group managing segments...
Jul 21 23:18:17.782297334 (72h27m59.630904403s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:17.782803757 (72h27m59.631410828s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:17.807443161 (72h27m59.656050239s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:18.754331863 (72h28m0.602938929s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:19.875439965 (72h28m1.724047036s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:22.232185161 (72h28m4.080792241s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:25.127197806 (72h28m6.975804875s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:25.352395099 (72h28m7.201002177s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:29.802237912 (72h28m11.650845006s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:32.623147661 (72h28m14.471754735s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:38.504797891 (72h28m20.353404969s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:40.260235005 (72h28m22.108842072s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:43.979948314 (72h28m25.828555395s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:43.979982630 (72h28m25.828589711s) [seg_pos={0 0}, upload=57962]: Segment reached optimal threshold of 13 pieces.
Jul 21 23:18:43.980515951 (72h28m25.829123035s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: rpc: dial tcp 104.36.82.5:4840: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 21 23:18:43.980555334 (72h28m25.829162419s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV): protocol: write tcp 159.223.143.39:35166->104.36.82.5:4837: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 21 23:18:43.980603676 (72h28m25.829210767s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: write tcp 159.223.143.39:59214->104.36.82.12:4149: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 21 23:18:43.980622868 (72h28m25.82922994s) [seg_pos={0 0}, upload=57962]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 21 23:18:43.980694561 (72h28m25.829301646s) [upload=57962]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 21 23:18:43.980699451 (72h28m25.829306525s) [upload=57962]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 21 23:18:44.029370168 (72h28m25.877977237s) [upload=57962]: Done uploading segments...
Jul 21 23:18:44.029388033 (72h28m25.877995113s) [upload=57962]: Upload finished. err: <nil>
Duration: 710.52715ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [...X..X.....................................................] duration:
24.506068301s
FAIL (2025-07-21 23:46:24 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 24.506105258s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............X............................................] duration:
845.516136ms
FAIL (2025-07-21 23:46:12 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: write tcp 159.223.143.39:45270->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 21 23:17:24.165004211 (72h27m6.013611298s) [seg_pos={0 0}, upload=57949]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 21 23:17:24.165120348 (72h27m6.013727437s) [upload=57949]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 21 23:17:24.165126833 (72h27m6.013733917s) [upload=57949]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 21 23:17:24.212200709 (72h27m6.060807788s) [upload=57949]: Done uploading segments...
Jul 21 23:17:24.212223698 (72h27m6.060830779s) [upload=57949]: Upload finished. err: <nil>
Jul 21 23:18:17.656177722 (72h27m59.504784801s) [upload=57962]: Starting upload
Jul 21 23:18:17.656257051 (72h27m59.504864134s) [upload=57962]: Done starting upload
Jul 21 23:18:17.656517180 (72h27m59.505124252s) [upload=57962]: Uploading segments...
Jul 21 23:18:17.656648005 (72h27m59.505255091s) [upload=57962]: Got next segment. Inline: false
Jul 21 23:18:17.656730310 (72h27m59.505337381s) [upload=57962]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 21 23:18:17.656755643 (72h27m59.505362732s) [upload=57962]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 21 23:18:17.688469593 (72h27m59.537076677s) [seg_pos={0 0}, upload=57962]: Begin upload segment...
Jul 21 23:18:17.688525948 (72h27m59.537133018s) [seg_pos={0 0}, upload=57962]: Done begin upload segment.
Jul 21 23:18:17.688554683 (72h27m59.53716178s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.688567851 (72h27m59.537174942s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689009443 (72h27m59.537616529s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689160244 (72h27m59.537767326s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689433427 (72h27m59.538040502s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689720722 (72h27m59.538327803s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689914229 (72h27m59.538521313s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690190143 (72h27m59.538797225s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690301325 (72h27m59.538908411s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690627802 (72h27m59.5392349s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690668175 (72h27m59.53927526s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690952592 (72h27m59.539559677s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.691126121 (72h27m59.539733203s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.691329514 (72h27m59.539936601s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.691589223 (72h27m59.540196305s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.691703336 (72h27m59.54031042s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.774337466 (72h27m59.622944538s) [upload=57962]: Next returned nil segment
Jul 21 23:18:17.774357849 (72h27m59.622964934s) [upload=57962]: Waiting for error group managing segments...
Jul 21 23:18:17.782297334 (72h27m59.630904403s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:17.782803757 (72h27m59.631410828s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:17.807443161 (72h27m59.656050239s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:18.754331863 (72h28m0.602938929s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:19.875439965 (72h28m1.724047036s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:22.232185161 (72h28m4.080792241s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:25.127197806 (72h28m6.975804875s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:25.352395099 (72h28m7.201002177s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:29.802237912 (72h28m11.650845006s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:32.623147661 (72h28m14.471754735s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:38.504797891 (72h28m20.353404969s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:40.260235005 (72h28m22.108842072s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:43.979948314 (72h28m25.828555395s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:43.979982630 (72h28m25.828589711s) [seg_pos={0 0}, upload=57962]: Segment reached optimal threshold of 13 pieces.
Jul 21 23:18:43.980515951 (72h28m25.829123035s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: rpc: dial tcp 104.36.82.5:4840: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 21 23:18:43.980555334 (72h28m25.829162419s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV): protocol: write tcp 159.223.143.39:35166->104.36.82.5:4837: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 21 23:18:43.980603676 (72h28m25.829210767s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: write tcp 159.223.143.39:59214->104.36.82.12:4149: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 21 23:18:43.980622868 (72h28m25.82922994s) [seg_pos={0 0}, upload=57962]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 21 23:18:43.980694561 (72h28m25.829301646s) [upload=57962]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 21 23:18:43.980699451 (72h28m25.829306525s) [upload=57962]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 21 23:18:44.029370168 (72h28m25.877977237s) [upload=57962]: Done uploading segments...
Jul 21 23:18:44.029388033 (72h28m25.877995113s) [upload=57962]: Upload finished. err: <nil>
Duration: 845.539618ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [................X...........................................] duration:
508.697033ms
FAIL (2025-07-21 23:45:47 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: write tcp 159.223.143.39:45270->104.36.82.9:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 21 23:17:24.165004211 (72h27m6.013611298s) [seg_pos={0 0}, upload=57949]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 21 23:17:24.165120348 (72h27m6.013727437s) [upload=57949]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 21 23:17:24.165126833 (72h27m6.013733917s) [upload=57949]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 21 23:17:24.212200709 (72h27m6.060807788s) [upload=57949]: Done uploading segments...
Jul 21 23:17:24.212223698 (72h27m6.060830779s) [upload=57949]: Upload finished. err: <nil>
Jul 21 23:18:17.656177722 (72h27m59.504784801s) [upload=57962]: Starting upload
Jul 21 23:18:17.656257051 (72h27m59.504864134s) [upload=57962]: Done starting upload
Jul 21 23:18:17.656517180 (72h27m59.505124252s) [upload=57962]: Uploading segments...
Jul 21 23:18:17.656648005 (72h27m59.505255091s) [upload=57962]: Got next segment. Inline: false
Jul 21 23:18:17.656730310 (72h27m59.505337381s) [upload=57962]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 21 23:18:17.656755643 (72h27m59.505362732s) [upload=57962]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 21 23:18:17.688469593 (72h27m59.537076677s) [seg_pos={0 0}, upload=57962]: Begin upload segment...
Jul 21 23:18:17.688525948 (72h27m59.537133018s) [seg_pos={0 0}, upload=57962]: Done begin upload segment.
Jul 21 23:18:17.688554683 (72h27m59.53716178s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.688567851 (72h27m59.537174942s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689009443 (72h27m59.537616529s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689160244 (72h27m59.537767326s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689433427 (72h27m59.538040502s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689720722 (72h27m59.538327803s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.689914229 (72h27m59.538521313s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690190143 (72h27m59.538797225s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690301325 (72h27m59.538908411s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690627802 (72h27m59.5392349s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690668175 (72h27m59.53927526s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.690952592 (72h27m59.539559677s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.691126121 (72h27m59.539733203s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.691329514 (72h27m59.539936601s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.691589223 (72h27m59.540196305s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.691703336 (72h27m59.54031042s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Uploading piece...
Jul 21 23:18:17.774337466 (72h27m59.622944538s) [upload=57962]: Next returned nil segment
Jul 21 23:18:17.774357849 (72h27m59.622964934s) [upload=57962]: Waiting for error group managing segments...
Jul 21 23:18:17.782297334 (72h27m59.630904403s) [noise=true , address=45.91.116.10:4081 , piece_id=QGWABLQFXH4BCY6VLIVRFPX6EIVIGYN5W4YM4HNZV472DJNJWAYA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:17.782803757 (72h27m59.631410828s) [noise=true , address=45.91.116.10:4082 , piece_id=U7FCZHMVO5LCXCBK4PQAIL6A5JC2JLZT2W4FITHBUL57QCHDQOWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:17.807443161 (72h27m59.656050239s) [noise=true , address=45.91.116.14:4189 , piece_id=ENMR47FPTQWQZQBWENXIDYMML3JMWZT7OQPLQ5XIYW2OFE25JTFA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:18.754331863 (72h28m0.602938929s) [noise=true , address=45.91.116.6:4974 , piece_id=D2U3FO6SP3Y23VVK23E7HPID4ATW7IF5WTEJWYJXIXECMQ76Z42Q, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:19.875439965 (72h28m1.724047036s) [noise=true , address=104.36.82.8:4972 , piece_id=RFC45ZPOE4N7EIPUJD4OWQF6LWRC4U23VGXPIDPUZO4P5RLYIEDQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:22.232185161 (72h28m4.080792241s) [noise=true , address=104.36.82.10:4077 , piece_id=DGU7NRUSSW76MVRTT4B56PK6ZJXJ56IQAHZPRV77TYYKX3XFIHZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:25.127197806 (72h28m6.975804875s) [noise=true , address=104.36.82.6:4875 , piece_id=NIUN4M3U4PL44TP2SZLPUM3M5WGJYQVPAQKWKWERC3SJHPCARZZA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:25.352395099 (72h28m7.201002177s) [noise=true , address=104.36.82.14:4223 , piece_id=6F5AHVOCOLRSPAFXL7BVRLAHBJ2WGTVR3RS5W4ZOHARLZJPDSJSQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:29.802237912 (72h28m11.650845006s) [noise=true , address=104.36.82.15:4259 , piece_id=FC32OQ7L2TVXDFGEIUJ5KEI57MTVB5RB5IRRVPW6SGTWAHSOOPOA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:32.623147661 (72h28m14.471754735s) [noise=true , address=104.36.82.3:4767 , piece_id=ZL72TANQJVRKYLUYGQPAX7DNGGKD3TTBY5HS6Q7SEX6DU6KJK3TQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:38.504797891 (72h28m20.353404969s) [noise=true , address=104.36.82.16:4293 , piece_id=3KK3I7A3DBZRXIRYWAXJJKRG7TH6LGSKUTQSEGFGX35X5A7OAMCA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:40.260235005 (72h28m22.108842072s) [noise=true , address=104.36.82.16:4295 , piece_id=IMVZ4O3RFMW2QAGGKIJ6DJ6DUDA6ADBUT5FHHUZDPJU75TI6U5UQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:43.979948314 (72h28m25.828555395s) [noise=true , address=104.36.82.9:4005 , piece_id=D64RV2XL6QVUDBMNXEU7NAJ6VHTK5CRY47B2WKWLN6RAJMHSCJWA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: <nil>
Jul 21 23:18:43.979982630 (72h28m25.828589711s) [seg_pos={0 0}, upload=57962]: Segment reached optimal threshold of 13 pieces.
Jul 21 23:18:43.980515951 (72h28m25.829123035s) [noise=true , address=104.36.82.5:4840 , piece_id=3XVG5NOLOCHDLWNC7GKPAJ4EZL7M26ENZW34KLJGZ5NRJGMVYP2A, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww): protocol: rpc: dial tcp 104.36.82.5:4840: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 21 23:18:43.980555334 (72h28m25.829162419s) [noise=true , address=104.36.82.5:4837 , piece_id=66XZNWE3RXNILN65IMNX5X7TQYCKF6QXJDFGUUIOYAYS2GQC3SJQ, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV): protocol: write tcp 159.223.143.39:35166->104.36.82.5:4837: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 21 23:18:43.980603676 (72h28m25.829210767s) [noise=true , address=104.36.82.12:4149 , piece_id=O3SWGMZ6S4SZQEGI3MXB7JVKHBJRKAQFXOKNII37VHO4BSGGLFXA, seg_pos={0 0}, upload=57962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: write tcp 159.223.143.39:59214->104.36.82.12:4149: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 21 23:18:43.980622868 (72h28m25.82922994s) [seg_pos={0 0}, upload=57962]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 21 23:18:43.980694561 (72h28m25.829301646s) [upload=57962]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 21 23:18:43.980699451 (72h28m25.829306525s) [upload=57962]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 21 23:18:44.029370168 (72h28m25.877977237s) [upload=57962]: Done uploading segments...
Jul 21 23:18:44.029388033 (72h28m25.877995113s) [upload=57962]: Upload finished. err: <nil>
Duration: 508.723921ms
core
uplink
us-select-1
|