Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
418.876083ms
FAIL (2025-07-02 03:36:20 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
ece_id=4HWNWPU27EN6NDCHLIDTA7TR6SEDEGEEH374FY42E4S2YB64JIQA, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650643965 (209h38m42.57693864s) [noise=true , address=104.36.82.6:4873 , piece_id=U5UUFIN6D3CA65BQNOBGH2PXRXXUPJIHM4ICHMT2HGTNVYMYCQDQ, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650681786 (209h38m42.576976485s) [seg_pos={0 0}, upload=167939]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jul 2 02:50:15.650762048 (209h38m42.577056758s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:50:15.650767470 (209h38m42.577062168s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:50:15.720813246 (209h38m42.647107948s) [upload=167939]: Done uploading segments...
Jul 2 02:50:15.720854178 (209h38m42.647148871s) [upload=167939]: Upload finished. err: <nil>
Jul 2 02:51:28.868670051 (209h39m55.79496473s) [upload=167956]: Starting upload
Jul 2 02:51:28.868780851 (209h39m55.795075543s) [upload=167956]: Done starting upload
Jul 2 02:51:28.868884538 (209h39m55.795179225s) [upload=167956]: Uploading segments...
Jul 2 02:51:28.869381167 (209h39m55.795675855s) [upload=167956]: Got next segment. Inline: false
Jul 2 02:51:28.869486445 (209h39m55.795781135s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 2 02:51:28.869493196 (209h39m55.795787888s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 2 02:51:28.898707948 (209h39m55.825002664s) [seg_pos={0 0}, upload=167956]: Begin upload segment...
Jul 2 02:51:28.898787644 (209h39m55.825082342s) [seg_pos={0 0}, upload=167956]: Done begin upload segment.
Jul 2 02:51:28.898825319 (209h39m55.825120018s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.898864369 (209h39m55.825159066s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899423277 (209h39m55.825717966s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899547497 (209h39m55.825842195s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899793450 (209h39m55.82608814s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900152858 (209h39m55.826447544s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900210034 (209h39m55.826504727s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900634257 (209h39m55.826928949s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900678544 (209h39m55.826973233s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900990193 (209h39m55.827284883s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901196315 (209h39m55.827491002s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901407463 (209h39m55.827702159s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901732148 (209h39m55.828026843s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901817419 (209h39m55.828112106s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902270961 (209h39m55.828565648s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902808316 (209h39m55.829103005s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:29.002833067 (209h39m55.929127747s) [upload=167956]: Next returned nil segment
Jul 2 02:51:29.002849035 (209h39m55.929143729s) [upload=167956]: Waiting for error group managing segments...
Jul 2 02:51:29.009136933 (209h39m55.935431621s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.009959595 (209h39m55.936254298s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.010805328 (209h39m55.937100001s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.011547112 (209h39m55.9378418s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012145259 (209h39m55.938439932s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012854902 (209h39m55.939149581s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013365184 (209h39m55.939659859s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013906104 (209h39m55.940200783s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.014907872 (209h39m55.941202559s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015448646 (209h39m55.941743332s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015581494 (209h39m55.941876167s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015893548 (209h39m55.942188225s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017028456 (209h39m55.943323135s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017043139 (209h39m55.943337815s) [seg_pos={0 0}, upload=167956]: Segment reached optimal threshold of 13 pieces.
Jul 2 02:51:29.017178229 (209h39m55.943472905s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017310273 (209h39m55.94360495s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017394571 (209h39m55.943689253s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017409321 (209h39m55.943704012s) [seg_pos={0 0}, upload=167956]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 2 02:51:29.017486214 (209h39m55.943780908s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:51:29.017491827 (209h39m55.943786629s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:53:50.138713219 (209h42m17.065007921s) [upload=167956]: Done uploading segments...
Jul 2 02:53:50.138739023 (209h42m17.065033711s) [upload=167956]: Upload finished. err: metaclient: context deadline exceeded
Duration: 418.906763ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
501.461548ms
FAIL (2025-07-02 03:35:15 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
ece_id=4HWNWPU27EN6NDCHLIDTA7TR6SEDEGEEH374FY42E4S2YB64JIQA, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650643965 (209h38m42.57693864s) [noise=true , address=104.36.82.6:4873 , piece_id=U5UUFIN6D3CA65BQNOBGH2PXRXXUPJIHM4ICHMT2HGTNVYMYCQDQ, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650681786 (209h38m42.576976485s) [seg_pos={0 0}, upload=167939]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jul 2 02:50:15.650762048 (209h38m42.577056758s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:50:15.650767470 (209h38m42.577062168s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:50:15.720813246 (209h38m42.647107948s) [upload=167939]: Done uploading segments...
Jul 2 02:50:15.720854178 (209h38m42.647148871s) [upload=167939]: Upload finished. err: <nil>
Jul 2 02:51:28.868670051 (209h39m55.79496473s) [upload=167956]: Starting upload
Jul 2 02:51:28.868780851 (209h39m55.795075543s) [upload=167956]: Done starting upload
Jul 2 02:51:28.868884538 (209h39m55.795179225s) [upload=167956]: Uploading segments...
Jul 2 02:51:28.869381167 (209h39m55.795675855s) [upload=167956]: Got next segment. Inline: false
Jul 2 02:51:28.869486445 (209h39m55.795781135s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 2 02:51:28.869493196 (209h39m55.795787888s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 2 02:51:28.898707948 (209h39m55.825002664s) [seg_pos={0 0}, upload=167956]: Begin upload segment...
Jul 2 02:51:28.898787644 (209h39m55.825082342s) [seg_pos={0 0}, upload=167956]: Done begin upload segment.
Jul 2 02:51:28.898825319 (209h39m55.825120018s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.898864369 (209h39m55.825159066s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899423277 (209h39m55.825717966s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899547497 (209h39m55.825842195s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899793450 (209h39m55.82608814s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900152858 (209h39m55.826447544s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900210034 (209h39m55.826504727s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900634257 (209h39m55.826928949s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900678544 (209h39m55.826973233s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900990193 (209h39m55.827284883s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901196315 (209h39m55.827491002s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901407463 (209h39m55.827702159s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901732148 (209h39m55.828026843s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901817419 (209h39m55.828112106s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902270961 (209h39m55.828565648s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902808316 (209h39m55.829103005s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:29.002833067 (209h39m55.929127747s) [upload=167956]: Next returned nil segment
Jul 2 02:51:29.002849035 (209h39m55.929143729s) [upload=167956]: Waiting for error group managing segments...
Jul 2 02:51:29.009136933 (209h39m55.935431621s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.009959595 (209h39m55.936254298s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.010805328 (209h39m55.937100001s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.011547112 (209h39m55.9378418s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012145259 (209h39m55.938439932s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012854902 (209h39m55.939149581s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013365184 (209h39m55.939659859s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013906104 (209h39m55.940200783s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.014907872 (209h39m55.941202559s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015448646 (209h39m55.941743332s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015581494 (209h39m55.941876167s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015893548 (209h39m55.942188225s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017028456 (209h39m55.943323135s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017043139 (209h39m55.943337815s) [seg_pos={0 0}, upload=167956]: Segment reached optimal threshold of 13 pieces.
Jul 2 02:51:29.017178229 (209h39m55.943472905s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017310273 (209h39m55.94360495s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017394571 (209h39m55.943689253s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017409321 (209h39m55.943704012s) [seg_pos={0 0}, upload=167956]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 2 02:51:29.017486214 (209h39m55.943780908s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:51:29.017491827 (209h39m55.943786629s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:53:50.138713219 (209h42m17.065007921s) [upload=167956]: Done uploading segments...
Jul 2 02:53:50.138739023 (209h42m17.065033711s) [upload=167956]: Upload finished. err: metaclient: context deadline exceeded
Duration: 501.488075ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................] duration:
462.280006ms
FAIL (2025-07-02 03:34:27 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
ece_id=4HWNWPU27EN6NDCHLIDTA7TR6SEDEGEEH374FY42E4S2YB64JIQA, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650643965 (209h38m42.57693864s) [noise=true , address=104.36.82.6:4873 , piece_id=U5UUFIN6D3CA65BQNOBGH2PXRXXUPJIHM4ICHMT2HGTNVYMYCQDQ, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650681786 (209h38m42.576976485s) [seg_pos={0 0}, upload=167939]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jul 2 02:50:15.650762048 (209h38m42.577056758s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:50:15.650767470 (209h38m42.577062168s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:50:15.720813246 (209h38m42.647107948s) [upload=167939]: Done uploading segments...
Jul 2 02:50:15.720854178 (209h38m42.647148871s) [upload=167939]: Upload finished. err: <nil>
Jul 2 02:51:28.868670051 (209h39m55.79496473s) [upload=167956]: Starting upload
Jul 2 02:51:28.868780851 (209h39m55.795075543s) [upload=167956]: Done starting upload
Jul 2 02:51:28.868884538 (209h39m55.795179225s) [upload=167956]: Uploading segments...
Jul 2 02:51:28.869381167 (209h39m55.795675855s) [upload=167956]: Got next segment. Inline: false
Jul 2 02:51:28.869486445 (209h39m55.795781135s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 2 02:51:28.869493196 (209h39m55.795787888s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 2 02:51:28.898707948 (209h39m55.825002664s) [seg_pos={0 0}, upload=167956]: Begin upload segment...
Jul 2 02:51:28.898787644 (209h39m55.825082342s) [seg_pos={0 0}, upload=167956]: Done begin upload segment.
Jul 2 02:51:28.898825319 (209h39m55.825120018s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.898864369 (209h39m55.825159066s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899423277 (209h39m55.825717966s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899547497 (209h39m55.825842195s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899793450 (209h39m55.82608814s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900152858 (209h39m55.826447544s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900210034 (209h39m55.826504727s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900634257 (209h39m55.826928949s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900678544 (209h39m55.826973233s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900990193 (209h39m55.827284883s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901196315 (209h39m55.827491002s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901407463 (209h39m55.827702159s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901732148 (209h39m55.828026843s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901817419 (209h39m55.828112106s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902270961 (209h39m55.828565648s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902808316 (209h39m55.829103005s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:29.002833067 (209h39m55.929127747s) [upload=167956]: Next returned nil segment
Jul 2 02:51:29.002849035 (209h39m55.929143729s) [upload=167956]: Waiting for error group managing segments...
Jul 2 02:51:29.009136933 (209h39m55.935431621s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.009959595 (209h39m55.936254298s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.010805328 (209h39m55.937100001s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.011547112 (209h39m55.9378418s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012145259 (209h39m55.938439932s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012854902 (209h39m55.939149581s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013365184 (209h39m55.939659859s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013906104 (209h39m55.940200783s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.014907872 (209h39m55.941202559s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015448646 (209h39m55.941743332s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015581494 (209h39m55.941876167s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015893548 (209h39m55.942188225s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017028456 (209h39m55.943323135s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017043139 (209h39m55.943337815s) [seg_pos={0 0}, upload=167956]: Segment reached optimal threshold of 13 pieces.
Jul 2 02:51:29.017178229 (209h39m55.943472905s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017310273 (209h39m55.94360495s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017394571 (209h39m55.943689253s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017409321 (209h39m55.943704012s) [seg_pos={0 0}, upload=167956]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 2 02:51:29.017486214 (209h39m55.943780908s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:51:29.017491827 (209h39m55.943786629s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:53:50.138713219 (209h42m17.065007921s) [upload=167956]: Done uploading segments...
Jul 2 02:53:50.138739023 (209h42m17.065033711s) [upload=167956]: Upload finished. err: metaclient: context deadline exceeded
Duration: 462.320975ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
442.986409ms
FAIL (2025-07-02 03:33:50 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
ece_id=4HWNWPU27EN6NDCHLIDTA7TR6SEDEGEEH374FY42E4S2YB64JIQA, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650643965 (209h38m42.57693864s) [noise=true , address=104.36.82.6:4873 , piece_id=U5UUFIN6D3CA65BQNOBGH2PXRXXUPJIHM4ICHMT2HGTNVYMYCQDQ, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650681786 (209h38m42.576976485s) [seg_pos={0 0}, upload=167939]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jul 2 02:50:15.650762048 (209h38m42.577056758s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:50:15.650767470 (209h38m42.577062168s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:50:15.720813246 (209h38m42.647107948s) [upload=167939]: Done uploading segments...
Jul 2 02:50:15.720854178 (209h38m42.647148871s) [upload=167939]: Upload finished. err: <nil>
Jul 2 02:51:28.868670051 (209h39m55.79496473s) [upload=167956]: Starting upload
Jul 2 02:51:28.868780851 (209h39m55.795075543s) [upload=167956]: Done starting upload
Jul 2 02:51:28.868884538 (209h39m55.795179225s) [upload=167956]: Uploading segments...
Jul 2 02:51:28.869381167 (209h39m55.795675855s) [upload=167956]: Got next segment. Inline: false
Jul 2 02:51:28.869486445 (209h39m55.795781135s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 2 02:51:28.869493196 (209h39m55.795787888s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 2 02:51:28.898707948 (209h39m55.825002664s) [seg_pos={0 0}, upload=167956]: Begin upload segment...
Jul 2 02:51:28.898787644 (209h39m55.825082342s) [seg_pos={0 0}, upload=167956]: Done begin upload segment.
Jul 2 02:51:28.898825319 (209h39m55.825120018s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.898864369 (209h39m55.825159066s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899423277 (209h39m55.825717966s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899547497 (209h39m55.825842195s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899793450 (209h39m55.82608814s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900152858 (209h39m55.826447544s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900210034 (209h39m55.826504727s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900634257 (209h39m55.826928949s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900678544 (209h39m55.826973233s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900990193 (209h39m55.827284883s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901196315 (209h39m55.827491002s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901407463 (209h39m55.827702159s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901732148 (209h39m55.828026843s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901817419 (209h39m55.828112106s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902270961 (209h39m55.828565648s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902808316 (209h39m55.829103005s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:29.002833067 (209h39m55.929127747s) [upload=167956]: Next returned nil segment
Jul 2 02:51:29.002849035 (209h39m55.929143729s) [upload=167956]: Waiting for error group managing segments...
Jul 2 02:51:29.009136933 (209h39m55.935431621s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.009959595 (209h39m55.936254298s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.010805328 (209h39m55.937100001s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.011547112 (209h39m55.9378418s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012145259 (209h39m55.938439932s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012854902 (209h39m55.939149581s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013365184 (209h39m55.939659859s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013906104 (209h39m55.940200783s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.014907872 (209h39m55.941202559s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015448646 (209h39m55.941743332s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015581494 (209h39m55.941876167s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015893548 (209h39m55.942188225s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017028456 (209h39m55.943323135s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017043139 (209h39m55.943337815s) [seg_pos={0 0}, upload=167956]: Segment reached optimal threshold of 13 pieces.
Jul 2 02:51:29.017178229 (209h39m55.943472905s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017310273 (209h39m55.94360495s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017394571 (209h39m55.943689253s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017409321 (209h39m55.943704012s) [seg_pos={0 0}, upload=167956]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 2 02:51:29.017486214 (209h39m55.943780908s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:51:29.017491827 (209h39m55.943786629s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:53:50.138713219 (209h42m17.065007921s) [upload=167956]: Done uploading segments...
Jul 2 02:53:50.138739023 (209h42m17.065033711s) [upload=167956]: Upload finished. err: metaclient: context deadline exceeded
Duration: 443.015515ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................] duration:
555.891968ms
FAIL (2025-07-02 03:32:48 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
ece_id=4HWNWPU27EN6NDCHLIDTA7TR6SEDEGEEH374FY42E4S2YB64JIQA, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650643965 (209h38m42.57693864s) [noise=true , address=104.36.82.6:4873 , piece_id=U5UUFIN6D3CA65BQNOBGH2PXRXXUPJIHM4ICHMT2HGTNVYMYCQDQ, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650681786 (209h38m42.576976485s) [seg_pos={0 0}, upload=167939]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jul 2 02:50:15.650762048 (209h38m42.577056758s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:50:15.650767470 (209h38m42.577062168s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:50:15.720813246 (209h38m42.647107948s) [upload=167939]: Done uploading segments...
Jul 2 02:50:15.720854178 (209h38m42.647148871s) [upload=167939]: Upload finished. err: <nil>
Jul 2 02:51:28.868670051 (209h39m55.79496473s) [upload=167956]: Starting upload
Jul 2 02:51:28.868780851 (209h39m55.795075543s) [upload=167956]: Done starting upload
Jul 2 02:51:28.868884538 (209h39m55.795179225s) [upload=167956]: Uploading segments...
Jul 2 02:51:28.869381167 (209h39m55.795675855s) [upload=167956]: Got next segment. Inline: false
Jul 2 02:51:28.869486445 (209h39m55.795781135s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 2 02:51:28.869493196 (209h39m55.795787888s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 2 02:51:28.898707948 (209h39m55.825002664s) [seg_pos={0 0}, upload=167956]: Begin upload segment...
Jul 2 02:51:28.898787644 (209h39m55.825082342s) [seg_pos={0 0}, upload=167956]: Done begin upload segment.
Jul 2 02:51:28.898825319 (209h39m55.825120018s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.898864369 (209h39m55.825159066s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899423277 (209h39m55.825717966s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899547497 (209h39m55.825842195s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899793450 (209h39m55.82608814s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900152858 (209h39m55.826447544s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900210034 (209h39m55.826504727s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900634257 (209h39m55.826928949s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900678544 (209h39m55.826973233s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900990193 (209h39m55.827284883s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901196315 (209h39m55.827491002s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901407463 (209h39m55.827702159s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901732148 (209h39m55.828026843s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901817419 (209h39m55.828112106s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902270961 (209h39m55.828565648s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902808316 (209h39m55.829103005s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:29.002833067 (209h39m55.929127747s) [upload=167956]: Next returned nil segment
Jul 2 02:51:29.002849035 (209h39m55.929143729s) [upload=167956]: Waiting for error group managing segments...
Jul 2 02:51:29.009136933 (209h39m55.935431621s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.009959595 (209h39m55.936254298s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.010805328 (209h39m55.937100001s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.011547112 (209h39m55.9378418s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012145259 (209h39m55.938439932s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012854902 (209h39m55.939149581s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013365184 (209h39m55.939659859s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013906104 (209h39m55.940200783s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.014907872 (209h39m55.941202559s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015448646 (209h39m55.941743332s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015581494 (209h39m55.941876167s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015893548 (209h39m55.942188225s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017028456 (209h39m55.943323135s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017043139 (209h39m55.943337815s) [seg_pos={0 0}, upload=167956]: Segment reached optimal threshold of 13 pieces.
Jul 2 02:51:29.017178229 (209h39m55.943472905s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017310273 (209h39m55.94360495s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017394571 (209h39m55.943689253s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017409321 (209h39m55.943704012s) [seg_pos={0 0}, upload=167956]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 2 02:51:29.017486214 (209h39m55.943780908s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:51:29.017491827 (209h39m55.943786629s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:53:50.138713219 (209h42m17.065007921s) [upload=167956]: Done uploading segments...
Jul 2 02:53:50.138739023 (209h42m17.065033711s) [upload=167956]: Upload finished. err: metaclient: context deadline exceeded
Duration: 555.92209ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......X.....................................................] duration:
516.647738ms
FAIL (2025-07-02 03:31:34 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
ece_id=4HWNWPU27EN6NDCHLIDTA7TR6SEDEGEEH374FY42E4S2YB64JIQA, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650643965 (209h38m42.57693864s) [noise=true , address=104.36.82.6:4873 , piece_id=U5UUFIN6D3CA65BQNOBGH2PXRXXUPJIHM4ICHMT2HGTNVYMYCQDQ, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650681786 (209h38m42.576976485s) [seg_pos={0 0}, upload=167939]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jul 2 02:50:15.650762048 (209h38m42.577056758s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:50:15.650767470 (209h38m42.577062168s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:50:15.720813246 (209h38m42.647107948s) [upload=167939]: Done uploading segments...
Jul 2 02:50:15.720854178 (209h38m42.647148871s) [upload=167939]: Upload finished. err: <nil>
Jul 2 02:51:28.868670051 (209h39m55.79496473s) [upload=167956]: Starting upload
Jul 2 02:51:28.868780851 (209h39m55.795075543s) [upload=167956]: Done starting upload
Jul 2 02:51:28.868884538 (209h39m55.795179225s) [upload=167956]: Uploading segments...
Jul 2 02:51:28.869381167 (209h39m55.795675855s) [upload=167956]: Got next segment. Inline: false
Jul 2 02:51:28.869486445 (209h39m55.795781135s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 2 02:51:28.869493196 (209h39m55.795787888s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 2 02:51:28.898707948 (209h39m55.825002664s) [seg_pos={0 0}, upload=167956]: Begin upload segment...
Jul 2 02:51:28.898787644 (209h39m55.825082342s) [seg_pos={0 0}, upload=167956]: Done begin upload segment.
Jul 2 02:51:28.898825319 (209h39m55.825120018s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.898864369 (209h39m55.825159066s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899423277 (209h39m55.825717966s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899547497 (209h39m55.825842195s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899793450 (209h39m55.82608814s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900152858 (209h39m55.826447544s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900210034 (209h39m55.826504727s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900634257 (209h39m55.826928949s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900678544 (209h39m55.826973233s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900990193 (209h39m55.827284883s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901196315 (209h39m55.827491002s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901407463 (209h39m55.827702159s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901732148 (209h39m55.828026843s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901817419 (209h39m55.828112106s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902270961 (209h39m55.828565648s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902808316 (209h39m55.829103005s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:29.002833067 (209h39m55.929127747s) [upload=167956]: Next returned nil segment
Jul 2 02:51:29.002849035 (209h39m55.929143729s) [upload=167956]: Waiting for error group managing segments...
Jul 2 02:51:29.009136933 (209h39m55.935431621s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.009959595 (209h39m55.936254298s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.010805328 (209h39m55.937100001s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.011547112 (209h39m55.9378418s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012145259 (209h39m55.938439932s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012854902 (209h39m55.939149581s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013365184 (209h39m55.939659859s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013906104 (209h39m55.940200783s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.014907872 (209h39m55.941202559s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015448646 (209h39m55.941743332s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015581494 (209h39m55.941876167s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015893548 (209h39m55.942188225s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017028456 (209h39m55.943323135s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017043139 (209h39m55.943337815s) [seg_pos={0 0}, upload=167956]: Segment reached optimal threshold of 13 pieces.
Jul 2 02:51:29.017178229 (209h39m55.943472905s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017310273 (209h39m55.94360495s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017394571 (209h39m55.943689253s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017409321 (209h39m55.943704012s) [seg_pos={0 0}, upload=167956]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 2 02:51:29.017486214 (209h39m55.943780908s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:51:29.017491827 (209h39m55.943786629s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:53:50.138713219 (209h42m17.065007921s) [upload=167956]: Done uploading segments...
Jul 2 02:53:50.138739023 (209h42m17.065033711s) [upload=167956]: Upload finished. err: metaclient: context deadline exceeded
Duration: 516.676867ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......X....................................................] duration:
636.804586ms
FAIL (2025-07-02 03:31:19 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
ece_id=4HWNWPU27EN6NDCHLIDTA7TR6SEDEGEEH374FY42E4S2YB64JIQA, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650643965 (209h38m42.57693864s) [noise=true , address=104.36.82.6:4873 , piece_id=U5UUFIN6D3CA65BQNOBGH2PXRXXUPJIHM4ICHMT2HGTNVYMYCQDQ, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650681786 (209h38m42.576976485s) [seg_pos={0 0}, upload=167939]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jul 2 02:50:15.650762048 (209h38m42.577056758s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:50:15.650767470 (209h38m42.577062168s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:50:15.720813246 (209h38m42.647107948s) [upload=167939]: Done uploading segments...
Jul 2 02:50:15.720854178 (209h38m42.647148871s) [upload=167939]: Upload finished. err: <nil>
Jul 2 02:51:28.868670051 (209h39m55.79496473s) [upload=167956]: Starting upload
Jul 2 02:51:28.868780851 (209h39m55.795075543s) [upload=167956]: Done starting upload
Jul 2 02:51:28.868884538 (209h39m55.795179225s) [upload=167956]: Uploading segments...
Jul 2 02:51:28.869381167 (209h39m55.795675855s) [upload=167956]: Got next segment. Inline: false
Jul 2 02:51:28.869486445 (209h39m55.795781135s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 2 02:51:28.869493196 (209h39m55.795787888s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 2 02:51:28.898707948 (209h39m55.825002664s) [seg_pos={0 0}, upload=167956]: Begin upload segment...
Jul 2 02:51:28.898787644 (209h39m55.825082342s) [seg_pos={0 0}, upload=167956]: Done begin upload segment.
Jul 2 02:51:28.898825319 (209h39m55.825120018s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.898864369 (209h39m55.825159066s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899423277 (209h39m55.825717966s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899547497 (209h39m55.825842195s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899793450 (209h39m55.82608814s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900152858 (209h39m55.826447544s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900210034 (209h39m55.826504727s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900634257 (209h39m55.826928949s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900678544 (209h39m55.826973233s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900990193 (209h39m55.827284883s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901196315 (209h39m55.827491002s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901407463 (209h39m55.827702159s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901732148 (209h39m55.828026843s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901817419 (209h39m55.828112106s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902270961 (209h39m55.828565648s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902808316 (209h39m55.829103005s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:29.002833067 (209h39m55.929127747s) [upload=167956]: Next returned nil segment
Jul 2 02:51:29.002849035 (209h39m55.929143729s) [upload=167956]: Waiting for error group managing segments...
Jul 2 02:51:29.009136933 (209h39m55.935431621s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.009959595 (209h39m55.936254298s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.010805328 (209h39m55.937100001s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.011547112 (209h39m55.9378418s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012145259 (209h39m55.938439932s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012854902 (209h39m55.939149581s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013365184 (209h39m55.939659859s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013906104 (209h39m55.940200783s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.014907872 (209h39m55.941202559s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015448646 (209h39m55.941743332s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015581494 (209h39m55.941876167s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015893548 (209h39m55.942188225s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017028456 (209h39m55.943323135s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017043139 (209h39m55.943337815s) [seg_pos={0 0}, upload=167956]: Segment reached optimal threshold of 13 pieces.
Jul 2 02:51:29.017178229 (209h39m55.943472905s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017310273 (209h39m55.94360495s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017394571 (209h39m55.943689253s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017409321 (209h39m55.943704012s) [seg_pos={0 0}, upload=167956]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 2 02:51:29.017486214 (209h39m55.943780908s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:51:29.017491827 (209h39m55.943786629s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:53:50.138713219 (209h42m17.065007921s) [upload=167956]: Done uploading segments...
Jul 2 02:53:50.138739023 (209h42m17.065033711s) [upload=167956]: Upload finished. err: metaclient: context deadline exceeded
Duration: 636.836811ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........X...................................................] duration:
571.072463ms
FAIL (2025-07-02 03:30:39 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
ece_id=4HWNWPU27EN6NDCHLIDTA7TR6SEDEGEEH374FY42E4S2YB64JIQA, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650643965 (209h38m42.57693864s) [noise=true , address=104.36.82.6:4873 , piece_id=U5UUFIN6D3CA65BQNOBGH2PXRXXUPJIHM4ICHMT2HGTNVYMYCQDQ, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650681786 (209h38m42.576976485s) [seg_pos={0 0}, upload=167939]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jul 2 02:50:15.650762048 (209h38m42.577056758s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:50:15.650767470 (209h38m42.577062168s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:50:15.720813246 (209h38m42.647107948s) [upload=167939]: Done uploading segments...
Jul 2 02:50:15.720854178 (209h38m42.647148871s) [upload=167939]: Upload finished. err: <nil>
Jul 2 02:51:28.868670051 (209h39m55.79496473s) [upload=167956]: Starting upload
Jul 2 02:51:28.868780851 (209h39m55.795075543s) [upload=167956]: Done starting upload
Jul 2 02:51:28.868884538 (209h39m55.795179225s) [upload=167956]: Uploading segments...
Jul 2 02:51:28.869381167 (209h39m55.795675855s) [upload=167956]: Got next segment. Inline: false
Jul 2 02:51:28.869486445 (209h39m55.795781135s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 2 02:51:28.869493196 (209h39m55.795787888s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 2 02:51:28.898707948 (209h39m55.825002664s) [seg_pos={0 0}, upload=167956]: Begin upload segment...
Jul 2 02:51:28.898787644 (209h39m55.825082342s) [seg_pos={0 0}, upload=167956]: Done begin upload segment.
Jul 2 02:51:28.898825319 (209h39m55.825120018s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.898864369 (209h39m55.825159066s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899423277 (209h39m55.825717966s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899547497 (209h39m55.825842195s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899793450 (209h39m55.82608814s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900152858 (209h39m55.826447544s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900210034 (209h39m55.826504727s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900634257 (209h39m55.826928949s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900678544 (209h39m55.826973233s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900990193 (209h39m55.827284883s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901196315 (209h39m55.827491002s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901407463 (209h39m55.827702159s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901732148 (209h39m55.828026843s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901817419 (209h39m55.828112106s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902270961 (209h39m55.828565648s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902808316 (209h39m55.829103005s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:29.002833067 (209h39m55.929127747s) [upload=167956]: Next returned nil segment
Jul 2 02:51:29.002849035 (209h39m55.929143729s) [upload=167956]: Waiting for error group managing segments...
Jul 2 02:51:29.009136933 (209h39m55.935431621s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.009959595 (209h39m55.936254298s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.010805328 (209h39m55.937100001s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.011547112 (209h39m55.9378418s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012145259 (209h39m55.938439932s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012854902 (209h39m55.939149581s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013365184 (209h39m55.939659859s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013906104 (209h39m55.940200783s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.014907872 (209h39m55.941202559s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015448646 (209h39m55.941743332s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015581494 (209h39m55.941876167s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015893548 (209h39m55.942188225s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017028456 (209h39m55.943323135s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017043139 (209h39m55.943337815s) [seg_pos={0 0}, upload=167956]: Segment reached optimal threshold of 13 pieces.
Jul 2 02:51:29.017178229 (209h39m55.943472905s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017310273 (209h39m55.94360495s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017394571 (209h39m55.943689253s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017409321 (209h39m55.943704012s) [seg_pos={0 0}, upload=167956]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 2 02:51:29.017486214 (209h39m55.943780908s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:51:29.017491827 (209h39m55.943786629s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:53:50.138713219 (209h42m17.065007921s) [upload=167956]: Done uploading segments...
Jul 2 02:53:50.138739023 (209h42m17.065033711s) [upload=167956]: Upload finished. err: metaclient: context deadline exceeded
Duration: 571.101607ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........X..................................................] duration:
461.175221ms
FAIL (2025-07-02 03:29:42 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
ece_id=4HWNWPU27EN6NDCHLIDTA7TR6SEDEGEEH374FY42E4S2YB64JIQA, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650643965 (209h38m42.57693864s) [noise=true , address=104.36.82.6:4873 , piece_id=U5UUFIN6D3CA65BQNOBGH2PXRXXUPJIHM4ICHMT2HGTNVYMYCQDQ, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650681786 (209h38m42.576976485s) [seg_pos={0 0}, upload=167939]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jul 2 02:50:15.650762048 (209h38m42.577056758s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:50:15.650767470 (209h38m42.577062168s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:50:15.720813246 (209h38m42.647107948s) [upload=167939]: Done uploading segments...
Jul 2 02:50:15.720854178 (209h38m42.647148871s) [upload=167939]: Upload finished. err: <nil>
Jul 2 02:51:28.868670051 (209h39m55.79496473s) [upload=167956]: Starting upload
Jul 2 02:51:28.868780851 (209h39m55.795075543s) [upload=167956]: Done starting upload
Jul 2 02:51:28.868884538 (209h39m55.795179225s) [upload=167956]: Uploading segments...
Jul 2 02:51:28.869381167 (209h39m55.795675855s) [upload=167956]: Got next segment. Inline: false
Jul 2 02:51:28.869486445 (209h39m55.795781135s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 2 02:51:28.869493196 (209h39m55.795787888s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 2 02:51:28.898707948 (209h39m55.825002664s) [seg_pos={0 0}, upload=167956]: Begin upload segment...
Jul 2 02:51:28.898787644 (209h39m55.825082342s) [seg_pos={0 0}, upload=167956]: Done begin upload segment.
Jul 2 02:51:28.898825319 (209h39m55.825120018s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.898864369 (209h39m55.825159066s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899423277 (209h39m55.825717966s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899547497 (209h39m55.825842195s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899793450 (209h39m55.82608814s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900152858 (209h39m55.826447544s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900210034 (209h39m55.826504727s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900634257 (209h39m55.826928949s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900678544 (209h39m55.826973233s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900990193 (209h39m55.827284883s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901196315 (209h39m55.827491002s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901407463 (209h39m55.827702159s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901732148 (209h39m55.828026843s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901817419 (209h39m55.828112106s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902270961 (209h39m55.828565648s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902808316 (209h39m55.829103005s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:29.002833067 (209h39m55.929127747s) [upload=167956]: Next returned nil segment
Jul 2 02:51:29.002849035 (209h39m55.929143729s) [upload=167956]: Waiting for error group managing segments...
Jul 2 02:51:29.009136933 (209h39m55.935431621s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.009959595 (209h39m55.936254298s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.010805328 (209h39m55.937100001s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.011547112 (209h39m55.9378418s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012145259 (209h39m55.938439932s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012854902 (209h39m55.939149581s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013365184 (209h39m55.939659859s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013906104 (209h39m55.940200783s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.014907872 (209h39m55.941202559s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015448646 (209h39m55.941743332s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015581494 (209h39m55.941876167s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015893548 (209h39m55.942188225s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017028456 (209h39m55.943323135s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017043139 (209h39m55.943337815s) [seg_pos={0 0}, upload=167956]: Segment reached optimal threshold of 13 pieces.
Jul 2 02:51:29.017178229 (209h39m55.943472905s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017310273 (209h39m55.94360495s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017394571 (209h39m55.943689253s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017409321 (209h39m55.943704012s) [seg_pos={0 0}, upload=167956]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 2 02:51:29.017486214 (209h39m55.943780908s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:51:29.017491827 (209h39m55.943786629s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:53:50.138713219 (209h42m17.065007921s) [upload=167956]: Done uploading segments...
Jul 2 02:53:50.138739023 (209h42m17.065033711s) [upload=167956]: Upload finished. err: metaclient: context deadline exceeded
Duration: 461.369058ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........X.................................................] duration:
617.853194ms
FAIL (2025-07-02 03:28:38 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
ece_id=4HWNWPU27EN6NDCHLIDTA7TR6SEDEGEEH374FY42E4S2YB64JIQA, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650643965 (209h38m42.57693864s) [noise=true , address=104.36.82.6:4873 , piece_id=U5UUFIN6D3CA65BQNOBGH2PXRXXUPJIHM4ICHMT2HGTNVYMYCQDQ, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650681786 (209h38m42.576976485s) [seg_pos={0 0}, upload=167939]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jul 2 02:50:15.650762048 (209h38m42.577056758s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:50:15.650767470 (209h38m42.577062168s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:50:15.720813246 (209h38m42.647107948s) [upload=167939]: Done uploading segments...
Jul 2 02:50:15.720854178 (209h38m42.647148871s) [upload=167939]: Upload finished. err: <nil>
Jul 2 02:51:28.868670051 (209h39m55.79496473s) [upload=167956]: Starting upload
Jul 2 02:51:28.868780851 (209h39m55.795075543s) [upload=167956]: Done starting upload
Jul 2 02:51:28.868884538 (209h39m55.795179225s) [upload=167956]: Uploading segments...
Jul 2 02:51:28.869381167 (209h39m55.795675855s) [upload=167956]: Got next segment. Inline: false
Jul 2 02:51:28.869486445 (209h39m55.795781135s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 2 02:51:28.869493196 (209h39m55.795787888s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 2 02:51:28.898707948 (209h39m55.825002664s) [seg_pos={0 0}, upload=167956]: Begin upload segment...
Jul 2 02:51:28.898787644 (209h39m55.825082342s) [seg_pos={0 0}, upload=167956]: Done begin upload segment.
Jul 2 02:51:28.898825319 (209h39m55.825120018s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.898864369 (209h39m55.825159066s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899423277 (209h39m55.825717966s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899547497 (209h39m55.825842195s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899793450 (209h39m55.82608814s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900152858 (209h39m55.826447544s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900210034 (209h39m55.826504727s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900634257 (209h39m55.826928949s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900678544 (209h39m55.826973233s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900990193 (209h39m55.827284883s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901196315 (209h39m55.827491002s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901407463 (209h39m55.827702159s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901732148 (209h39m55.828026843s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901817419 (209h39m55.828112106s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902270961 (209h39m55.828565648s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902808316 (209h39m55.829103005s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:29.002833067 (209h39m55.929127747s) [upload=167956]: Next returned nil segment
Jul 2 02:51:29.002849035 (209h39m55.929143729s) [upload=167956]: Waiting for error group managing segments...
Jul 2 02:51:29.009136933 (209h39m55.935431621s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.009959595 (209h39m55.936254298s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.010805328 (209h39m55.937100001s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.011547112 (209h39m55.9378418s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012145259 (209h39m55.938439932s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012854902 (209h39m55.939149581s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013365184 (209h39m55.939659859s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013906104 (209h39m55.940200783s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.014907872 (209h39m55.941202559s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015448646 (209h39m55.941743332s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015581494 (209h39m55.941876167s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015893548 (209h39m55.942188225s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017028456 (209h39m55.943323135s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017043139 (209h39m55.943337815s) [seg_pos={0 0}, upload=167956]: Segment reached optimal threshold of 13 pieces.
Jul 2 02:51:29.017178229 (209h39m55.943472905s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017310273 (209h39m55.94360495s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017394571 (209h39m55.943689253s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017409321 (209h39m55.943704012s) [seg_pos={0 0}, upload=167956]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 2 02:51:29.017486214 (209h39m55.943780908s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:51:29.017491827 (209h39m55.943786629s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:53:50.138713219 (209h42m17.065007921s) [upload=167956]: Done uploading segments...
Jul 2 02:53:50.138739023 (209h42m17.065033711s) [upload=167956]: Upload finished. err: metaclient: context deadline exceeded
Duration: 617.924562ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........X................................................] duration:
593.700436ms
FAIL (2025-07-02 03:28:19 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
ece_id=4HWNWPU27EN6NDCHLIDTA7TR6SEDEGEEH374FY42E4S2YB64JIQA, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650643965 (209h38m42.57693864s) [noise=true , address=104.36.82.6:4873 , piece_id=U5UUFIN6D3CA65BQNOBGH2PXRXXUPJIHM4ICHMT2HGTNVYMYCQDQ, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650681786 (209h38m42.576976485s) [seg_pos={0 0}, upload=167939]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jul 2 02:50:15.650762048 (209h38m42.577056758s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:50:15.650767470 (209h38m42.577062168s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:50:15.720813246 (209h38m42.647107948s) [upload=167939]: Done uploading segments...
Jul 2 02:50:15.720854178 (209h38m42.647148871s) [upload=167939]: Upload finished. err: <nil>
Jul 2 02:51:28.868670051 (209h39m55.79496473s) [upload=167956]: Starting upload
Jul 2 02:51:28.868780851 (209h39m55.795075543s) [upload=167956]: Done starting upload
Jul 2 02:51:28.868884538 (209h39m55.795179225s) [upload=167956]: Uploading segments...
Jul 2 02:51:28.869381167 (209h39m55.795675855s) [upload=167956]: Got next segment. Inline: false
Jul 2 02:51:28.869486445 (209h39m55.795781135s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 2 02:51:28.869493196 (209h39m55.795787888s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 2 02:51:28.898707948 (209h39m55.825002664s) [seg_pos={0 0}, upload=167956]: Begin upload segment...
Jul 2 02:51:28.898787644 (209h39m55.825082342s) [seg_pos={0 0}, upload=167956]: Done begin upload segment.
Jul 2 02:51:28.898825319 (209h39m55.825120018s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.898864369 (209h39m55.825159066s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899423277 (209h39m55.825717966s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899547497 (209h39m55.825842195s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899793450 (209h39m55.82608814s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900152858 (209h39m55.826447544s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900210034 (209h39m55.826504727s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900634257 (209h39m55.826928949s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900678544 (209h39m55.826973233s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900990193 (209h39m55.827284883s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901196315 (209h39m55.827491002s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901407463 (209h39m55.827702159s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901732148 (209h39m55.828026843s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901817419 (209h39m55.828112106s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902270961 (209h39m55.828565648s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902808316 (209h39m55.829103005s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:29.002833067 (209h39m55.929127747s) [upload=167956]: Next returned nil segment
Jul 2 02:51:29.002849035 (209h39m55.929143729s) [upload=167956]: Waiting for error group managing segments...
Jul 2 02:51:29.009136933 (209h39m55.935431621s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.009959595 (209h39m55.936254298s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.010805328 (209h39m55.937100001s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.011547112 (209h39m55.9378418s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012145259 (209h39m55.938439932s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012854902 (209h39m55.939149581s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013365184 (209h39m55.939659859s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013906104 (209h39m55.940200783s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.014907872 (209h39m55.941202559s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015448646 (209h39m55.941743332s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015581494 (209h39m55.941876167s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015893548 (209h39m55.942188225s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017028456 (209h39m55.943323135s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017043139 (209h39m55.943337815s) [seg_pos={0 0}, upload=167956]: Segment reached optimal threshold of 13 pieces.
Jul 2 02:51:29.017178229 (209h39m55.943472905s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017310273 (209h39m55.94360495s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017394571 (209h39m55.943689253s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017409321 (209h39m55.943704012s) [seg_pos={0 0}, upload=167956]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 2 02:51:29.017486214 (209h39m55.943780908s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:51:29.017491827 (209h39m55.943786629s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:53:50.138713219 (209h42m17.065007921s) [upload=167956]: Done uploading segments...
Jul 2 02:53:50.138739023 (209h42m17.065033711s) [upload=167956]: Upload finished. err: metaclient: context deadline exceeded
Duration: 593.796344ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [............X...............................................] duration:
463.836667ms
FAIL (2025-07-02 03:27:25 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
ece_id=4HWNWPU27EN6NDCHLIDTA7TR6SEDEGEEH374FY42E4S2YB64JIQA, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650643965 (209h38m42.57693864s) [noise=true , address=104.36.82.6:4873 , piece_id=U5UUFIN6D3CA65BQNOBGH2PXRXXUPJIHM4ICHMT2HGTNVYMYCQDQ, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650681786 (209h38m42.576976485s) [seg_pos={0 0}, upload=167939]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jul 2 02:50:15.650762048 (209h38m42.577056758s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:50:15.650767470 (209h38m42.577062168s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:50:15.720813246 (209h38m42.647107948s) [upload=167939]: Done uploading segments...
Jul 2 02:50:15.720854178 (209h38m42.647148871s) [upload=167939]: Upload finished. err: <nil>
Jul 2 02:51:28.868670051 (209h39m55.79496473s) [upload=167956]: Starting upload
Jul 2 02:51:28.868780851 (209h39m55.795075543s) [upload=167956]: Done starting upload
Jul 2 02:51:28.868884538 (209h39m55.795179225s) [upload=167956]: Uploading segments...
Jul 2 02:51:28.869381167 (209h39m55.795675855s) [upload=167956]: Got next segment. Inline: false
Jul 2 02:51:28.869486445 (209h39m55.795781135s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 2 02:51:28.869493196 (209h39m55.795787888s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 2 02:51:28.898707948 (209h39m55.825002664s) [seg_pos={0 0}, upload=167956]: Begin upload segment...
Jul 2 02:51:28.898787644 (209h39m55.825082342s) [seg_pos={0 0}, upload=167956]: Done begin upload segment.
Jul 2 02:51:28.898825319 (209h39m55.825120018s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.898864369 (209h39m55.825159066s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899423277 (209h39m55.825717966s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899547497 (209h39m55.825842195s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899793450 (209h39m55.82608814s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900152858 (209h39m55.826447544s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900210034 (209h39m55.826504727s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900634257 (209h39m55.826928949s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900678544 (209h39m55.826973233s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900990193 (209h39m55.827284883s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901196315 (209h39m55.827491002s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901407463 (209h39m55.827702159s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901732148 (209h39m55.828026843s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901817419 (209h39m55.828112106s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902270961 (209h39m55.828565648s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902808316 (209h39m55.829103005s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:29.002833067 (209h39m55.929127747s) [upload=167956]: Next returned nil segment
Jul 2 02:51:29.002849035 (209h39m55.929143729s) [upload=167956]: Waiting for error group managing segments...
Jul 2 02:51:29.009136933 (209h39m55.935431621s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.009959595 (209h39m55.936254298s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.010805328 (209h39m55.937100001s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.011547112 (209h39m55.9378418s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012145259 (209h39m55.938439932s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012854902 (209h39m55.939149581s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013365184 (209h39m55.939659859s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013906104 (209h39m55.940200783s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.014907872 (209h39m55.941202559s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015448646 (209h39m55.941743332s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015581494 (209h39m55.941876167s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015893548 (209h39m55.942188225s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017028456 (209h39m55.943323135s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017043139 (209h39m55.943337815s) [seg_pos={0 0}, upload=167956]: Segment reached optimal threshold of 13 pieces.
Jul 2 02:51:29.017178229 (209h39m55.943472905s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017310273 (209h39m55.94360495s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017394571 (209h39m55.943689253s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017409321 (209h39m55.943704012s) [seg_pos={0 0}, upload=167956]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 2 02:51:29.017486214 (209h39m55.943780908s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:51:29.017491827 (209h39m55.943786629s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:53:50.138713219 (209h42m17.065007921s) [upload=167956]: Done uploading segments...
Jul 2 02:53:50.138739023 (209h42m17.065033711s) [upload=167956]: Upload finished. err: metaclient: context deadline exceeded
Duration: 463.860576ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............X..............................................] duration:
453.617382ms
FAIL (2025-07-02 03:26:11 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
ece_id=4HWNWPU27EN6NDCHLIDTA7TR6SEDEGEEH374FY42E4S2YB64JIQA, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650643965 (209h38m42.57693864s) [noise=true , address=104.36.82.6:4873 , piece_id=U5UUFIN6D3CA65BQNOBGH2PXRXXUPJIHM4ICHMT2HGTNVYMYCQDQ, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650681786 (209h38m42.576976485s) [seg_pos={0 0}, upload=167939]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jul 2 02:50:15.650762048 (209h38m42.577056758s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:50:15.650767470 (209h38m42.577062168s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:50:15.720813246 (209h38m42.647107948s) [upload=167939]: Done uploading segments...
Jul 2 02:50:15.720854178 (209h38m42.647148871s) [upload=167939]: Upload finished. err: <nil>
Jul 2 02:51:28.868670051 (209h39m55.79496473s) [upload=167956]: Starting upload
Jul 2 02:51:28.868780851 (209h39m55.795075543s) [upload=167956]: Done starting upload
Jul 2 02:51:28.868884538 (209h39m55.795179225s) [upload=167956]: Uploading segments...
Jul 2 02:51:28.869381167 (209h39m55.795675855s) [upload=167956]: Got next segment. Inline: false
Jul 2 02:51:28.869486445 (209h39m55.795781135s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 2 02:51:28.869493196 (209h39m55.795787888s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 2 02:51:28.898707948 (209h39m55.825002664s) [seg_pos={0 0}, upload=167956]: Begin upload segment...
Jul 2 02:51:28.898787644 (209h39m55.825082342s) [seg_pos={0 0}, upload=167956]: Done begin upload segment.
Jul 2 02:51:28.898825319 (209h39m55.825120018s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.898864369 (209h39m55.825159066s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899423277 (209h39m55.825717966s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899547497 (209h39m55.825842195s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899793450 (209h39m55.82608814s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900152858 (209h39m55.826447544s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900210034 (209h39m55.826504727s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900634257 (209h39m55.826928949s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900678544 (209h39m55.826973233s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900990193 (209h39m55.827284883s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901196315 (209h39m55.827491002s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901407463 (209h39m55.827702159s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901732148 (209h39m55.828026843s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901817419 (209h39m55.828112106s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902270961 (209h39m55.828565648s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902808316 (209h39m55.829103005s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:29.002833067 (209h39m55.929127747s) [upload=167956]: Next returned nil segment
Jul 2 02:51:29.002849035 (209h39m55.929143729s) [upload=167956]: Waiting for error group managing segments...
Jul 2 02:51:29.009136933 (209h39m55.935431621s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.009959595 (209h39m55.936254298s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.010805328 (209h39m55.937100001s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.011547112 (209h39m55.9378418s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012145259 (209h39m55.938439932s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012854902 (209h39m55.939149581s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013365184 (209h39m55.939659859s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013906104 (209h39m55.940200783s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.014907872 (209h39m55.941202559s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015448646 (209h39m55.941743332s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015581494 (209h39m55.941876167s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015893548 (209h39m55.942188225s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017028456 (209h39m55.943323135s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017043139 (209h39m55.943337815s) [seg_pos={0 0}, upload=167956]: Segment reached optimal threshold of 13 pieces.
Jul 2 02:51:29.017178229 (209h39m55.943472905s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017310273 (209h39m55.94360495s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017394571 (209h39m55.943689253s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017409321 (209h39m55.943704012s) [seg_pos={0 0}, upload=167956]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 2 02:51:29.017486214 (209h39m55.943780908s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:51:29.017491827 (209h39m55.943786629s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:53:50.138713219 (209h42m17.065007921s) [upload=167956]: Done uploading segments...
Jul 2 02:53:50.138739023 (209h42m17.065033711s) [upload=167956]: Upload finished. err: metaclient: context deadline exceeded
Duration: 453.6445ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............X.............................................] duration:
550.690233ms
FAIL (2025-07-02 03:25:12 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
ece_id=4HWNWPU27EN6NDCHLIDTA7TR6SEDEGEEH374FY42E4S2YB64JIQA, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650643965 (209h38m42.57693864s) [noise=true , address=104.36.82.6:4873 , piece_id=U5UUFIN6D3CA65BQNOBGH2PXRXXUPJIHM4ICHMT2HGTNVYMYCQDQ, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650681786 (209h38m42.576976485s) [seg_pos={0 0}, upload=167939]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jul 2 02:50:15.650762048 (209h38m42.577056758s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:50:15.650767470 (209h38m42.577062168s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:50:15.720813246 (209h38m42.647107948s) [upload=167939]: Done uploading segments...
Jul 2 02:50:15.720854178 (209h38m42.647148871s) [upload=167939]: Upload finished. err: <nil>
Jul 2 02:51:28.868670051 (209h39m55.79496473s) [upload=167956]: Starting upload
Jul 2 02:51:28.868780851 (209h39m55.795075543s) [upload=167956]: Done starting upload
Jul 2 02:51:28.868884538 (209h39m55.795179225s) [upload=167956]: Uploading segments...
Jul 2 02:51:28.869381167 (209h39m55.795675855s) [upload=167956]: Got next segment. Inline: false
Jul 2 02:51:28.869486445 (209h39m55.795781135s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 2 02:51:28.869493196 (209h39m55.795787888s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 2 02:51:28.898707948 (209h39m55.825002664s) [seg_pos={0 0}, upload=167956]: Begin upload segment...
Jul 2 02:51:28.898787644 (209h39m55.825082342s) [seg_pos={0 0}, upload=167956]: Done begin upload segment.
Jul 2 02:51:28.898825319 (209h39m55.825120018s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.898864369 (209h39m55.825159066s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899423277 (209h39m55.825717966s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899547497 (209h39m55.825842195s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899793450 (209h39m55.82608814s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900152858 (209h39m55.826447544s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900210034 (209h39m55.826504727s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900634257 (209h39m55.826928949s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900678544 (209h39m55.826973233s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900990193 (209h39m55.827284883s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901196315 (209h39m55.827491002s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901407463 (209h39m55.827702159s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901732148 (209h39m55.828026843s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901817419 (209h39m55.828112106s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902270961 (209h39m55.828565648s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902808316 (209h39m55.829103005s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:29.002833067 (209h39m55.929127747s) [upload=167956]: Next returned nil segment
Jul 2 02:51:29.002849035 (209h39m55.929143729s) [upload=167956]: Waiting for error group managing segments...
Jul 2 02:51:29.009136933 (209h39m55.935431621s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.009959595 (209h39m55.936254298s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.010805328 (209h39m55.937100001s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.011547112 (209h39m55.9378418s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012145259 (209h39m55.938439932s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012854902 (209h39m55.939149581s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013365184 (209h39m55.939659859s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013906104 (209h39m55.940200783s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.014907872 (209h39m55.941202559s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015448646 (209h39m55.941743332s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015581494 (209h39m55.941876167s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015893548 (209h39m55.942188225s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017028456 (209h39m55.943323135s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017043139 (209h39m55.943337815s) [seg_pos={0 0}, upload=167956]: Segment reached optimal threshold of 13 pieces.
Jul 2 02:51:29.017178229 (209h39m55.943472905s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017310273 (209h39m55.94360495s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017394571 (209h39m55.943689253s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017409321 (209h39m55.943704012s) [seg_pos={0 0}, upload=167956]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 2 02:51:29.017486214 (209h39m55.943780908s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:51:29.017491827 (209h39m55.943786629s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:53:50.138713219 (209h42m17.065007921s) [upload=167956]: Done uploading segments...
Jul 2 02:53:50.138739023 (209h42m17.065033711s) [upload=167956]: Upload finished. err: metaclient: context deadline exceeded
Duration: 550.726897ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............X............................................] duration:
452.384869ms
FAIL (2025-07-02 03:24:07 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
ece_id=4HWNWPU27EN6NDCHLIDTA7TR6SEDEGEEH374FY42E4S2YB64JIQA, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650643965 (209h38m42.57693864s) [noise=true , address=104.36.82.6:4873 , piece_id=U5UUFIN6D3CA65BQNOBGH2PXRXXUPJIHM4ICHMT2HGTNVYMYCQDQ, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650681786 (209h38m42.576976485s) [seg_pos={0 0}, upload=167939]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jul 2 02:50:15.650762048 (209h38m42.577056758s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:50:15.650767470 (209h38m42.577062168s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:50:15.720813246 (209h38m42.647107948s) [upload=167939]: Done uploading segments...
Jul 2 02:50:15.720854178 (209h38m42.647148871s) [upload=167939]: Upload finished. err: <nil>
Jul 2 02:51:28.868670051 (209h39m55.79496473s) [upload=167956]: Starting upload
Jul 2 02:51:28.868780851 (209h39m55.795075543s) [upload=167956]: Done starting upload
Jul 2 02:51:28.868884538 (209h39m55.795179225s) [upload=167956]: Uploading segments...
Jul 2 02:51:28.869381167 (209h39m55.795675855s) [upload=167956]: Got next segment. Inline: false
Jul 2 02:51:28.869486445 (209h39m55.795781135s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 2 02:51:28.869493196 (209h39m55.795787888s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 2 02:51:28.898707948 (209h39m55.825002664s) [seg_pos={0 0}, upload=167956]: Begin upload segment...
Jul 2 02:51:28.898787644 (209h39m55.825082342s) [seg_pos={0 0}, upload=167956]: Done begin upload segment.
Jul 2 02:51:28.898825319 (209h39m55.825120018s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.898864369 (209h39m55.825159066s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899423277 (209h39m55.825717966s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899547497 (209h39m55.825842195s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899793450 (209h39m55.82608814s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900152858 (209h39m55.826447544s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900210034 (209h39m55.826504727s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900634257 (209h39m55.826928949s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900678544 (209h39m55.826973233s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900990193 (209h39m55.827284883s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901196315 (209h39m55.827491002s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901407463 (209h39m55.827702159s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901732148 (209h39m55.828026843s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901817419 (209h39m55.828112106s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902270961 (209h39m55.828565648s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902808316 (209h39m55.829103005s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:29.002833067 (209h39m55.929127747s) [upload=167956]: Next returned nil segment
Jul 2 02:51:29.002849035 (209h39m55.929143729s) [upload=167956]: Waiting for error group managing segments...
Jul 2 02:51:29.009136933 (209h39m55.935431621s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.009959595 (209h39m55.936254298s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.010805328 (209h39m55.937100001s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.011547112 (209h39m55.9378418s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012145259 (209h39m55.938439932s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012854902 (209h39m55.939149581s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013365184 (209h39m55.939659859s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013906104 (209h39m55.940200783s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.014907872 (209h39m55.941202559s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015448646 (209h39m55.941743332s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015581494 (209h39m55.941876167s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015893548 (209h39m55.942188225s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017028456 (209h39m55.943323135s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017043139 (209h39m55.943337815s) [seg_pos={0 0}, upload=167956]: Segment reached optimal threshold of 13 pieces.
Jul 2 02:51:29.017178229 (209h39m55.943472905s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017310273 (209h39m55.94360495s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017394571 (209h39m55.943689253s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017409321 (209h39m55.943704012s) [seg_pos={0 0}, upload=167956]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 2 02:51:29.017486214 (209h39m55.943780908s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:51:29.017491827 (209h39m55.943786629s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:53:50.138713219 (209h42m17.065007921s) [upload=167956]: Done uploading segments...
Jul 2 02:53:50.138739023 (209h42m17.065033711s) [upload=167956]: Upload finished. err: metaclient: context deadline exceeded
Duration: 452.4228ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [................X...........................................] duration:
537.196738ms
FAIL (2025-07-02 03:22:54 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
ece_id=4HWNWPU27EN6NDCHLIDTA7TR6SEDEGEEH374FY42E4S2YB64JIQA, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650643965 (209h38m42.57693864s) [noise=true , address=104.36.82.6:4873 , piece_id=U5UUFIN6D3CA65BQNOBGH2PXRXXUPJIHM4ICHMT2HGTNVYMYCQDQ, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650681786 (209h38m42.576976485s) [seg_pos={0 0}, upload=167939]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jul 2 02:50:15.650762048 (209h38m42.577056758s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:50:15.650767470 (209h38m42.577062168s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:50:15.720813246 (209h38m42.647107948s) [upload=167939]: Done uploading segments...
Jul 2 02:50:15.720854178 (209h38m42.647148871s) [upload=167939]: Upload finished. err: <nil>
Jul 2 02:51:28.868670051 (209h39m55.79496473s) [upload=167956]: Starting upload
Jul 2 02:51:28.868780851 (209h39m55.795075543s) [upload=167956]: Done starting upload
Jul 2 02:51:28.868884538 (209h39m55.795179225s) [upload=167956]: Uploading segments...
Jul 2 02:51:28.869381167 (209h39m55.795675855s) [upload=167956]: Got next segment. Inline: false
Jul 2 02:51:28.869486445 (209h39m55.795781135s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 2 02:51:28.869493196 (209h39m55.795787888s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 2 02:51:28.898707948 (209h39m55.825002664s) [seg_pos={0 0}, upload=167956]: Begin upload segment...
Jul 2 02:51:28.898787644 (209h39m55.825082342s) [seg_pos={0 0}, upload=167956]: Done begin upload segment.
Jul 2 02:51:28.898825319 (209h39m55.825120018s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.898864369 (209h39m55.825159066s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899423277 (209h39m55.825717966s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899547497 (209h39m55.825842195s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899793450 (209h39m55.82608814s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900152858 (209h39m55.826447544s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900210034 (209h39m55.826504727s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900634257 (209h39m55.826928949s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900678544 (209h39m55.826973233s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900990193 (209h39m55.827284883s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901196315 (209h39m55.827491002s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901407463 (209h39m55.827702159s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901732148 (209h39m55.828026843s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901817419 (209h39m55.828112106s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902270961 (209h39m55.828565648s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902808316 (209h39m55.829103005s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:29.002833067 (209h39m55.929127747s) [upload=167956]: Next returned nil segment
Jul 2 02:51:29.002849035 (209h39m55.929143729s) [upload=167956]: Waiting for error group managing segments...
Jul 2 02:51:29.009136933 (209h39m55.935431621s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.009959595 (209h39m55.936254298s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.010805328 (209h39m55.937100001s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.011547112 (209h39m55.9378418s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012145259 (209h39m55.938439932s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012854902 (209h39m55.939149581s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013365184 (209h39m55.939659859s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013906104 (209h39m55.940200783s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.014907872 (209h39m55.941202559s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015448646 (209h39m55.941743332s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015581494 (209h39m55.941876167s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015893548 (209h39m55.942188225s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017028456 (209h39m55.943323135s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017043139 (209h39m55.943337815s) [seg_pos={0 0}, upload=167956]: Segment reached optimal threshold of 13 pieces.
Jul 2 02:51:29.017178229 (209h39m55.943472905s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017310273 (209h39m55.94360495s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017394571 (209h39m55.943689253s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017409321 (209h39m55.943704012s) [seg_pos={0 0}, upload=167956]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 2 02:51:29.017486214 (209h39m55.943780908s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:51:29.017491827 (209h39m55.943786629s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:53:50.138713219 (209h42m17.065007921s) [upload=167956]: Done uploading segments...
Jul 2 02:53:50.138739023 (209h42m17.065033711s) [upload=167956]: Upload finished. err: metaclient: context deadline exceeded
Duration: 537.227199ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................X..........................................] duration:
549.810105ms
FAIL (2025-07-02 03:22:18 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
ece_id=4HWNWPU27EN6NDCHLIDTA7TR6SEDEGEEH374FY42E4S2YB64JIQA, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650643965 (209h38m42.57693864s) [noise=true , address=104.36.82.6:4873 , piece_id=U5UUFIN6D3CA65BQNOBGH2PXRXXUPJIHM4ICHMT2HGTNVYMYCQDQ, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650681786 (209h38m42.576976485s) [seg_pos={0 0}, upload=167939]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jul 2 02:50:15.650762048 (209h38m42.577056758s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:50:15.650767470 (209h38m42.577062168s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:50:15.720813246 (209h38m42.647107948s) [upload=167939]: Done uploading segments...
Jul 2 02:50:15.720854178 (209h38m42.647148871s) [upload=167939]: Upload finished. err: <nil>
Jul 2 02:51:28.868670051 (209h39m55.79496473s) [upload=167956]: Starting upload
Jul 2 02:51:28.868780851 (209h39m55.795075543s) [upload=167956]: Done starting upload
Jul 2 02:51:28.868884538 (209h39m55.795179225s) [upload=167956]: Uploading segments...
Jul 2 02:51:28.869381167 (209h39m55.795675855s) [upload=167956]: Got next segment. Inline: false
Jul 2 02:51:28.869486445 (209h39m55.795781135s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 2 02:51:28.869493196 (209h39m55.795787888s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 2 02:51:28.898707948 (209h39m55.825002664s) [seg_pos={0 0}, upload=167956]: Begin upload segment...
Jul 2 02:51:28.898787644 (209h39m55.825082342s) [seg_pos={0 0}, upload=167956]: Done begin upload segment.
Jul 2 02:51:28.898825319 (209h39m55.825120018s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.898864369 (209h39m55.825159066s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899423277 (209h39m55.825717966s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899547497 (209h39m55.825842195s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899793450 (209h39m55.82608814s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900152858 (209h39m55.826447544s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900210034 (209h39m55.826504727s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900634257 (209h39m55.826928949s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900678544 (209h39m55.826973233s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900990193 (209h39m55.827284883s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901196315 (209h39m55.827491002s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901407463 (209h39m55.827702159s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901732148 (209h39m55.828026843s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901817419 (209h39m55.828112106s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902270961 (209h39m55.828565648s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902808316 (209h39m55.829103005s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:29.002833067 (209h39m55.929127747s) [upload=167956]: Next returned nil segment
Jul 2 02:51:29.002849035 (209h39m55.929143729s) [upload=167956]: Waiting for error group managing segments...
Jul 2 02:51:29.009136933 (209h39m55.935431621s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.009959595 (209h39m55.936254298s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.010805328 (209h39m55.937100001s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.011547112 (209h39m55.9378418s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012145259 (209h39m55.938439932s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012854902 (209h39m55.939149581s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013365184 (209h39m55.939659859s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013906104 (209h39m55.940200783s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.014907872 (209h39m55.941202559s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015448646 (209h39m55.941743332s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015581494 (209h39m55.941876167s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015893548 (209h39m55.942188225s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017028456 (209h39m55.943323135s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017043139 (209h39m55.943337815s) [seg_pos={0 0}, upload=167956]: Segment reached optimal threshold of 13 pieces.
Jul 2 02:51:29.017178229 (209h39m55.943472905s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017310273 (209h39m55.94360495s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017394571 (209h39m55.943689253s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017409321 (209h39m55.943704012s) [seg_pos={0 0}, upload=167956]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 2 02:51:29.017486214 (209h39m55.943780908s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:51:29.017491827 (209h39m55.943786629s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:53:50.138713219 (209h42m17.065007921s) [upload=167956]: Done uploading segments...
Jul 2 02:53:50.138739023 (209h42m17.065033711s) [upload=167956]: Upload finished. err: metaclient: context deadline exceeded
Duration: 549.838928ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................X.........................................] duration:
443.562786ms
FAIL (2025-07-02 03:21:13 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
ece_id=4HWNWPU27EN6NDCHLIDTA7TR6SEDEGEEH374FY42E4S2YB64JIQA, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650643965 (209h38m42.57693864s) [noise=true , address=104.36.82.6:4873 , piece_id=U5UUFIN6D3CA65BQNOBGH2PXRXXUPJIHM4ICHMT2HGTNVYMYCQDQ, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650681786 (209h38m42.576976485s) [seg_pos={0 0}, upload=167939]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jul 2 02:50:15.650762048 (209h38m42.577056758s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:50:15.650767470 (209h38m42.577062168s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:50:15.720813246 (209h38m42.647107948s) [upload=167939]: Done uploading segments...
Jul 2 02:50:15.720854178 (209h38m42.647148871s) [upload=167939]: Upload finished. err: <nil>
Jul 2 02:51:28.868670051 (209h39m55.79496473s) [upload=167956]: Starting upload
Jul 2 02:51:28.868780851 (209h39m55.795075543s) [upload=167956]: Done starting upload
Jul 2 02:51:28.868884538 (209h39m55.795179225s) [upload=167956]: Uploading segments...
Jul 2 02:51:28.869381167 (209h39m55.795675855s) [upload=167956]: Got next segment. Inline: false
Jul 2 02:51:28.869486445 (209h39m55.795781135s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 2 02:51:28.869493196 (209h39m55.795787888s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 2 02:51:28.898707948 (209h39m55.825002664s) [seg_pos={0 0}, upload=167956]: Begin upload segment...
Jul 2 02:51:28.898787644 (209h39m55.825082342s) [seg_pos={0 0}, upload=167956]: Done begin upload segment.
Jul 2 02:51:28.898825319 (209h39m55.825120018s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.898864369 (209h39m55.825159066s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899423277 (209h39m55.825717966s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899547497 (209h39m55.825842195s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899793450 (209h39m55.82608814s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900152858 (209h39m55.826447544s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900210034 (209h39m55.826504727s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900634257 (209h39m55.826928949s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900678544 (209h39m55.826973233s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900990193 (209h39m55.827284883s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901196315 (209h39m55.827491002s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901407463 (209h39m55.827702159s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901732148 (209h39m55.828026843s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901817419 (209h39m55.828112106s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902270961 (209h39m55.828565648s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902808316 (209h39m55.829103005s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:29.002833067 (209h39m55.929127747s) [upload=167956]: Next returned nil segment
Jul 2 02:51:29.002849035 (209h39m55.929143729s) [upload=167956]: Waiting for error group managing segments...
Jul 2 02:51:29.009136933 (209h39m55.935431621s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.009959595 (209h39m55.936254298s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.010805328 (209h39m55.937100001s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.011547112 (209h39m55.9378418s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012145259 (209h39m55.938439932s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012854902 (209h39m55.939149581s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013365184 (209h39m55.939659859s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013906104 (209h39m55.940200783s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.014907872 (209h39m55.941202559s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015448646 (209h39m55.941743332s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015581494 (209h39m55.941876167s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015893548 (209h39m55.942188225s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017028456 (209h39m55.943323135s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017043139 (209h39m55.943337815s) [seg_pos={0 0}, upload=167956]: Segment reached optimal threshold of 13 pieces.
Jul 2 02:51:29.017178229 (209h39m55.943472905s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017310273 (209h39m55.94360495s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017394571 (209h39m55.943689253s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017409321 (209h39m55.943704012s) [seg_pos={0 0}, upload=167956]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 2 02:51:29.017486214 (209h39m55.943780908s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:51:29.017491827 (209h39m55.943786629s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:53:50.138713219 (209h42m17.065007921s) [upload=167956]: Done uploading segments...
Jul 2 02:53:50.138739023 (209h42m17.065033711s) [upload=167956]: Upload finished. err: metaclient: context deadline exceeded
Duration: 443.593845ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................X........................................] duration:
506.892489ms
FAIL (2025-07-02 03:20:56 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
ece_id=4HWNWPU27EN6NDCHLIDTA7TR6SEDEGEEH374FY42E4S2YB64JIQA, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650643965 (209h38m42.57693864s) [noise=true , address=104.36.82.6:4873 , piece_id=U5UUFIN6D3CA65BQNOBGH2PXRXXUPJIHM4ICHMT2HGTNVYMYCQDQ, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650681786 (209h38m42.576976485s) [seg_pos={0 0}, upload=167939]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jul 2 02:50:15.650762048 (209h38m42.577056758s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:50:15.650767470 (209h38m42.577062168s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:50:15.720813246 (209h38m42.647107948s) [upload=167939]: Done uploading segments...
Jul 2 02:50:15.720854178 (209h38m42.647148871s) [upload=167939]: Upload finished. err: <nil>
Jul 2 02:51:28.868670051 (209h39m55.79496473s) [upload=167956]: Starting upload
Jul 2 02:51:28.868780851 (209h39m55.795075543s) [upload=167956]: Done starting upload
Jul 2 02:51:28.868884538 (209h39m55.795179225s) [upload=167956]: Uploading segments...
Jul 2 02:51:28.869381167 (209h39m55.795675855s) [upload=167956]: Got next segment. Inline: false
Jul 2 02:51:28.869486445 (209h39m55.795781135s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 2 02:51:28.869493196 (209h39m55.795787888s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 2 02:51:28.898707948 (209h39m55.825002664s) [seg_pos={0 0}, upload=167956]: Begin upload segment...
Jul 2 02:51:28.898787644 (209h39m55.825082342s) [seg_pos={0 0}, upload=167956]: Done begin upload segment.
Jul 2 02:51:28.898825319 (209h39m55.825120018s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.898864369 (209h39m55.825159066s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899423277 (209h39m55.825717966s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899547497 (209h39m55.825842195s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899793450 (209h39m55.82608814s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900152858 (209h39m55.826447544s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900210034 (209h39m55.826504727s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900634257 (209h39m55.826928949s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900678544 (209h39m55.826973233s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900990193 (209h39m55.827284883s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901196315 (209h39m55.827491002s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901407463 (209h39m55.827702159s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901732148 (209h39m55.828026843s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901817419 (209h39m55.828112106s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902270961 (209h39m55.828565648s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902808316 (209h39m55.829103005s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:29.002833067 (209h39m55.929127747s) [upload=167956]: Next returned nil segment
Jul 2 02:51:29.002849035 (209h39m55.929143729s) [upload=167956]: Waiting for error group managing segments...
Jul 2 02:51:29.009136933 (209h39m55.935431621s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.009959595 (209h39m55.936254298s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.010805328 (209h39m55.937100001s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.011547112 (209h39m55.9378418s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012145259 (209h39m55.938439932s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012854902 (209h39m55.939149581s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013365184 (209h39m55.939659859s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013906104 (209h39m55.940200783s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.014907872 (209h39m55.941202559s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015448646 (209h39m55.941743332s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015581494 (209h39m55.941876167s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015893548 (209h39m55.942188225s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017028456 (209h39m55.943323135s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017043139 (209h39m55.943337815s) [seg_pos={0 0}, upload=167956]: Segment reached optimal threshold of 13 pieces.
Jul 2 02:51:29.017178229 (209h39m55.943472905s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017310273 (209h39m55.94360495s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017394571 (209h39m55.943689253s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017409321 (209h39m55.943704012s) [seg_pos={0 0}, upload=167956]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 2 02:51:29.017486214 (209h39m55.943780908s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:51:29.017491827 (209h39m55.943786629s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:53:50.138713219 (209h42m17.065007921s) [upload=167956]: Done uploading segments...
Jul 2 02:53:50.138739023 (209h42m17.065033711s) [upload=167956]: Upload finished. err: metaclient: context deadline exceeded
Duration: 506.928448ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................X.......................................] duration:
760.778904ms
FAIL (2025-07-02 03:20:02 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
ece_id=4HWNWPU27EN6NDCHLIDTA7TR6SEDEGEEH374FY42E4S2YB64JIQA, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650643965 (209h38m42.57693864s) [noise=true , address=104.36.82.6:4873 , piece_id=U5UUFIN6D3CA65BQNOBGH2PXRXXUPJIHM4ICHMT2HGTNVYMYCQDQ, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650681786 (209h38m42.576976485s) [seg_pos={0 0}, upload=167939]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jul 2 02:50:15.650762048 (209h38m42.577056758s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:50:15.650767470 (209h38m42.577062168s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:50:15.720813246 (209h38m42.647107948s) [upload=167939]: Done uploading segments...
Jul 2 02:50:15.720854178 (209h38m42.647148871s) [upload=167939]: Upload finished. err: <nil>
Jul 2 02:51:28.868670051 (209h39m55.79496473s) [upload=167956]: Starting upload
Jul 2 02:51:28.868780851 (209h39m55.795075543s) [upload=167956]: Done starting upload
Jul 2 02:51:28.868884538 (209h39m55.795179225s) [upload=167956]: Uploading segments...
Jul 2 02:51:28.869381167 (209h39m55.795675855s) [upload=167956]: Got next segment. Inline: false
Jul 2 02:51:28.869486445 (209h39m55.795781135s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 2 02:51:28.869493196 (209h39m55.795787888s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 2 02:51:28.898707948 (209h39m55.825002664s) [seg_pos={0 0}, upload=167956]: Begin upload segment...
Jul 2 02:51:28.898787644 (209h39m55.825082342s) [seg_pos={0 0}, upload=167956]: Done begin upload segment.
Jul 2 02:51:28.898825319 (209h39m55.825120018s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.898864369 (209h39m55.825159066s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899423277 (209h39m55.825717966s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899547497 (209h39m55.825842195s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899793450 (209h39m55.82608814s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900152858 (209h39m55.826447544s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900210034 (209h39m55.826504727s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900634257 (209h39m55.826928949s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900678544 (209h39m55.826973233s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900990193 (209h39m55.827284883s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901196315 (209h39m55.827491002s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901407463 (209h39m55.827702159s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901732148 (209h39m55.828026843s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901817419 (209h39m55.828112106s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902270961 (209h39m55.828565648s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902808316 (209h39m55.829103005s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:29.002833067 (209h39m55.929127747s) [upload=167956]: Next returned nil segment
Jul 2 02:51:29.002849035 (209h39m55.929143729s) [upload=167956]: Waiting for error group managing segments...
Jul 2 02:51:29.009136933 (209h39m55.935431621s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.009959595 (209h39m55.936254298s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.010805328 (209h39m55.937100001s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.011547112 (209h39m55.9378418s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012145259 (209h39m55.938439932s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012854902 (209h39m55.939149581s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013365184 (209h39m55.939659859s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013906104 (209h39m55.940200783s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.014907872 (209h39m55.941202559s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015448646 (209h39m55.941743332s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015581494 (209h39m55.941876167s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015893548 (209h39m55.942188225s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017028456 (209h39m55.943323135s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017043139 (209h39m55.943337815s) [seg_pos={0 0}, upload=167956]: Segment reached optimal threshold of 13 pieces.
Jul 2 02:51:29.017178229 (209h39m55.943472905s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017310273 (209h39m55.94360495s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017394571 (209h39m55.943689253s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017409321 (209h39m55.943704012s) [seg_pos={0 0}, upload=167956]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 2 02:51:29.017486214 (209h39m55.943780908s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:51:29.017491827 (209h39m55.943786629s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:53:50.138713219 (209h42m17.065007921s) [upload=167956]: Done uploading segments...
Jul 2 02:53:50.138739023 (209h42m17.065033711s) [upload=167956]: Upload finished. err: metaclient: context deadline exceeded
Duration: 760.816077ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................X......................................] duration:
722.410266ms
FAIL (2025-07-02 03:19:38 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
ece_id=4HWNWPU27EN6NDCHLIDTA7TR6SEDEGEEH374FY42E4S2YB64JIQA, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650643965 (209h38m42.57693864s) [noise=true , address=104.36.82.6:4873 , piece_id=U5UUFIN6D3CA65BQNOBGH2PXRXXUPJIHM4ICHMT2HGTNVYMYCQDQ, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650681786 (209h38m42.576976485s) [seg_pos={0 0}, upload=167939]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jul 2 02:50:15.650762048 (209h38m42.577056758s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:50:15.650767470 (209h38m42.577062168s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:50:15.720813246 (209h38m42.647107948s) [upload=167939]: Done uploading segments...
Jul 2 02:50:15.720854178 (209h38m42.647148871s) [upload=167939]: Upload finished. err: <nil>
Jul 2 02:51:28.868670051 (209h39m55.79496473s) [upload=167956]: Starting upload
Jul 2 02:51:28.868780851 (209h39m55.795075543s) [upload=167956]: Done starting upload
Jul 2 02:51:28.868884538 (209h39m55.795179225s) [upload=167956]: Uploading segments...
Jul 2 02:51:28.869381167 (209h39m55.795675855s) [upload=167956]: Got next segment. Inline: false
Jul 2 02:51:28.869486445 (209h39m55.795781135s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 2 02:51:28.869493196 (209h39m55.795787888s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 2 02:51:28.898707948 (209h39m55.825002664s) [seg_pos={0 0}, upload=167956]: Begin upload segment...
Jul 2 02:51:28.898787644 (209h39m55.825082342s) [seg_pos={0 0}, upload=167956]: Done begin upload segment.
Jul 2 02:51:28.898825319 (209h39m55.825120018s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.898864369 (209h39m55.825159066s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899423277 (209h39m55.825717966s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899547497 (209h39m55.825842195s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899793450 (209h39m55.82608814s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900152858 (209h39m55.826447544s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900210034 (209h39m55.826504727s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900634257 (209h39m55.826928949s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900678544 (209h39m55.826973233s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900990193 (209h39m55.827284883s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901196315 (209h39m55.827491002s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901407463 (209h39m55.827702159s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901732148 (209h39m55.828026843s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901817419 (209h39m55.828112106s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902270961 (209h39m55.828565648s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902808316 (209h39m55.829103005s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:29.002833067 (209h39m55.929127747s) [upload=167956]: Next returned nil segment
Jul 2 02:51:29.002849035 (209h39m55.929143729s) [upload=167956]: Waiting for error group managing segments...
Jul 2 02:51:29.009136933 (209h39m55.935431621s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.009959595 (209h39m55.936254298s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.010805328 (209h39m55.937100001s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.011547112 (209h39m55.9378418s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012145259 (209h39m55.938439932s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012854902 (209h39m55.939149581s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013365184 (209h39m55.939659859s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013906104 (209h39m55.940200783s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.014907872 (209h39m55.941202559s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015448646 (209h39m55.941743332s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015581494 (209h39m55.941876167s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015893548 (209h39m55.942188225s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017028456 (209h39m55.943323135s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017043139 (209h39m55.943337815s) [seg_pos={0 0}, upload=167956]: Segment reached optimal threshold of 13 pieces.
Jul 2 02:51:29.017178229 (209h39m55.943472905s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017310273 (209h39m55.94360495s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017394571 (209h39m55.943689253s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017409321 (209h39m55.943704012s) [seg_pos={0 0}, upload=167956]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 2 02:51:29.017486214 (209h39m55.943780908s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:51:29.017491827 (209h39m55.943786629s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:53:50.138713219 (209h42m17.065007921s) [upload=167956]: Done uploading segments...
Jul 2 02:53:50.138739023 (209h42m17.065033711s) [upload=167956]: Upload finished. err: metaclient: context deadline exceeded
Duration: 722.449878ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................X.....................................] duration:
449.431131ms
FAIL (2025-07-02 03:18:36 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
ece_id=4HWNWPU27EN6NDCHLIDTA7TR6SEDEGEEH374FY42E4S2YB64JIQA, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650643965 (209h38m42.57693864s) [noise=true , address=104.36.82.6:4873 , piece_id=U5UUFIN6D3CA65BQNOBGH2PXRXXUPJIHM4ICHMT2HGTNVYMYCQDQ, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650681786 (209h38m42.576976485s) [seg_pos={0 0}, upload=167939]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jul 2 02:50:15.650762048 (209h38m42.577056758s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:50:15.650767470 (209h38m42.577062168s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:50:15.720813246 (209h38m42.647107948s) [upload=167939]: Done uploading segments...
Jul 2 02:50:15.720854178 (209h38m42.647148871s) [upload=167939]: Upload finished. err: <nil>
Jul 2 02:51:28.868670051 (209h39m55.79496473s) [upload=167956]: Starting upload
Jul 2 02:51:28.868780851 (209h39m55.795075543s) [upload=167956]: Done starting upload
Jul 2 02:51:28.868884538 (209h39m55.795179225s) [upload=167956]: Uploading segments...
Jul 2 02:51:28.869381167 (209h39m55.795675855s) [upload=167956]: Got next segment. Inline: false
Jul 2 02:51:28.869486445 (209h39m55.795781135s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 2 02:51:28.869493196 (209h39m55.795787888s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 2 02:51:28.898707948 (209h39m55.825002664s) [seg_pos={0 0}, upload=167956]: Begin upload segment...
Jul 2 02:51:28.898787644 (209h39m55.825082342s) [seg_pos={0 0}, upload=167956]: Done begin upload segment.
Jul 2 02:51:28.898825319 (209h39m55.825120018s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.898864369 (209h39m55.825159066s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899423277 (209h39m55.825717966s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899547497 (209h39m55.825842195s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899793450 (209h39m55.82608814s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900152858 (209h39m55.826447544s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900210034 (209h39m55.826504727s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900634257 (209h39m55.826928949s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900678544 (209h39m55.826973233s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900990193 (209h39m55.827284883s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901196315 (209h39m55.827491002s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901407463 (209h39m55.827702159s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901732148 (209h39m55.828026843s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901817419 (209h39m55.828112106s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902270961 (209h39m55.828565648s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902808316 (209h39m55.829103005s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:29.002833067 (209h39m55.929127747s) [upload=167956]: Next returned nil segment
Jul 2 02:51:29.002849035 (209h39m55.929143729s) [upload=167956]: Waiting for error group managing segments...
Jul 2 02:51:29.009136933 (209h39m55.935431621s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.009959595 (209h39m55.936254298s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.010805328 (209h39m55.937100001s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.011547112 (209h39m55.9378418s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012145259 (209h39m55.938439932s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012854902 (209h39m55.939149581s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013365184 (209h39m55.939659859s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013906104 (209h39m55.940200783s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.014907872 (209h39m55.941202559s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015448646 (209h39m55.941743332s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015581494 (209h39m55.941876167s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015893548 (209h39m55.942188225s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017028456 (209h39m55.943323135s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017043139 (209h39m55.943337815s) [seg_pos={0 0}, upload=167956]: Segment reached optimal threshold of 13 pieces.
Jul 2 02:51:29.017178229 (209h39m55.943472905s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017310273 (209h39m55.94360495s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017394571 (209h39m55.943689253s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017409321 (209h39m55.943704012s) [seg_pos={0 0}, upload=167956]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 2 02:51:29.017486214 (209h39m55.943780908s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:51:29.017491827 (209h39m55.943786629s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:53:50.138713219 (209h42m17.065007921s) [upload=167956]: Done uploading segments...
Jul 2 02:53:50.138739023 (209h42m17.065033711s) [upload=167956]: Upload finished. err: metaclient: context deadline exceeded
Duration: 449.459655ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......................X....................................] duration:
425.436072ms
FAIL (2025-07-02 03:18:17 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
ece_id=4HWNWPU27EN6NDCHLIDTA7TR6SEDEGEEH374FY42E4S2YB64JIQA, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650643965 (209h38m42.57693864s) [noise=true , address=104.36.82.6:4873 , piece_id=U5UUFIN6D3CA65BQNOBGH2PXRXXUPJIHM4ICHMT2HGTNVYMYCQDQ, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650681786 (209h38m42.576976485s) [seg_pos={0 0}, upload=167939]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jul 2 02:50:15.650762048 (209h38m42.577056758s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:50:15.650767470 (209h38m42.577062168s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:50:15.720813246 (209h38m42.647107948s) [upload=167939]: Done uploading segments...
Jul 2 02:50:15.720854178 (209h38m42.647148871s) [upload=167939]: Upload finished. err: <nil>
Jul 2 02:51:28.868670051 (209h39m55.79496473s) [upload=167956]: Starting upload
Jul 2 02:51:28.868780851 (209h39m55.795075543s) [upload=167956]: Done starting upload
Jul 2 02:51:28.868884538 (209h39m55.795179225s) [upload=167956]: Uploading segments...
Jul 2 02:51:28.869381167 (209h39m55.795675855s) [upload=167956]: Got next segment. Inline: false
Jul 2 02:51:28.869486445 (209h39m55.795781135s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 2 02:51:28.869493196 (209h39m55.795787888s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 2 02:51:28.898707948 (209h39m55.825002664s) [seg_pos={0 0}, upload=167956]: Begin upload segment...
Jul 2 02:51:28.898787644 (209h39m55.825082342s) [seg_pos={0 0}, upload=167956]: Done begin upload segment.
Jul 2 02:51:28.898825319 (209h39m55.825120018s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.898864369 (209h39m55.825159066s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899423277 (209h39m55.825717966s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899547497 (209h39m55.825842195s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899793450 (209h39m55.82608814s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900152858 (209h39m55.826447544s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900210034 (209h39m55.826504727s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900634257 (209h39m55.826928949s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900678544 (209h39m55.826973233s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900990193 (209h39m55.827284883s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901196315 (209h39m55.827491002s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901407463 (209h39m55.827702159s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901732148 (209h39m55.828026843s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901817419 (209h39m55.828112106s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902270961 (209h39m55.828565648s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902808316 (209h39m55.829103005s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:29.002833067 (209h39m55.929127747s) [upload=167956]: Next returned nil segment
Jul 2 02:51:29.002849035 (209h39m55.929143729s) [upload=167956]: Waiting for error group managing segments...
Jul 2 02:51:29.009136933 (209h39m55.935431621s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.009959595 (209h39m55.936254298s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.010805328 (209h39m55.937100001s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.011547112 (209h39m55.9378418s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012145259 (209h39m55.938439932s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012854902 (209h39m55.939149581s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013365184 (209h39m55.939659859s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013906104 (209h39m55.940200783s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.014907872 (209h39m55.941202559s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015448646 (209h39m55.941743332s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015581494 (209h39m55.941876167s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015893548 (209h39m55.942188225s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017028456 (209h39m55.943323135s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017043139 (209h39m55.943337815s) [seg_pos={0 0}, upload=167956]: Segment reached optimal threshold of 13 pieces.
Jul 2 02:51:29.017178229 (209h39m55.943472905s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017310273 (209h39m55.94360495s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017394571 (209h39m55.943689253s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017409321 (209h39m55.943704012s) [seg_pos={0 0}, upload=167956]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 2 02:51:29.017486214 (209h39m55.943780908s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:51:29.017491827 (209h39m55.943786629s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:53:50.138713219 (209h42m17.065007921s) [upload=167956]: Done uploading segments...
Jul 2 02:53:50.138739023 (209h42m17.065033711s) [upload=167956]: Upload finished. err: metaclient: context deadline exceeded
Duration: 425.497732ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................X...................................] duration:
491.818952ms
FAIL (2025-07-02 03:17:59 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
ece_id=4HWNWPU27EN6NDCHLIDTA7TR6SEDEGEEH374FY42E4S2YB64JIQA, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650643965 (209h38m42.57693864s) [noise=true , address=104.36.82.6:4873 , piece_id=U5UUFIN6D3CA65BQNOBGH2PXRXXUPJIHM4ICHMT2HGTNVYMYCQDQ, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650681786 (209h38m42.576976485s) [seg_pos={0 0}, upload=167939]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jul 2 02:50:15.650762048 (209h38m42.577056758s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:50:15.650767470 (209h38m42.577062168s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:50:15.720813246 (209h38m42.647107948s) [upload=167939]: Done uploading segments...
Jul 2 02:50:15.720854178 (209h38m42.647148871s) [upload=167939]: Upload finished. err: <nil>
Jul 2 02:51:28.868670051 (209h39m55.79496473s) [upload=167956]: Starting upload
Jul 2 02:51:28.868780851 (209h39m55.795075543s) [upload=167956]: Done starting upload
Jul 2 02:51:28.868884538 (209h39m55.795179225s) [upload=167956]: Uploading segments...
Jul 2 02:51:28.869381167 (209h39m55.795675855s) [upload=167956]: Got next segment. Inline: false
Jul 2 02:51:28.869486445 (209h39m55.795781135s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 2 02:51:28.869493196 (209h39m55.795787888s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 2 02:51:28.898707948 (209h39m55.825002664s) [seg_pos={0 0}, upload=167956]: Begin upload segment...
Jul 2 02:51:28.898787644 (209h39m55.825082342s) [seg_pos={0 0}, upload=167956]: Done begin upload segment.
Jul 2 02:51:28.898825319 (209h39m55.825120018s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.898864369 (209h39m55.825159066s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899423277 (209h39m55.825717966s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899547497 (209h39m55.825842195s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899793450 (209h39m55.82608814s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900152858 (209h39m55.826447544s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900210034 (209h39m55.826504727s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900634257 (209h39m55.826928949s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900678544 (209h39m55.826973233s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900990193 (209h39m55.827284883s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901196315 (209h39m55.827491002s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901407463 (209h39m55.827702159s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901732148 (209h39m55.828026843s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901817419 (209h39m55.828112106s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902270961 (209h39m55.828565648s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902808316 (209h39m55.829103005s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:29.002833067 (209h39m55.929127747s) [upload=167956]: Next returned nil segment
Jul 2 02:51:29.002849035 (209h39m55.929143729s) [upload=167956]: Waiting for error group managing segments...
Jul 2 02:51:29.009136933 (209h39m55.935431621s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.009959595 (209h39m55.936254298s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.010805328 (209h39m55.937100001s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.011547112 (209h39m55.9378418s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012145259 (209h39m55.938439932s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012854902 (209h39m55.939149581s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013365184 (209h39m55.939659859s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013906104 (209h39m55.940200783s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.014907872 (209h39m55.941202559s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015448646 (209h39m55.941743332s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015581494 (209h39m55.941876167s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015893548 (209h39m55.942188225s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017028456 (209h39m55.943323135s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017043139 (209h39m55.943337815s) [seg_pos={0 0}, upload=167956]: Segment reached optimal threshold of 13 pieces.
Jul 2 02:51:29.017178229 (209h39m55.943472905s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017310273 (209h39m55.94360495s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017394571 (209h39m55.943689253s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017409321 (209h39m55.943704012s) [seg_pos={0 0}, upload=167956]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 2 02:51:29.017486214 (209h39m55.943780908s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:51:29.017491827 (209h39m55.943786629s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:53:50.138713219 (209h42m17.065007921s) [upload=167956]: Done uploading segments...
Jul 2 02:53:50.138739023 (209h42m17.065033711s) [upload=167956]: Upload finished. err: metaclient: context deadline exceeded
Duration: 491.937392ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................X..................................] duration:
437.770335ms
FAIL (2025-07-02 03:16:52 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
ece_id=4HWNWPU27EN6NDCHLIDTA7TR6SEDEGEEH374FY42E4S2YB64JIQA, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650643965 (209h38m42.57693864s) [noise=true , address=104.36.82.6:4873 , piece_id=U5UUFIN6D3CA65BQNOBGH2PXRXXUPJIHM4ICHMT2HGTNVYMYCQDQ, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650681786 (209h38m42.576976485s) [seg_pos={0 0}, upload=167939]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jul 2 02:50:15.650762048 (209h38m42.577056758s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:50:15.650767470 (209h38m42.577062168s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:50:15.720813246 (209h38m42.647107948s) [upload=167939]: Done uploading segments...
Jul 2 02:50:15.720854178 (209h38m42.647148871s) [upload=167939]: Upload finished. err: <nil>
Jul 2 02:51:28.868670051 (209h39m55.79496473s) [upload=167956]: Starting upload
Jul 2 02:51:28.868780851 (209h39m55.795075543s) [upload=167956]: Done starting upload
Jul 2 02:51:28.868884538 (209h39m55.795179225s) [upload=167956]: Uploading segments...
Jul 2 02:51:28.869381167 (209h39m55.795675855s) [upload=167956]: Got next segment. Inline: false
Jul 2 02:51:28.869486445 (209h39m55.795781135s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 2 02:51:28.869493196 (209h39m55.795787888s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 2 02:51:28.898707948 (209h39m55.825002664s) [seg_pos={0 0}, upload=167956]: Begin upload segment...
Jul 2 02:51:28.898787644 (209h39m55.825082342s) [seg_pos={0 0}, upload=167956]: Done begin upload segment.
Jul 2 02:51:28.898825319 (209h39m55.825120018s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.898864369 (209h39m55.825159066s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899423277 (209h39m55.825717966s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899547497 (209h39m55.825842195s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899793450 (209h39m55.82608814s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900152858 (209h39m55.826447544s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900210034 (209h39m55.826504727s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900634257 (209h39m55.826928949s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900678544 (209h39m55.826973233s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900990193 (209h39m55.827284883s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901196315 (209h39m55.827491002s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901407463 (209h39m55.827702159s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901732148 (209h39m55.828026843s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901817419 (209h39m55.828112106s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902270961 (209h39m55.828565648s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902808316 (209h39m55.829103005s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:29.002833067 (209h39m55.929127747s) [upload=167956]: Next returned nil segment
Jul 2 02:51:29.002849035 (209h39m55.929143729s) [upload=167956]: Waiting for error group managing segments...
Jul 2 02:51:29.009136933 (209h39m55.935431621s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.009959595 (209h39m55.936254298s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.010805328 (209h39m55.937100001s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.011547112 (209h39m55.9378418s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012145259 (209h39m55.938439932s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012854902 (209h39m55.939149581s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013365184 (209h39m55.939659859s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013906104 (209h39m55.940200783s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.014907872 (209h39m55.941202559s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015448646 (209h39m55.941743332s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015581494 (209h39m55.941876167s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015893548 (209h39m55.942188225s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017028456 (209h39m55.943323135s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017043139 (209h39m55.943337815s) [seg_pos={0 0}, upload=167956]: Segment reached optimal threshold of 13 pieces.
Jul 2 02:51:29.017178229 (209h39m55.943472905s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017310273 (209h39m55.94360495s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017394571 (209h39m55.943689253s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017409321 (209h39m55.943704012s) [seg_pos={0 0}, upload=167956]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 2 02:51:29.017486214 (209h39m55.943780908s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:51:29.017491827 (209h39m55.943786629s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:53:50.138713219 (209h42m17.065007921s) [upload=167956]: Done uploading segments...
Jul 2 02:53:50.138739023 (209h42m17.065033711s) [upload=167956]: Upload finished. err: metaclient: context deadline exceeded
Duration: 437.798275ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................X.................................] duration:
633.851977ms
FAIL (2025-07-02 03:15:45 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
ece_id=4HWNWPU27EN6NDCHLIDTA7TR6SEDEGEEH374FY42E4S2YB64JIQA, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650643965 (209h38m42.57693864s) [noise=true , address=104.36.82.6:4873 , piece_id=U5UUFIN6D3CA65BQNOBGH2PXRXXUPJIHM4ICHMT2HGTNVYMYCQDQ, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650681786 (209h38m42.576976485s) [seg_pos={0 0}, upload=167939]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jul 2 02:50:15.650762048 (209h38m42.577056758s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:50:15.650767470 (209h38m42.577062168s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:50:15.720813246 (209h38m42.647107948s) [upload=167939]: Done uploading segments...
Jul 2 02:50:15.720854178 (209h38m42.647148871s) [upload=167939]: Upload finished. err: <nil>
Jul 2 02:51:28.868670051 (209h39m55.79496473s) [upload=167956]: Starting upload
Jul 2 02:51:28.868780851 (209h39m55.795075543s) [upload=167956]: Done starting upload
Jul 2 02:51:28.868884538 (209h39m55.795179225s) [upload=167956]: Uploading segments...
Jul 2 02:51:28.869381167 (209h39m55.795675855s) [upload=167956]: Got next segment. Inline: false
Jul 2 02:51:28.869486445 (209h39m55.795781135s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 2 02:51:28.869493196 (209h39m55.795787888s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 2 02:51:28.898707948 (209h39m55.825002664s) [seg_pos={0 0}, upload=167956]: Begin upload segment...
Jul 2 02:51:28.898787644 (209h39m55.825082342s) [seg_pos={0 0}, upload=167956]: Done begin upload segment.
Jul 2 02:51:28.898825319 (209h39m55.825120018s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.898864369 (209h39m55.825159066s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899423277 (209h39m55.825717966s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899547497 (209h39m55.825842195s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899793450 (209h39m55.82608814s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900152858 (209h39m55.826447544s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900210034 (209h39m55.826504727s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900634257 (209h39m55.826928949s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900678544 (209h39m55.826973233s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900990193 (209h39m55.827284883s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901196315 (209h39m55.827491002s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901407463 (209h39m55.827702159s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901732148 (209h39m55.828026843s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901817419 (209h39m55.828112106s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902270961 (209h39m55.828565648s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902808316 (209h39m55.829103005s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:29.002833067 (209h39m55.929127747s) [upload=167956]: Next returned nil segment
Jul 2 02:51:29.002849035 (209h39m55.929143729s) [upload=167956]: Waiting for error group managing segments...
Jul 2 02:51:29.009136933 (209h39m55.935431621s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.009959595 (209h39m55.936254298s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.010805328 (209h39m55.937100001s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.011547112 (209h39m55.9378418s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012145259 (209h39m55.938439932s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012854902 (209h39m55.939149581s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013365184 (209h39m55.939659859s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013906104 (209h39m55.940200783s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.014907872 (209h39m55.941202559s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015448646 (209h39m55.941743332s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015581494 (209h39m55.941876167s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015893548 (209h39m55.942188225s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017028456 (209h39m55.943323135s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017043139 (209h39m55.943337815s) [seg_pos={0 0}, upload=167956]: Segment reached optimal threshold of 13 pieces.
Jul 2 02:51:29.017178229 (209h39m55.943472905s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017310273 (209h39m55.94360495s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017394571 (209h39m55.943689253s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017409321 (209h39m55.943704012s) [seg_pos={0 0}, upload=167956]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 2 02:51:29.017486214 (209h39m55.943780908s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:51:29.017491827 (209h39m55.943786629s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:53:50.138713219 (209h42m17.065007921s) [upload=167956]: Done uploading segments...
Jul 2 02:53:50.138739023 (209h42m17.065033711s) [upload=167956]: Upload finished. err: metaclient: context deadline exceeded
Duration: 633.87807ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................X................................] duration:
605.346564ms
FAIL (2025-07-02 03:15:05 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
ece_id=4HWNWPU27EN6NDCHLIDTA7TR6SEDEGEEH374FY42E4S2YB64JIQA, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650643965 (209h38m42.57693864s) [noise=true , address=104.36.82.6:4873 , piece_id=U5UUFIN6D3CA65BQNOBGH2PXRXXUPJIHM4ICHMT2HGTNVYMYCQDQ, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650681786 (209h38m42.576976485s) [seg_pos={0 0}, upload=167939]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jul 2 02:50:15.650762048 (209h38m42.577056758s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:50:15.650767470 (209h38m42.577062168s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:50:15.720813246 (209h38m42.647107948s) [upload=167939]: Done uploading segments...
Jul 2 02:50:15.720854178 (209h38m42.647148871s) [upload=167939]: Upload finished. err: <nil>
Jul 2 02:51:28.868670051 (209h39m55.79496473s) [upload=167956]: Starting upload
Jul 2 02:51:28.868780851 (209h39m55.795075543s) [upload=167956]: Done starting upload
Jul 2 02:51:28.868884538 (209h39m55.795179225s) [upload=167956]: Uploading segments...
Jul 2 02:51:28.869381167 (209h39m55.795675855s) [upload=167956]: Got next segment. Inline: false
Jul 2 02:51:28.869486445 (209h39m55.795781135s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 2 02:51:28.869493196 (209h39m55.795787888s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 2 02:51:28.898707948 (209h39m55.825002664s) [seg_pos={0 0}, upload=167956]: Begin upload segment...
Jul 2 02:51:28.898787644 (209h39m55.825082342s) [seg_pos={0 0}, upload=167956]: Done begin upload segment.
Jul 2 02:51:28.898825319 (209h39m55.825120018s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.898864369 (209h39m55.825159066s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899423277 (209h39m55.825717966s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899547497 (209h39m55.825842195s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899793450 (209h39m55.82608814s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900152858 (209h39m55.826447544s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900210034 (209h39m55.826504727s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900634257 (209h39m55.826928949s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900678544 (209h39m55.826973233s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900990193 (209h39m55.827284883s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901196315 (209h39m55.827491002s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901407463 (209h39m55.827702159s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901732148 (209h39m55.828026843s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901817419 (209h39m55.828112106s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902270961 (209h39m55.828565648s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902808316 (209h39m55.829103005s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:29.002833067 (209h39m55.929127747s) [upload=167956]: Next returned nil segment
Jul 2 02:51:29.002849035 (209h39m55.929143729s) [upload=167956]: Waiting for error group managing segments...
Jul 2 02:51:29.009136933 (209h39m55.935431621s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.009959595 (209h39m55.936254298s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.010805328 (209h39m55.937100001s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.011547112 (209h39m55.9378418s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012145259 (209h39m55.938439932s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012854902 (209h39m55.939149581s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013365184 (209h39m55.939659859s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013906104 (209h39m55.940200783s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.014907872 (209h39m55.941202559s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015448646 (209h39m55.941743332s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015581494 (209h39m55.941876167s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015893548 (209h39m55.942188225s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017028456 (209h39m55.943323135s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017043139 (209h39m55.943337815s) [seg_pos={0 0}, upload=167956]: Segment reached optimal threshold of 13 pieces.
Jul 2 02:51:29.017178229 (209h39m55.943472905s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017310273 (209h39m55.94360495s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017394571 (209h39m55.943689253s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017409321 (209h39m55.943704012s) [seg_pos={0 0}, upload=167956]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 2 02:51:29.017486214 (209h39m55.943780908s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:51:29.017491827 (209h39m55.943786629s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:53:50.138713219 (209h42m17.065007921s) [upload=167956]: Done uploading segments...
Jul 2 02:53:50.138739023 (209h42m17.065033711s) [upload=167956]: Upload finished. err: metaclient: context deadline exceeded
Duration: 605.374636ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [............................X...............................] duration:
507.169735ms
FAIL (2025-07-02 03:13:59 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
ece_id=4HWNWPU27EN6NDCHLIDTA7TR6SEDEGEEH374FY42E4S2YB64JIQA, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650643965 (209h38m42.57693864s) [noise=true , address=104.36.82.6:4873 , piece_id=U5UUFIN6D3CA65BQNOBGH2PXRXXUPJIHM4ICHMT2HGTNVYMYCQDQ, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650681786 (209h38m42.576976485s) [seg_pos={0 0}, upload=167939]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jul 2 02:50:15.650762048 (209h38m42.577056758s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:50:15.650767470 (209h38m42.577062168s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:50:15.720813246 (209h38m42.647107948s) [upload=167939]: Done uploading segments...
Jul 2 02:50:15.720854178 (209h38m42.647148871s) [upload=167939]: Upload finished. err: <nil>
Jul 2 02:51:28.868670051 (209h39m55.79496473s) [upload=167956]: Starting upload
Jul 2 02:51:28.868780851 (209h39m55.795075543s) [upload=167956]: Done starting upload
Jul 2 02:51:28.868884538 (209h39m55.795179225s) [upload=167956]: Uploading segments...
Jul 2 02:51:28.869381167 (209h39m55.795675855s) [upload=167956]: Got next segment. Inline: false
Jul 2 02:51:28.869486445 (209h39m55.795781135s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 2 02:51:28.869493196 (209h39m55.795787888s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 2 02:51:28.898707948 (209h39m55.825002664s) [seg_pos={0 0}, upload=167956]: Begin upload segment...
Jul 2 02:51:28.898787644 (209h39m55.825082342s) [seg_pos={0 0}, upload=167956]: Done begin upload segment.
Jul 2 02:51:28.898825319 (209h39m55.825120018s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.898864369 (209h39m55.825159066s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899423277 (209h39m55.825717966s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899547497 (209h39m55.825842195s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899793450 (209h39m55.82608814s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900152858 (209h39m55.826447544s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900210034 (209h39m55.826504727s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900634257 (209h39m55.826928949s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900678544 (209h39m55.826973233s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900990193 (209h39m55.827284883s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901196315 (209h39m55.827491002s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901407463 (209h39m55.827702159s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901732148 (209h39m55.828026843s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901817419 (209h39m55.828112106s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902270961 (209h39m55.828565648s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902808316 (209h39m55.829103005s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:29.002833067 (209h39m55.929127747s) [upload=167956]: Next returned nil segment
Jul 2 02:51:29.002849035 (209h39m55.929143729s) [upload=167956]: Waiting for error group managing segments...
Jul 2 02:51:29.009136933 (209h39m55.935431621s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.009959595 (209h39m55.936254298s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.010805328 (209h39m55.937100001s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.011547112 (209h39m55.9378418s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012145259 (209h39m55.938439932s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012854902 (209h39m55.939149581s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013365184 (209h39m55.939659859s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013906104 (209h39m55.940200783s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.014907872 (209h39m55.941202559s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015448646 (209h39m55.941743332s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015581494 (209h39m55.941876167s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015893548 (209h39m55.942188225s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017028456 (209h39m55.943323135s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017043139 (209h39m55.943337815s) [seg_pos={0 0}, upload=167956]: Segment reached optimal threshold of 13 pieces.
Jul 2 02:51:29.017178229 (209h39m55.943472905s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017310273 (209h39m55.94360495s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017394571 (209h39m55.943689253s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017409321 (209h39m55.943704012s) [seg_pos={0 0}, upload=167956]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 2 02:51:29.017486214 (209h39m55.943780908s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:51:29.017491827 (209h39m55.943786629s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:53:50.138713219 (209h42m17.065007921s) [upload=167956]: Done uploading segments...
Jul 2 02:53:50.138739023 (209h42m17.065033711s) [upload=167956]: Upload finished. err: metaclient: context deadline exceeded
Duration: 507.200744ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............................X..............................] duration:
504.157036ms
FAIL (2025-07-02 03:13:08 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
ece_id=4HWNWPU27EN6NDCHLIDTA7TR6SEDEGEEH374FY42E4S2YB64JIQA, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650643965 (209h38m42.57693864s) [noise=true , address=104.36.82.6:4873 , piece_id=U5UUFIN6D3CA65BQNOBGH2PXRXXUPJIHM4ICHMT2HGTNVYMYCQDQ, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650681786 (209h38m42.576976485s) [seg_pos={0 0}, upload=167939]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jul 2 02:50:15.650762048 (209h38m42.577056758s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:50:15.650767470 (209h38m42.577062168s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:50:15.720813246 (209h38m42.647107948s) [upload=167939]: Done uploading segments...
Jul 2 02:50:15.720854178 (209h38m42.647148871s) [upload=167939]: Upload finished. err: <nil>
Jul 2 02:51:28.868670051 (209h39m55.79496473s) [upload=167956]: Starting upload
Jul 2 02:51:28.868780851 (209h39m55.795075543s) [upload=167956]: Done starting upload
Jul 2 02:51:28.868884538 (209h39m55.795179225s) [upload=167956]: Uploading segments...
Jul 2 02:51:28.869381167 (209h39m55.795675855s) [upload=167956]: Got next segment. Inline: false
Jul 2 02:51:28.869486445 (209h39m55.795781135s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 2 02:51:28.869493196 (209h39m55.795787888s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 2 02:51:28.898707948 (209h39m55.825002664s) [seg_pos={0 0}, upload=167956]: Begin upload segment...
Jul 2 02:51:28.898787644 (209h39m55.825082342s) [seg_pos={0 0}, upload=167956]: Done begin upload segment.
Jul 2 02:51:28.898825319 (209h39m55.825120018s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.898864369 (209h39m55.825159066s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899423277 (209h39m55.825717966s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899547497 (209h39m55.825842195s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899793450 (209h39m55.82608814s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900152858 (209h39m55.826447544s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900210034 (209h39m55.826504727s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900634257 (209h39m55.826928949s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900678544 (209h39m55.826973233s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900990193 (209h39m55.827284883s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901196315 (209h39m55.827491002s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901407463 (209h39m55.827702159s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901732148 (209h39m55.828026843s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901817419 (209h39m55.828112106s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902270961 (209h39m55.828565648s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902808316 (209h39m55.829103005s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:29.002833067 (209h39m55.929127747s) [upload=167956]: Next returned nil segment
Jul 2 02:51:29.002849035 (209h39m55.929143729s) [upload=167956]: Waiting for error group managing segments...
Jul 2 02:51:29.009136933 (209h39m55.935431621s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.009959595 (209h39m55.936254298s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.010805328 (209h39m55.937100001s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.011547112 (209h39m55.9378418s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012145259 (209h39m55.938439932s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012854902 (209h39m55.939149581s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013365184 (209h39m55.939659859s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013906104 (209h39m55.940200783s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.014907872 (209h39m55.941202559s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015448646 (209h39m55.941743332s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015581494 (209h39m55.941876167s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015893548 (209h39m55.942188225s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017028456 (209h39m55.943323135s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017043139 (209h39m55.943337815s) [seg_pos={0 0}, upload=167956]: Segment reached optimal threshold of 13 pieces.
Jul 2 02:51:29.017178229 (209h39m55.943472905s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017310273 (209h39m55.94360495s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017394571 (209h39m55.943689253s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017409321 (209h39m55.943704012s) [seg_pos={0 0}, upload=167956]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 2 02:51:29.017486214 (209h39m55.943780908s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:51:29.017491827 (209h39m55.943786629s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:53:50.138713219 (209h42m17.065007921s) [upload=167956]: Done uploading segments...
Jul 2 02:53:50.138739023 (209h42m17.065033711s) [upload=167956]: Upload finished. err: metaclient: context deadline exceeded
Duration: 504.192098ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............................X.............................] duration:
594.19906ms
FAIL (2025-07-02 03:12:33 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
ece_id=4HWNWPU27EN6NDCHLIDTA7TR6SEDEGEEH374FY42E4S2YB64JIQA, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650643965 (209h38m42.57693864s) [noise=true , address=104.36.82.6:4873 , piece_id=U5UUFIN6D3CA65BQNOBGH2PXRXXUPJIHM4ICHMT2HGTNVYMYCQDQ, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650681786 (209h38m42.576976485s) [seg_pos={0 0}, upload=167939]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jul 2 02:50:15.650762048 (209h38m42.577056758s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:50:15.650767470 (209h38m42.577062168s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:50:15.720813246 (209h38m42.647107948s) [upload=167939]: Done uploading segments...
Jul 2 02:50:15.720854178 (209h38m42.647148871s) [upload=167939]: Upload finished. err: <nil>
Jul 2 02:51:28.868670051 (209h39m55.79496473s) [upload=167956]: Starting upload
Jul 2 02:51:28.868780851 (209h39m55.795075543s) [upload=167956]: Done starting upload
Jul 2 02:51:28.868884538 (209h39m55.795179225s) [upload=167956]: Uploading segments...
Jul 2 02:51:28.869381167 (209h39m55.795675855s) [upload=167956]: Got next segment. Inline: false
Jul 2 02:51:28.869486445 (209h39m55.795781135s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 2 02:51:28.869493196 (209h39m55.795787888s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 2 02:51:28.898707948 (209h39m55.825002664s) [seg_pos={0 0}, upload=167956]: Begin upload segment...
Jul 2 02:51:28.898787644 (209h39m55.825082342s) [seg_pos={0 0}, upload=167956]: Done begin upload segment.
Jul 2 02:51:28.898825319 (209h39m55.825120018s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.898864369 (209h39m55.825159066s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899423277 (209h39m55.825717966s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899547497 (209h39m55.825842195s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899793450 (209h39m55.82608814s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900152858 (209h39m55.826447544s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900210034 (209h39m55.826504727s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900634257 (209h39m55.826928949s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900678544 (209h39m55.826973233s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900990193 (209h39m55.827284883s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901196315 (209h39m55.827491002s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901407463 (209h39m55.827702159s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901732148 (209h39m55.828026843s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901817419 (209h39m55.828112106s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902270961 (209h39m55.828565648s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902808316 (209h39m55.829103005s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:29.002833067 (209h39m55.929127747s) [upload=167956]: Next returned nil segment
Jul 2 02:51:29.002849035 (209h39m55.929143729s) [upload=167956]: Waiting for error group managing segments...
Jul 2 02:51:29.009136933 (209h39m55.935431621s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.009959595 (209h39m55.936254298s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.010805328 (209h39m55.937100001s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.011547112 (209h39m55.9378418s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012145259 (209h39m55.938439932s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012854902 (209h39m55.939149581s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013365184 (209h39m55.939659859s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013906104 (209h39m55.940200783s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.014907872 (209h39m55.941202559s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015448646 (209h39m55.941743332s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015581494 (209h39m55.941876167s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015893548 (209h39m55.942188225s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017028456 (209h39m55.943323135s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017043139 (209h39m55.943337815s) [seg_pos={0 0}, upload=167956]: Segment reached optimal threshold of 13 pieces.
Jul 2 02:51:29.017178229 (209h39m55.943472905s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017310273 (209h39m55.94360495s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017394571 (209h39m55.943689253s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017409321 (209h39m55.943704012s) [seg_pos={0 0}, upload=167956]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 2 02:51:29.017486214 (209h39m55.943780908s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:51:29.017491827 (209h39m55.943786629s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:53:50.138713219 (209h42m17.065007921s) [upload=167956]: Done uploading segments...
Jul 2 02:53:50.138739023 (209h42m17.065033711s) [upload=167956]: Upload finished. err: metaclient: context deadline exceeded
Duration: 594.243465ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............................X............................] duration:
567.209622ms
FAIL (2025-07-02 03:12:05 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
ece_id=4HWNWPU27EN6NDCHLIDTA7TR6SEDEGEEH374FY42E4S2YB64JIQA, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650643965 (209h38m42.57693864s) [noise=true , address=104.36.82.6:4873 , piece_id=U5UUFIN6D3CA65BQNOBGH2PXRXXUPJIHM4ICHMT2HGTNVYMYCQDQ, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650681786 (209h38m42.576976485s) [seg_pos={0 0}, upload=167939]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jul 2 02:50:15.650762048 (209h38m42.577056758s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:50:15.650767470 (209h38m42.577062168s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:50:15.720813246 (209h38m42.647107948s) [upload=167939]: Done uploading segments...
Jul 2 02:50:15.720854178 (209h38m42.647148871s) [upload=167939]: Upload finished. err: <nil>
Jul 2 02:51:28.868670051 (209h39m55.79496473s) [upload=167956]: Starting upload
Jul 2 02:51:28.868780851 (209h39m55.795075543s) [upload=167956]: Done starting upload
Jul 2 02:51:28.868884538 (209h39m55.795179225s) [upload=167956]: Uploading segments...
Jul 2 02:51:28.869381167 (209h39m55.795675855s) [upload=167956]: Got next segment. Inline: false
Jul 2 02:51:28.869486445 (209h39m55.795781135s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 2 02:51:28.869493196 (209h39m55.795787888s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 2 02:51:28.898707948 (209h39m55.825002664s) [seg_pos={0 0}, upload=167956]: Begin upload segment...
Jul 2 02:51:28.898787644 (209h39m55.825082342s) [seg_pos={0 0}, upload=167956]: Done begin upload segment.
Jul 2 02:51:28.898825319 (209h39m55.825120018s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.898864369 (209h39m55.825159066s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899423277 (209h39m55.825717966s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899547497 (209h39m55.825842195s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899793450 (209h39m55.82608814s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900152858 (209h39m55.826447544s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900210034 (209h39m55.826504727s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900634257 (209h39m55.826928949s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900678544 (209h39m55.826973233s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900990193 (209h39m55.827284883s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901196315 (209h39m55.827491002s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901407463 (209h39m55.827702159s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901732148 (209h39m55.828026843s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901817419 (209h39m55.828112106s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902270961 (209h39m55.828565648s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902808316 (209h39m55.829103005s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:29.002833067 (209h39m55.929127747s) [upload=167956]: Next returned nil segment
Jul 2 02:51:29.002849035 (209h39m55.929143729s) [upload=167956]: Waiting for error group managing segments...
Jul 2 02:51:29.009136933 (209h39m55.935431621s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.009959595 (209h39m55.936254298s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.010805328 (209h39m55.937100001s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.011547112 (209h39m55.9378418s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012145259 (209h39m55.938439932s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012854902 (209h39m55.939149581s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013365184 (209h39m55.939659859s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013906104 (209h39m55.940200783s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.014907872 (209h39m55.941202559s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015448646 (209h39m55.941743332s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015581494 (209h39m55.941876167s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015893548 (209h39m55.942188225s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017028456 (209h39m55.943323135s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017043139 (209h39m55.943337815s) [seg_pos={0 0}, upload=167956]: Segment reached optimal threshold of 13 pieces.
Jul 2 02:51:29.017178229 (209h39m55.943472905s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017310273 (209h39m55.94360495s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017394571 (209h39m55.943689253s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017409321 (209h39m55.943704012s) [seg_pos={0 0}, upload=167956]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 2 02:51:29.017486214 (209h39m55.943780908s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:51:29.017491827 (209h39m55.943786629s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:53:50.138713219 (209h42m17.065007921s) [upload=167956]: Done uploading segments...
Jul 2 02:53:50.138739023 (209h42m17.065033711s) [upload=167956]: Upload finished. err: metaclient: context deadline exceeded
Duration: 567.267316ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [................................X...........................] duration:
481.799581ms
FAIL (2025-07-02 03:11:17 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
ece_id=4HWNWPU27EN6NDCHLIDTA7TR6SEDEGEEH374FY42E4S2YB64JIQA, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650643965 (209h38m42.57693864s) [noise=true , address=104.36.82.6:4873 , piece_id=U5UUFIN6D3CA65BQNOBGH2PXRXXUPJIHM4ICHMT2HGTNVYMYCQDQ, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650681786 (209h38m42.576976485s) [seg_pos={0 0}, upload=167939]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jul 2 02:50:15.650762048 (209h38m42.577056758s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:50:15.650767470 (209h38m42.577062168s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:50:15.720813246 (209h38m42.647107948s) [upload=167939]: Done uploading segments...
Jul 2 02:50:15.720854178 (209h38m42.647148871s) [upload=167939]: Upload finished. err: <nil>
Jul 2 02:51:28.868670051 (209h39m55.79496473s) [upload=167956]: Starting upload
Jul 2 02:51:28.868780851 (209h39m55.795075543s) [upload=167956]: Done starting upload
Jul 2 02:51:28.868884538 (209h39m55.795179225s) [upload=167956]: Uploading segments...
Jul 2 02:51:28.869381167 (209h39m55.795675855s) [upload=167956]: Got next segment. Inline: false
Jul 2 02:51:28.869486445 (209h39m55.795781135s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 2 02:51:28.869493196 (209h39m55.795787888s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 2 02:51:28.898707948 (209h39m55.825002664s) [seg_pos={0 0}, upload=167956]: Begin upload segment...
Jul 2 02:51:28.898787644 (209h39m55.825082342s) [seg_pos={0 0}, upload=167956]: Done begin upload segment.
Jul 2 02:51:28.898825319 (209h39m55.825120018s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.898864369 (209h39m55.825159066s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899423277 (209h39m55.825717966s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899547497 (209h39m55.825842195s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899793450 (209h39m55.82608814s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900152858 (209h39m55.826447544s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900210034 (209h39m55.826504727s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900634257 (209h39m55.826928949s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900678544 (209h39m55.826973233s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900990193 (209h39m55.827284883s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901196315 (209h39m55.827491002s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901407463 (209h39m55.827702159s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901732148 (209h39m55.828026843s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901817419 (209h39m55.828112106s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902270961 (209h39m55.828565648s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902808316 (209h39m55.829103005s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:29.002833067 (209h39m55.929127747s) [upload=167956]: Next returned nil segment
Jul 2 02:51:29.002849035 (209h39m55.929143729s) [upload=167956]: Waiting for error group managing segments...
Jul 2 02:51:29.009136933 (209h39m55.935431621s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.009959595 (209h39m55.936254298s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.010805328 (209h39m55.937100001s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.011547112 (209h39m55.9378418s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012145259 (209h39m55.938439932s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012854902 (209h39m55.939149581s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013365184 (209h39m55.939659859s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013906104 (209h39m55.940200783s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.014907872 (209h39m55.941202559s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015448646 (209h39m55.941743332s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015581494 (209h39m55.941876167s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015893548 (209h39m55.942188225s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017028456 (209h39m55.943323135s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017043139 (209h39m55.943337815s) [seg_pos={0 0}, upload=167956]: Segment reached optimal threshold of 13 pieces.
Jul 2 02:51:29.017178229 (209h39m55.943472905s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017310273 (209h39m55.94360495s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017394571 (209h39m55.943689253s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017409321 (209h39m55.943704012s) [seg_pos={0 0}, upload=167956]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 2 02:51:29.017486214 (209h39m55.943780908s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:51:29.017491827 (209h39m55.943786629s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:53:50.138713219 (209h42m17.065007921s) [upload=167956]: Done uploading segments...
Jul 2 02:53:50.138739023 (209h42m17.065033711s) [upload=167956]: Upload finished. err: metaclient: context deadline exceeded
Duration: 481.826007ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................................X..........................] duration:
575.836373ms
FAIL (2025-07-02 03:10:52 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
ece_id=4HWNWPU27EN6NDCHLIDTA7TR6SEDEGEEH374FY42E4S2YB64JIQA, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650643965 (209h38m42.57693864s) [noise=true , address=104.36.82.6:4873 , piece_id=U5UUFIN6D3CA65BQNOBGH2PXRXXUPJIHM4ICHMT2HGTNVYMYCQDQ, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650681786 (209h38m42.576976485s) [seg_pos={0 0}, upload=167939]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jul 2 02:50:15.650762048 (209h38m42.577056758s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:50:15.650767470 (209h38m42.577062168s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:50:15.720813246 (209h38m42.647107948s) [upload=167939]: Done uploading segments...
Jul 2 02:50:15.720854178 (209h38m42.647148871s) [upload=167939]: Upload finished. err: <nil>
Jul 2 02:51:28.868670051 (209h39m55.79496473s) [upload=167956]: Starting upload
Jul 2 02:51:28.868780851 (209h39m55.795075543s) [upload=167956]: Done starting upload
Jul 2 02:51:28.868884538 (209h39m55.795179225s) [upload=167956]: Uploading segments...
Jul 2 02:51:28.869381167 (209h39m55.795675855s) [upload=167956]: Got next segment. Inline: false
Jul 2 02:51:28.869486445 (209h39m55.795781135s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 2 02:51:28.869493196 (209h39m55.795787888s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 2 02:51:28.898707948 (209h39m55.825002664s) [seg_pos={0 0}, upload=167956]: Begin upload segment...
Jul 2 02:51:28.898787644 (209h39m55.825082342s) [seg_pos={0 0}, upload=167956]: Done begin upload segment.
Jul 2 02:51:28.898825319 (209h39m55.825120018s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.898864369 (209h39m55.825159066s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899423277 (209h39m55.825717966s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899547497 (209h39m55.825842195s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899793450 (209h39m55.82608814s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900152858 (209h39m55.826447544s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900210034 (209h39m55.826504727s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900634257 (209h39m55.826928949s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900678544 (209h39m55.826973233s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900990193 (209h39m55.827284883s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901196315 (209h39m55.827491002s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901407463 (209h39m55.827702159s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901732148 (209h39m55.828026843s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901817419 (209h39m55.828112106s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902270961 (209h39m55.828565648s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902808316 (209h39m55.829103005s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:29.002833067 (209h39m55.929127747s) [upload=167956]: Next returned nil segment
Jul 2 02:51:29.002849035 (209h39m55.929143729s) [upload=167956]: Waiting for error group managing segments...
Jul 2 02:51:29.009136933 (209h39m55.935431621s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.009959595 (209h39m55.936254298s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.010805328 (209h39m55.937100001s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.011547112 (209h39m55.9378418s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012145259 (209h39m55.938439932s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012854902 (209h39m55.939149581s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013365184 (209h39m55.939659859s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013906104 (209h39m55.940200783s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.014907872 (209h39m55.941202559s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015448646 (209h39m55.941743332s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015581494 (209h39m55.941876167s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015893548 (209h39m55.942188225s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017028456 (209h39m55.943323135s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017043139 (209h39m55.943337815s) [seg_pos={0 0}, upload=167956]: Segment reached optimal threshold of 13 pieces.
Jul 2 02:51:29.017178229 (209h39m55.943472905s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017310273 (209h39m55.94360495s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017394571 (209h39m55.943689253s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017409321 (209h39m55.943704012s) [seg_pos={0 0}, upload=167956]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 2 02:51:29.017486214 (209h39m55.943780908s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:51:29.017491827 (209h39m55.943786629s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:53:50.138713219 (209h42m17.065007921s) [upload=167956]: Done uploading segments...
Jul 2 02:53:50.138739023 (209h42m17.065033711s) [upload=167956]: Upload finished. err: metaclient: context deadline exceeded
Duration: 575.87314ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................................X.........................] duration:
585.57041ms
FAIL (2025-07-02 03:09:47 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
ece_id=4HWNWPU27EN6NDCHLIDTA7TR6SEDEGEEH374FY42E4S2YB64JIQA, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650643965 (209h38m42.57693864s) [noise=true , address=104.36.82.6:4873 , piece_id=U5UUFIN6D3CA65BQNOBGH2PXRXXUPJIHM4ICHMT2HGTNVYMYCQDQ, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650681786 (209h38m42.576976485s) [seg_pos={0 0}, upload=167939]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jul 2 02:50:15.650762048 (209h38m42.577056758s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:50:15.650767470 (209h38m42.577062168s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:50:15.720813246 (209h38m42.647107948s) [upload=167939]: Done uploading segments...
Jul 2 02:50:15.720854178 (209h38m42.647148871s) [upload=167939]: Upload finished. err: <nil>
Jul 2 02:51:28.868670051 (209h39m55.79496473s) [upload=167956]: Starting upload
Jul 2 02:51:28.868780851 (209h39m55.795075543s) [upload=167956]: Done starting upload
Jul 2 02:51:28.868884538 (209h39m55.795179225s) [upload=167956]: Uploading segments...
Jul 2 02:51:28.869381167 (209h39m55.795675855s) [upload=167956]: Got next segment. Inline: false
Jul 2 02:51:28.869486445 (209h39m55.795781135s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 2 02:51:28.869493196 (209h39m55.795787888s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 2 02:51:28.898707948 (209h39m55.825002664s) [seg_pos={0 0}, upload=167956]: Begin upload segment...
Jul 2 02:51:28.898787644 (209h39m55.825082342s) [seg_pos={0 0}, upload=167956]: Done begin upload segment.
Jul 2 02:51:28.898825319 (209h39m55.825120018s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.898864369 (209h39m55.825159066s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899423277 (209h39m55.825717966s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899547497 (209h39m55.825842195s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899793450 (209h39m55.82608814s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900152858 (209h39m55.826447544s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900210034 (209h39m55.826504727s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900634257 (209h39m55.826928949s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900678544 (209h39m55.826973233s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900990193 (209h39m55.827284883s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901196315 (209h39m55.827491002s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901407463 (209h39m55.827702159s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901732148 (209h39m55.828026843s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901817419 (209h39m55.828112106s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902270961 (209h39m55.828565648s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902808316 (209h39m55.829103005s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:29.002833067 (209h39m55.929127747s) [upload=167956]: Next returned nil segment
Jul 2 02:51:29.002849035 (209h39m55.929143729s) [upload=167956]: Waiting for error group managing segments...
Jul 2 02:51:29.009136933 (209h39m55.935431621s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.009959595 (209h39m55.936254298s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.010805328 (209h39m55.937100001s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.011547112 (209h39m55.9378418s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012145259 (209h39m55.938439932s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012854902 (209h39m55.939149581s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013365184 (209h39m55.939659859s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013906104 (209h39m55.940200783s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.014907872 (209h39m55.941202559s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015448646 (209h39m55.941743332s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015581494 (209h39m55.941876167s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015893548 (209h39m55.942188225s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017028456 (209h39m55.943323135s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017043139 (209h39m55.943337815s) [seg_pos={0 0}, upload=167956]: Segment reached optimal threshold of 13 pieces.
Jul 2 02:51:29.017178229 (209h39m55.943472905s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017310273 (209h39m55.94360495s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017394571 (209h39m55.943689253s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017409321 (209h39m55.943704012s) [seg_pos={0 0}, upload=167956]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 2 02:51:29.017486214 (209h39m55.943780908s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:51:29.017491827 (209h39m55.943786629s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:53:50.138713219 (209h42m17.065007921s) [upload=167956]: Done uploading segments...
Jul 2 02:53:50.138739023 (209h42m17.065033711s) [upload=167956]: Upload finished. err: metaclient: context deadline exceeded
Duration: 585.599314ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................................X........................] duration:
749.949935ms
FAIL (2025-07-02 03:09:04 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
ece_id=4HWNWPU27EN6NDCHLIDTA7TR6SEDEGEEH374FY42E4S2YB64JIQA, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650643965 (209h38m42.57693864s) [noise=true , address=104.36.82.6:4873 , piece_id=U5UUFIN6D3CA65BQNOBGH2PXRXXUPJIHM4ICHMT2HGTNVYMYCQDQ, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650681786 (209h38m42.576976485s) [seg_pos={0 0}, upload=167939]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jul 2 02:50:15.650762048 (209h38m42.577056758s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:50:15.650767470 (209h38m42.577062168s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:50:15.720813246 (209h38m42.647107948s) [upload=167939]: Done uploading segments...
Jul 2 02:50:15.720854178 (209h38m42.647148871s) [upload=167939]: Upload finished. err: <nil>
Jul 2 02:51:28.868670051 (209h39m55.79496473s) [upload=167956]: Starting upload
Jul 2 02:51:28.868780851 (209h39m55.795075543s) [upload=167956]: Done starting upload
Jul 2 02:51:28.868884538 (209h39m55.795179225s) [upload=167956]: Uploading segments...
Jul 2 02:51:28.869381167 (209h39m55.795675855s) [upload=167956]: Got next segment. Inline: false
Jul 2 02:51:28.869486445 (209h39m55.795781135s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 2 02:51:28.869493196 (209h39m55.795787888s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 2 02:51:28.898707948 (209h39m55.825002664s) [seg_pos={0 0}, upload=167956]: Begin upload segment...
Jul 2 02:51:28.898787644 (209h39m55.825082342s) [seg_pos={0 0}, upload=167956]: Done begin upload segment.
Jul 2 02:51:28.898825319 (209h39m55.825120018s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.898864369 (209h39m55.825159066s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899423277 (209h39m55.825717966s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899547497 (209h39m55.825842195s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899793450 (209h39m55.82608814s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900152858 (209h39m55.826447544s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900210034 (209h39m55.826504727s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900634257 (209h39m55.826928949s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900678544 (209h39m55.826973233s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900990193 (209h39m55.827284883s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901196315 (209h39m55.827491002s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901407463 (209h39m55.827702159s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901732148 (209h39m55.828026843s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901817419 (209h39m55.828112106s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902270961 (209h39m55.828565648s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902808316 (209h39m55.829103005s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:29.002833067 (209h39m55.929127747s) [upload=167956]: Next returned nil segment
Jul 2 02:51:29.002849035 (209h39m55.929143729s) [upload=167956]: Waiting for error group managing segments...
Jul 2 02:51:29.009136933 (209h39m55.935431621s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.009959595 (209h39m55.936254298s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.010805328 (209h39m55.937100001s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.011547112 (209h39m55.9378418s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012145259 (209h39m55.938439932s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012854902 (209h39m55.939149581s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013365184 (209h39m55.939659859s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013906104 (209h39m55.940200783s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.014907872 (209h39m55.941202559s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015448646 (209h39m55.941743332s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015581494 (209h39m55.941876167s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015893548 (209h39m55.942188225s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017028456 (209h39m55.943323135s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017043139 (209h39m55.943337815s) [seg_pos={0 0}, upload=167956]: Segment reached optimal threshold of 13 pieces.
Jul 2 02:51:29.017178229 (209h39m55.943472905s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017310273 (209h39m55.94360495s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017394571 (209h39m55.943689253s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017409321 (209h39m55.943704012s) [seg_pos={0 0}, upload=167956]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 2 02:51:29.017486214 (209h39m55.943780908s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:51:29.017491827 (209h39m55.943786629s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:53:50.138713219 (209h42m17.065007921s) [upload=167956]: Done uploading segments...
Jul 2 02:53:50.138739023 (209h42m17.065033711s) [upload=167956]: Upload finished. err: metaclient: context deadline exceeded
Duration: 750.059823ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................................X.......................] duration:
552.30766ms
FAIL (2025-07-02 03:08:28 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
ece_id=4HWNWPU27EN6NDCHLIDTA7TR6SEDEGEEH374FY42E4S2YB64JIQA, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650643965 (209h38m42.57693864s) [noise=true , address=104.36.82.6:4873 , piece_id=U5UUFIN6D3CA65BQNOBGH2PXRXXUPJIHM4ICHMT2HGTNVYMYCQDQ, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650681786 (209h38m42.576976485s) [seg_pos={0 0}, upload=167939]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jul 2 02:50:15.650762048 (209h38m42.577056758s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:50:15.650767470 (209h38m42.577062168s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:50:15.720813246 (209h38m42.647107948s) [upload=167939]: Done uploading segments...
Jul 2 02:50:15.720854178 (209h38m42.647148871s) [upload=167939]: Upload finished. err: <nil>
Jul 2 02:51:28.868670051 (209h39m55.79496473s) [upload=167956]: Starting upload
Jul 2 02:51:28.868780851 (209h39m55.795075543s) [upload=167956]: Done starting upload
Jul 2 02:51:28.868884538 (209h39m55.795179225s) [upload=167956]: Uploading segments...
Jul 2 02:51:28.869381167 (209h39m55.795675855s) [upload=167956]: Got next segment. Inline: false
Jul 2 02:51:28.869486445 (209h39m55.795781135s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 2 02:51:28.869493196 (209h39m55.795787888s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 2 02:51:28.898707948 (209h39m55.825002664s) [seg_pos={0 0}, upload=167956]: Begin upload segment...
Jul 2 02:51:28.898787644 (209h39m55.825082342s) [seg_pos={0 0}, upload=167956]: Done begin upload segment.
Jul 2 02:51:28.898825319 (209h39m55.825120018s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.898864369 (209h39m55.825159066s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899423277 (209h39m55.825717966s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899547497 (209h39m55.825842195s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899793450 (209h39m55.82608814s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900152858 (209h39m55.826447544s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900210034 (209h39m55.826504727s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900634257 (209h39m55.826928949s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900678544 (209h39m55.826973233s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900990193 (209h39m55.827284883s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901196315 (209h39m55.827491002s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901407463 (209h39m55.827702159s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901732148 (209h39m55.828026843s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901817419 (209h39m55.828112106s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902270961 (209h39m55.828565648s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902808316 (209h39m55.829103005s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:29.002833067 (209h39m55.929127747s) [upload=167956]: Next returned nil segment
Jul 2 02:51:29.002849035 (209h39m55.929143729s) [upload=167956]: Waiting for error group managing segments...
Jul 2 02:51:29.009136933 (209h39m55.935431621s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.009959595 (209h39m55.936254298s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.010805328 (209h39m55.937100001s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.011547112 (209h39m55.9378418s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012145259 (209h39m55.938439932s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012854902 (209h39m55.939149581s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013365184 (209h39m55.939659859s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013906104 (209h39m55.940200783s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.014907872 (209h39m55.941202559s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015448646 (209h39m55.941743332s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015581494 (209h39m55.941876167s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015893548 (209h39m55.942188225s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017028456 (209h39m55.943323135s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017043139 (209h39m55.943337815s) [seg_pos={0 0}, upload=167956]: Segment reached optimal threshold of 13 pieces.
Jul 2 02:51:29.017178229 (209h39m55.943472905s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017310273 (209h39m55.94360495s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017394571 (209h39m55.943689253s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017409321 (209h39m55.943704012s) [seg_pos={0 0}, upload=167956]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 2 02:51:29.017486214 (209h39m55.943780908s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:51:29.017491827 (209h39m55.943786629s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:53:50.138713219 (209h42m17.065007921s) [upload=167956]: Done uploading segments...
Jul 2 02:53:50.138739023 (209h42m17.065033711s) [upload=167956]: Upload finished. err: metaclient: context deadline exceeded
Duration: 552.720495ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................................X......................] duration:
512.7168ms
FAIL (2025-07-02 03:07:21 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
ece_id=4HWNWPU27EN6NDCHLIDTA7TR6SEDEGEEH374FY42E4S2YB64JIQA, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650643965 (209h38m42.57693864s) [noise=true , address=104.36.82.6:4873 , piece_id=U5UUFIN6D3CA65BQNOBGH2PXRXXUPJIHM4ICHMT2HGTNVYMYCQDQ, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650681786 (209h38m42.576976485s) [seg_pos={0 0}, upload=167939]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jul 2 02:50:15.650762048 (209h38m42.577056758s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:50:15.650767470 (209h38m42.577062168s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:50:15.720813246 (209h38m42.647107948s) [upload=167939]: Done uploading segments...
Jul 2 02:50:15.720854178 (209h38m42.647148871s) [upload=167939]: Upload finished. err: <nil>
Jul 2 02:51:28.868670051 (209h39m55.79496473s) [upload=167956]: Starting upload
Jul 2 02:51:28.868780851 (209h39m55.795075543s) [upload=167956]: Done starting upload
Jul 2 02:51:28.868884538 (209h39m55.795179225s) [upload=167956]: Uploading segments...
Jul 2 02:51:28.869381167 (209h39m55.795675855s) [upload=167956]: Got next segment. Inline: false
Jul 2 02:51:28.869486445 (209h39m55.795781135s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 2 02:51:28.869493196 (209h39m55.795787888s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 2 02:51:28.898707948 (209h39m55.825002664s) [seg_pos={0 0}, upload=167956]: Begin upload segment...
Jul 2 02:51:28.898787644 (209h39m55.825082342s) [seg_pos={0 0}, upload=167956]: Done begin upload segment.
Jul 2 02:51:28.898825319 (209h39m55.825120018s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.898864369 (209h39m55.825159066s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899423277 (209h39m55.825717966s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899547497 (209h39m55.825842195s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899793450 (209h39m55.82608814s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900152858 (209h39m55.826447544s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900210034 (209h39m55.826504727s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900634257 (209h39m55.826928949s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900678544 (209h39m55.826973233s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900990193 (209h39m55.827284883s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901196315 (209h39m55.827491002s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901407463 (209h39m55.827702159s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901732148 (209h39m55.828026843s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901817419 (209h39m55.828112106s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902270961 (209h39m55.828565648s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902808316 (209h39m55.829103005s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:29.002833067 (209h39m55.929127747s) [upload=167956]: Next returned nil segment
Jul 2 02:51:29.002849035 (209h39m55.929143729s) [upload=167956]: Waiting for error group managing segments...
Jul 2 02:51:29.009136933 (209h39m55.935431621s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.009959595 (209h39m55.936254298s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.010805328 (209h39m55.937100001s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.011547112 (209h39m55.9378418s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012145259 (209h39m55.938439932s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012854902 (209h39m55.939149581s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013365184 (209h39m55.939659859s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013906104 (209h39m55.940200783s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.014907872 (209h39m55.941202559s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015448646 (209h39m55.941743332s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015581494 (209h39m55.941876167s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015893548 (209h39m55.942188225s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017028456 (209h39m55.943323135s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017043139 (209h39m55.943337815s) [seg_pos={0 0}, upload=167956]: Segment reached optimal threshold of 13 pieces.
Jul 2 02:51:29.017178229 (209h39m55.943472905s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017310273 (209h39m55.94360495s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017394571 (209h39m55.943689253s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017409321 (209h39m55.943704012s) [seg_pos={0 0}, upload=167956]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 2 02:51:29.017486214 (209h39m55.943780908s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:51:29.017491827 (209h39m55.943786629s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:53:50.138713219 (209h42m17.065007921s) [upload=167956]: Done uploading segments...
Jul 2 02:53:50.138739023 (209h42m17.065033711s) [upload=167956]: Upload finished. err: metaclient: context deadline exceeded
Duration: 512.750378ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................................X.....................] duration:
549.815105ms
FAIL (2025-07-02 03:07:04 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
ece_id=4HWNWPU27EN6NDCHLIDTA7TR6SEDEGEEH374FY42E4S2YB64JIQA, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650643965 (209h38m42.57693864s) [noise=true , address=104.36.82.6:4873 , piece_id=U5UUFIN6D3CA65BQNOBGH2PXRXXUPJIHM4ICHMT2HGTNVYMYCQDQ, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650681786 (209h38m42.576976485s) [seg_pos={0 0}, upload=167939]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jul 2 02:50:15.650762048 (209h38m42.577056758s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:50:15.650767470 (209h38m42.577062168s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:50:15.720813246 (209h38m42.647107948s) [upload=167939]: Done uploading segments...
Jul 2 02:50:15.720854178 (209h38m42.647148871s) [upload=167939]: Upload finished. err: <nil>
Jul 2 02:51:28.868670051 (209h39m55.79496473s) [upload=167956]: Starting upload
Jul 2 02:51:28.868780851 (209h39m55.795075543s) [upload=167956]: Done starting upload
Jul 2 02:51:28.868884538 (209h39m55.795179225s) [upload=167956]: Uploading segments...
Jul 2 02:51:28.869381167 (209h39m55.795675855s) [upload=167956]: Got next segment. Inline: false
Jul 2 02:51:28.869486445 (209h39m55.795781135s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 2 02:51:28.869493196 (209h39m55.795787888s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 2 02:51:28.898707948 (209h39m55.825002664s) [seg_pos={0 0}, upload=167956]: Begin upload segment...
Jul 2 02:51:28.898787644 (209h39m55.825082342s) [seg_pos={0 0}, upload=167956]: Done begin upload segment.
Jul 2 02:51:28.898825319 (209h39m55.825120018s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.898864369 (209h39m55.825159066s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899423277 (209h39m55.825717966s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899547497 (209h39m55.825842195s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899793450 (209h39m55.82608814s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900152858 (209h39m55.826447544s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900210034 (209h39m55.826504727s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900634257 (209h39m55.826928949s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900678544 (209h39m55.826973233s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900990193 (209h39m55.827284883s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901196315 (209h39m55.827491002s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901407463 (209h39m55.827702159s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901732148 (209h39m55.828026843s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901817419 (209h39m55.828112106s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902270961 (209h39m55.828565648s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902808316 (209h39m55.829103005s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:29.002833067 (209h39m55.929127747s) [upload=167956]: Next returned nil segment
Jul 2 02:51:29.002849035 (209h39m55.929143729s) [upload=167956]: Waiting for error group managing segments...
Jul 2 02:51:29.009136933 (209h39m55.935431621s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.009959595 (209h39m55.936254298s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.010805328 (209h39m55.937100001s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.011547112 (209h39m55.9378418s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012145259 (209h39m55.938439932s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012854902 (209h39m55.939149581s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013365184 (209h39m55.939659859s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013906104 (209h39m55.940200783s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.014907872 (209h39m55.941202559s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015448646 (209h39m55.941743332s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015581494 (209h39m55.941876167s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015893548 (209h39m55.942188225s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017028456 (209h39m55.943323135s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017043139 (209h39m55.943337815s) [seg_pos={0 0}, upload=167956]: Segment reached optimal threshold of 13 pieces.
Jul 2 02:51:29.017178229 (209h39m55.943472905s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017310273 (209h39m55.94360495s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017394571 (209h39m55.943689253s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017409321 (209h39m55.943704012s) [seg_pos={0 0}, upload=167956]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 2 02:51:29.017486214 (209h39m55.943780908s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:51:29.017491827 (209h39m55.943786629s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:53:50.138713219 (209h42m17.065007921s) [upload=167956]: Done uploading segments...
Jul 2 02:53:50.138739023 (209h42m17.065033711s) [upload=167956]: Upload finished. err: metaclient: context deadline exceeded
Duration: 549.841021ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......................................X....................] duration:
532.617925ms
FAIL (2025-07-02 03:06:20 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
ece_id=4HWNWPU27EN6NDCHLIDTA7TR6SEDEGEEH374FY42E4S2YB64JIQA, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650643965 (209h38m42.57693864s) [noise=true , address=104.36.82.6:4873 , piece_id=U5UUFIN6D3CA65BQNOBGH2PXRXXUPJIHM4ICHMT2HGTNVYMYCQDQ, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650681786 (209h38m42.576976485s) [seg_pos={0 0}, upload=167939]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jul 2 02:50:15.650762048 (209h38m42.577056758s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:50:15.650767470 (209h38m42.577062168s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:50:15.720813246 (209h38m42.647107948s) [upload=167939]: Done uploading segments...
Jul 2 02:50:15.720854178 (209h38m42.647148871s) [upload=167939]: Upload finished. err: <nil>
Jul 2 02:51:28.868670051 (209h39m55.79496473s) [upload=167956]: Starting upload
Jul 2 02:51:28.868780851 (209h39m55.795075543s) [upload=167956]: Done starting upload
Jul 2 02:51:28.868884538 (209h39m55.795179225s) [upload=167956]: Uploading segments...
Jul 2 02:51:28.869381167 (209h39m55.795675855s) [upload=167956]: Got next segment. Inline: false
Jul 2 02:51:28.869486445 (209h39m55.795781135s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 2 02:51:28.869493196 (209h39m55.795787888s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 2 02:51:28.898707948 (209h39m55.825002664s) [seg_pos={0 0}, upload=167956]: Begin upload segment...
Jul 2 02:51:28.898787644 (209h39m55.825082342s) [seg_pos={0 0}, upload=167956]: Done begin upload segment.
Jul 2 02:51:28.898825319 (209h39m55.825120018s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.898864369 (209h39m55.825159066s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899423277 (209h39m55.825717966s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899547497 (209h39m55.825842195s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899793450 (209h39m55.82608814s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900152858 (209h39m55.826447544s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900210034 (209h39m55.826504727s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900634257 (209h39m55.826928949s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900678544 (209h39m55.826973233s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900990193 (209h39m55.827284883s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901196315 (209h39m55.827491002s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901407463 (209h39m55.827702159s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901732148 (209h39m55.828026843s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901817419 (209h39m55.828112106s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902270961 (209h39m55.828565648s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902808316 (209h39m55.829103005s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:29.002833067 (209h39m55.929127747s) [upload=167956]: Next returned nil segment
Jul 2 02:51:29.002849035 (209h39m55.929143729s) [upload=167956]: Waiting for error group managing segments...
Jul 2 02:51:29.009136933 (209h39m55.935431621s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.009959595 (209h39m55.936254298s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.010805328 (209h39m55.937100001s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.011547112 (209h39m55.9378418s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012145259 (209h39m55.938439932s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012854902 (209h39m55.939149581s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013365184 (209h39m55.939659859s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013906104 (209h39m55.940200783s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.014907872 (209h39m55.941202559s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015448646 (209h39m55.941743332s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015581494 (209h39m55.941876167s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015893548 (209h39m55.942188225s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017028456 (209h39m55.943323135s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017043139 (209h39m55.943337815s) [seg_pos={0 0}, upload=167956]: Segment reached optimal threshold of 13 pieces.
Jul 2 02:51:29.017178229 (209h39m55.943472905s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017310273 (209h39m55.94360495s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017394571 (209h39m55.943689253s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017409321 (209h39m55.943704012s) [seg_pos={0 0}, upload=167956]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 2 02:51:29.017486214 (209h39m55.943780908s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:51:29.017491827 (209h39m55.943786629s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:53:50.138713219 (209h42m17.065007921s) [upload=167956]: Done uploading segments...
Jul 2 02:53:50.138739023 (209h42m17.065033711s) [upload=167956]: Upload finished. err: metaclient: context deadline exceeded
Duration: 532.65008ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................................X...................] duration:
480.899384ms
FAIL (2025-07-02 03:05:17 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
ece_id=4HWNWPU27EN6NDCHLIDTA7TR6SEDEGEEH374FY42E4S2YB64JIQA, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650643965 (209h38m42.57693864s) [noise=true , address=104.36.82.6:4873 , piece_id=U5UUFIN6D3CA65BQNOBGH2PXRXXUPJIHM4ICHMT2HGTNVYMYCQDQ, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650681786 (209h38m42.576976485s) [seg_pos={0 0}, upload=167939]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jul 2 02:50:15.650762048 (209h38m42.577056758s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:50:15.650767470 (209h38m42.577062168s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:50:15.720813246 (209h38m42.647107948s) [upload=167939]: Done uploading segments...
Jul 2 02:50:15.720854178 (209h38m42.647148871s) [upload=167939]: Upload finished. err: <nil>
Jul 2 02:51:28.868670051 (209h39m55.79496473s) [upload=167956]: Starting upload
Jul 2 02:51:28.868780851 (209h39m55.795075543s) [upload=167956]: Done starting upload
Jul 2 02:51:28.868884538 (209h39m55.795179225s) [upload=167956]: Uploading segments...
Jul 2 02:51:28.869381167 (209h39m55.795675855s) [upload=167956]: Got next segment. Inline: false
Jul 2 02:51:28.869486445 (209h39m55.795781135s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 2 02:51:28.869493196 (209h39m55.795787888s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 2 02:51:28.898707948 (209h39m55.825002664s) [seg_pos={0 0}, upload=167956]: Begin upload segment...
Jul 2 02:51:28.898787644 (209h39m55.825082342s) [seg_pos={0 0}, upload=167956]: Done begin upload segment.
Jul 2 02:51:28.898825319 (209h39m55.825120018s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.898864369 (209h39m55.825159066s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899423277 (209h39m55.825717966s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899547497 (209h39m55.825842195s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899793450 (209h39m55.82608814s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900152858 (209h39m55.826447544s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900210034 (209h39m55.826504727s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900634257 (209h39m55.826928949s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900678544 (209h39m55.826973233s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900990193 (209h39m55.827284883s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901196315 (209h39m55.827491002s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901407463 (209h39m55.827702159s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901732148 (209h39m55.828026843s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901817419 (209h39m55.828112106s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902270961 (209h39m55.828565648s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902808316 (209h39m55.829103005s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:29.002833067 (209h39m55.929127747s) [upload=167956]: Next returned nil segment
Jul 2 02:51:29.002849035 (209h39m55.929143729s) [upload=167956]: Waiting for error group managing segments...
Jul 2 02:51:29.009136933 (209h39m55.935431621s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.009959595 (209h39m55.936254298s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.010805328 (209h39m55.937100001s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.011547112 (209h39m55.9378418s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012145259 (209h39m55.938439932s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012854902 (209h39m55.939149581s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013365184 (209h39m55.939659859s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013906104 (209h39m55.940200783s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.014907872 (209h39m55.941202559s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015448646 (209h39m55.941743332s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015581494 (209h39m55.941876167s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015893548 (209h39m55.942188225s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017028456 (209h39m55.943323135s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017043139 (209h39m55.943337815s) [seg_pos={0 0}, upload=167956]: Segment reached optimal threshold of 13 pieces.
Jul 2 02:51:29.017178229 (209h39m55.943472905s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017310273 (209h39m55.94360495s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017394571 (209h39m55.943689253s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017409321 (209h39m55.943704012s) [seg_pos={0 0}, upload=167956]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 2 02:51:29.017486214 (209h39m55.943780908s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:51:29.017491827 (209h39m55.943786629s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:53:50.138713219 (209h42m17.065007921s) [upload=167956]: Done uploading segments...
Jul 2 02:53:50.138739023 (209h42m17.065033711s) [upload=167956]: Upload finished. err: metaclient: context deadline exceeded
Duration: 480.934536ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................................X..................] duration:
473.497259ms
FAIL (2025-07-02 03:04:10 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
ece_id=4HWNWPU27EN6NDCHLIDTA7TR6SEDEGEEH374FY42E4S2YB64JIQA, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650643965 (209h38m42.57693864s) [noise=true , address=104.36.82.6:4873 , piece_id=U5UUFIN6D3CA65BQNOBGH2PXRXXUPJIHM4ICHMT2HGTNVYMYCQDQ, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650681786 (209h38m42.576976485s) [seg_pos={0 0}, upload=167939]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jul 2 02:50:15.650762048 (209h38m42.577056758s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:50:15.650767470 (209h38m42.577062168s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:50:15.720813246 (209h38m42.647107948s) [upload=167939]: Done uploading segments...
Jul 2 02:50:15.720854178 (209h38m42.647148871s) [upload=167939]: Upload finished. err: <nil>
Jul 2 02:51:28.868670051 (209h39m55.79496473s) [upload=167956]: Starting upload
Jul 2 02:51:28.868780851 (209h39m55.795075543s) [upload=167956]: Done starting upload
Jul 2 02:51:28.868884538 (209h39m55.795179225s) [upload=167956]: Uploading segments...
Jul 2 02:51:28.869381167 (209h39m55.795675855s) [upload=167956]: Got next segment. Inline: false
Jul 2 02:51:28.869486445 (209h39m55.795781135s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 2 02:51:28.869493196 (209h39m55.795787888s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 2 02:51:28.898707948 (209h39m55.825002664s) [seg_pos={0 0}, upload=167956]: Begin upload segment...
Jul 2 02:51:28.898787644 (209h39m55.825082342s) [seg_pos={0 0}, upload=167956]: Done begin upload segment.
Jul 2 02:51:28.898825319 (209h39m55.825120018s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.898864369 (209h39m55.825159066s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899423277 (209h39m55.825717966s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899547497 (209h39m55.825842195s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899793450 (209h39m55.82608814s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900152858 (209h39m55.826447544s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900210034 (209h39m55.826504727s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900634257 (209h39m55.826928949s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900678544 (209h39m55.826973233s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900990193 (209h39m55.827284883s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901196315 (209h39m55.827491002s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901407463 (209h39m55.827702159s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901732148 (209h39m55.828026843s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901817419 (209h39m55.828112106s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902270961 (209h39m55.828565648s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902808316 (209h39m55.829103005s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:29.002833067 (209h39m55.929127747s) [upload=167956]: Next returned nil segment
Jul 2 02:51:29.002849035 (209h39m55.929143729s) [upload=167956]: Waiting for error group managing segments...
Jul 2 02:51:29.009136933 (209h39m55.935431621s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.009959595 (209h39m55.936254298s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.010805328 (209h39m55.937100001s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.011547112 (209h39m55.9378418s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012145259 (209h39m55.938439932s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012854902 (209h39m55.939149581s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013365184 (209h39m55.939659859s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013906104 (209h39m55.940200783s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.014907872 (209h39m55.941202559s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015448646 (209h39m55.941743332s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015581494 (209h39m55.941876167s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015893548 (209h39m55.942188225s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017028456 (209h39m55.943323135s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017043139 (209h39m55.943337815s) [seg_pos={0 0}, upload=167956]: Segment reached optimal threshold of 13 pieces.
Jul 2 02:51:29.017178229 (209h39m55.943472905s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017310273 (209h39m55.94360495s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017394571 (209h39m55.943689253s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017409321 (209h39m55.943704012s) [seg_pos={0 0}, upload=167956]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 2 02:51:29.017486214 (209h39m55.943780908s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:51:29.017491827 (209h39m55.943786629s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:53:50.138713219 (209h42m17.065007921s) [upload=167956]: Done uploading segments...
Jul 2 02:53:50.138739023 (209h42m17.065033711s) [upload=167956]: Upload finished. err: metaclient: context deadline exceeded
Duration: 473.527283ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................................X.................] duration:
389.385302ms
FAIL (2025-07-02 03:03:45 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
ece_id=4HWNWPU27EN6NDCHLIDTA7TR6SEDEGEEH374FY42E4S2YB64JIQA, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650643965 (209h38m42.57693864s) [noise=true , address=104.36.82.6:4873 , piece_id=U5UUFIN6D3CA65BQNOBGH2PXRXXUPJIHM4ICHMT2HGTNVYMYCQDQ, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650681786 (209h38m42.576976485s) [seg_pos={0 0}, upload=167939]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jul 2 02:50:15.650762048 (209h38m42.577056758s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:50:15.650767470 (209h38m42.577062168s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:50:15.720813246 (209h38m42.647107948s) [upload=167939]: Done uploading segments...
Jul 2 02:50:15.720854178 (209h38m42.647148871s) [upload=167939]: Upload finished. err: <nil>
Jul 2 02:51:28.868670051 (209h39m55.79496473s) [upload=167956]: Starting upload
Jul 2 02:51:28.868780851 (209h39m55.795075543s) [upload=167956]: Done starting upload
Jul 2 02:51:28.868884538 (209h39m55.795179225s) [upload=167956]: Uploading segments...
Jul 2 02:51:28.869381167 (209h39m55.795675855s) [upload=167956]: Got next segment. Inline: false
Jul 2 02:51:28.869486445 (209h39m55.795781135s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 2 02:51:28.869493196 (209h39m55.795787888s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 2 02:51:28.898707948 (209h39m55.825002664s) [seg_pos={0 0}, upload=167956]: Begin upload segment...
Jul 2 02:51:28.898787644 (209h39m55.825082342s) [seg_pos={0 0}, upload=167956]: Done begin upload segment.
Jul 2 02:51:28.898825319 (209h39m55.825120018s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.898864369 (209h39m55.825159066s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899423277 (209h39m55.825717966s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899547497 (209h39m55.825842195s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899793450 (209h39m55.82608814s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900152858 (209h39m55.826447544s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900210034 (209h39m55.826504727s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900634257 (209h39m55.826928949s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900678544 (209h39m55.826973233s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900990193 (209h39m55.827284883s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901196315 (209h39m55.827491002s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901407463 (209h39m55.827702159s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901732148 (209h39m55.828026843s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901817419 (209h39m55.828112106s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902270961 (209h39m55.828565648s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902808316 (209h39m55.829103005s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:29.002833067 (209h39m55.929127747s) [upload=167956]: Next returned nil segment
Jul 2 02:51:29.002849035 (209h39m55.929143729s) [upload=167956]: Waiting for error group managing segments...
Jul 2 02:51:29.009136933 (209h39m55.935431621s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.009959595 (209h39m55.936254298s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.010805328 (209h39m55.937100001s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.011547112 (209h39m55.9378418s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012145259 (209h39m55.938439932s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012854902 (209h39m55.939149581s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013365184 (209h39m55.939659859s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013906104 (209h39m55.940200783s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.014907872 (209h39m55.941202559s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015448646 (209h39m55.941743332s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015581494 (209h39m55.941876167s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015893548 (209h39m55.942188225s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017028456 (209h39m55.943323135s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017043139 (209h39m55.943337815s) [seg_pos={0 0}, upload=167956]: Segment reached optimal threshold of 13 pieces.
Jul 2 02:51:29.017178229 (209h39m55.943472905s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017310273 (209h39m55.94360495s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017394571 (209h39m55.943689253s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017409321 (209h39m55.943704012s) [seg_pos={0 0}, upload=167956]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 2 02:51:29.017486214 (209h39m55.943780908s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:51:29.017491827 (209h39m55.943786629s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:53:50.138713219 (209h42m17.065007921s) [upload=167956]: Done uploading segments...
Jul 2 02:53:50.138739023 (209h42m17.065033711s) [upload=167956]: Upload finished. err: metaclient: context deadline exceeded
Duration: 389.413129ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................] duration:
454.388036ms
FAIL (2025-07-02 03:03:25 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
ece_id=4HWNWPU27EN6NDCHLIDTA7TR6SEDEGEEH374FY42E4S2YB64JIQA, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650643965 (209h38m42.57693864s) [noise=true , address=104.36.82.6:4873 , piece_id=U5UUFIN6D3CA65BQNOBGH2PXRXXUPJIHM4ICHMT2HGTNVYMYCQDQ, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650681786 (209h38m42.576976485s) [seg_pos={0 0}, upload=167939]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jul 2 02:50:15.650762048 (209h38m42.577056758s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:50:15.650767470 (209h38m42.577062168s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:50:15.720813246 (209h38m42.647107948s) [upload=167939]: Done uploading segments...
Jul 2 02:50:15.720854178 (209h38m42.647148871s) [upload=167939]: Upload finished. err: <nil>
Jul 2 02:51:28.868670051 (209h39m55.79496473s) [upload=167956]: Starting upload
Jul 2 02:51:28.868780851 (209h39m55.795075543s) [upload=167956]: Done starting upload
Jul 2 02:51:28.868884538 (209h39m55.795179225s) [upload=167956]: Uploading segments...
Jul 2 02:51:28.869381167 (209h39m55.795675855s) [upload=167956]: Got next segment. Inline: false
Jul 2 02:51:28.869486445 (209h39m55.795781135s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 2 02:51:28.869493196 (209h39m55.795787888s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 2 02:51:28.898707948 (209h39m55.825002664s) [seg_pos={0 0}, upload=167956]: Begin upload segment...
Jul 2 02:51:28.898787644 (209h39m55.825082342s) [seg_pos={0 0}, upload=167956]: Done begin upload segment.
Jul 2 02:51:28.898825319 (209h39m55.825120018s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.898864369 (209h39m55.825159066s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899423277 (209h39m55.825717966s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899547497 (209h39m55.825842195s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899793450 (209h39m55.82608814s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900152858 (209h39m55.826447544s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900210034 (209h39m55.826504727s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900634257 (209h39m55.826928949s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900678544 (209h39m55.826973233s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900990193 (209h39m55.827284883s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901196315 (209h39m55.827491002s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901407463 (209h39m55.827702159s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901732148 (209h39m55.828026843s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901817419 (209h39m55.828112106s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902270961 (209h39m55.828565648s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902808316 (209h39m55.829103005s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:29.002833067 (209h39m55.929127747s) [upload=167956]: Next returned nil segment
Jul 2 02:51:29.002849035 (209h39m55.929143729s) [upload=167956]: Waiting for error group managing segments...
Jul 2 02:51:29.009136933 (209h39m55.935431621s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.009959595 (209h39m55.936254298s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.010805328 (209h39m55.937100001s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.011547112 (209h39m55.9378418s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012145259 (209h39m55.938439932s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012854902 (209h39m55.939149581s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013365184 (209h39m55.939659859s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013906104 (209h39m55.940200783s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.014907872 (209h39m55.941202559s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015448646 (209h39m55.941743332s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015581494 (209h39m55.941876167s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015893548 (209h39m55.942188225s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017028456 (209h39m55.943323135s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017043139 (209h39m55.943337815s) [seg_pos={0 0}, upload=167956]: Segment reached optimal threshold of 13 pieces.
Jul 2 02:51:29.017178229 (209h39m55.943472905s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017310273 (209h39m55.94360495s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017394571 (209h39m55.943689253s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017409321 (209h39m55.943704012s) [seg_pos={0 0}, upload=167956]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 2 02:51:29.017486214 (209h39m55.943780908s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:51:29.017491827 (209h39m55.943786629s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:53:50.138713219 (209h42m17.065007921s) [upload=167956]: Done uploading segments...
Jul 2 02:53:50.138739023 (209h42m17.065033711s) [upload=167956]: Upload finished. err: metaclient: context deadline exceeded
Duration: 454.415479ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............] duration:
528.335379ms
FAIL (2025-07-02 03:02:38 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
ece_id=4HWNWPU27EN6NDCHLIDTA7TR6SEDEGEEH374FY42E4S2YB64JIQA, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650643965 (209h38m42.57693864s) [noise=true , address=104.36.82.6:4873 , piece_id=U5UUFIN6D3CA65BQNOBGH2PXRXXUPJIHM4ICHMT2HGTNVYMYCQDQ, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650681786 (209h38m42.576976485s) [seg_pos={0 0}, upload=167939]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jul 2 02:50:15.650762048 (209h38m42.577056758s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:50:15.650767470 (209h38m42.577062168s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:50:15.720813246 (209h38m42.647107948s) [upload=167939]: Done uploading segments...
Jul 2 02:50:15.720854178 (209h38m42.647148871s) [upload=167939]: Upload finished. err: <nil>
Jul 2 02:51:28.868670051 (209h39m55.79496473s) [upload=167956]: Starting upload
Jul 2 02:51:28.868780851 (209h39m55.795075543s) [upload=167956]: Done starting upload
Jul 2 02:51:28.868884538 (209h39m55.795179225s) [upload=167956]: Uploading segments...
Jul 2 02:51:28.869381167 (209h39m55.795675855s) [upload=167956]: Got next segment. Inline: false
Jul 2 02:51:28.869486445 (209h39m55.795781135s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 2 02:51:28.869493196 (209h39m55.795787888s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 2 02:51:28.898707948 (209h39m55.825002664s) [seg_pos={0 0}, upload=167956]: Begin upload segment...
Jul 2 02:51:28.898787644 (209h39m55.825082342s) [seg_pos={0 0}, upload=167956]: Done begin upload segment.
Jul 2 02:51:28.898825319 (209h39m55.825120018s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.898864369 (209h39m55.825159066s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899423277 (209h39m55.825717966s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899547497 (209h39m55.825842195s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899793450 (209h39m55.82608814s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900152858 (209h39m55.826447544s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900210034 (209h39m55.826504727s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900634257 (209h39m55.826928949s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900678544 (209h39m55.826973233s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900990193 (209h39m55.827284883s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901196315 (209h39m55.827491002s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901407463 (209h39m55.827702159s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901732148 (209h39m55.828026843s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901817419 (209h39m55.828112106s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902270961 (209h39m55.828565648s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902808316 (209h39m55.829103005s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:29.002833067 (209h39m55.929127747s) [upload=167956]: Next returned nil segment
Jul 2 02:51:29.002849035 (209h39m55.929143729s) [upload=167956]: Waiting for error group managing segments...
Jul 2 02:51:29.009136933 (209h39m55.935431621s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.009959595 (209h39m55.936254298s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.010805328 (209h39m55.937100001s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.011547112 (209h39m55.9378418s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012145259 (209h39m55.938439932s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012854902 (209h39m55.939149581s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013365184 (209h39m55.939659859s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013906104 (209h39m55.940200783s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.014907872 (209h39m55.941202559s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015448646 (209h39m55.941743332s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015581494 (209h39m55.941876167s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015893548 (209h39m55.942188225s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017028456 (209h39m55.943323135s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017043139 (209h39m55.943337815s) [seg_pos={0 0}, upload=167956]: Segment reached optimal threshold of 13 pieces.
Jul 2 02:51:29.017178229 (209h39m55.943472905s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017310273 (209h39m55.94360495s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017394571 (209h39m55.943689253s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017409321 (209h39m55.943704012s) [seg_pos={0 0}, upload=167956]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 2 02:51:29.017486214 (209h39m55.943780908s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:51:29.017491827 (209h39m55.943786629s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:53:50.138713219 (209h42m17.065007921s) [upload=167956]: Done uploading segments...
Jul 2 02:53:50.138739023 (209h42m17.065033711s) [upload=167956]: Upload finished. err: metaclient: context deadline exceeded
Duration: 528.361669ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............] duration:
552.896511ms
FAIL (2025-07-02 03:02:01 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
ece_id=4HWNWPU27EN6NDCHLIDTA7TR6SEDEGEEH374FY42E4S2YB64JIQA, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650643965 (209h38m42.57693864s) [noise=true , address=104.36.82.6:4873 , piece_id=U5UUFIN6D3CA65BQNOBGH2PXRXXUPJIHM4ICHMT2HGTNVYMYCQDQ, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650681786 (209h38m42.576976485s) [seg_pos={0 0}, upload=167939]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jul 2 02:50:15.650762048 (209h38m42.577056758s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:50:15.650767470 (209h38m42.577062168s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:50:15.720813246 (209h38m42.647107948s) [upload=167939]: Done uploading segments...
Jul 2 02:50:15.720854178 (209h38m42.647148871s) [upload=167939]: Upload finished. err: <nil>
Jul 2 02:51:28.868670051 (209h39m55.79496473s) [upload=167956]: Starting upload
Jul 2 02:51:28.868780851 (209h39m55.795075543s) [upload=167956]: Done starting upload
Jul 2 02:51:28.868884538 (209h39m55.795179225s) [upload=167956]: Uploading segments...
Jul 2 02:51:28.869381167 (209h39m55.795675855s) [upload=167956]: Got next segment. Inline: false
Jul 2 02:51:28.869486445 (209h39m55.795781135s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 2 02:51:28.869493196 (209h39m55.795787888s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 2 02:51:28.898707948 (209h39m55.825002664s) [seg_pos={0 0}, upload=167956]: Begin upload segment...
Jul 2 02:51:28.898787644 (209h39m55.825082342s) [seg_pos={0 0}, upload=167956]: Done begin upload segment.
Jul 2 02:51:28.898825319 (209h39m55.825120018s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.898864369 (209h39m55.825159066s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899423277 (209h39m55.825717966s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899547497 (209h39m55.825842195s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899793450 (209h39m55.82608814s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900152858 (209h39m55.826447544s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900210034 (209h39m55.826504727s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900634257 (209h39m55.826928949s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900678544 (209h39m55.826973233s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900990193 (209h39m55.827284883s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901196315 (209h39m55.827491002s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901407463 (209h39m55.827702159s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901732148 (209h39m55.828026843s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901817419 (209h39m55.828112106s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902270961 (209h39m55.828565648s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902808316 (209h39m55.829103005s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:29.002833067 (209h39m55.929127747s) [upload=167956]: Next returned nil segment
Jul 2 02:51:29.002849035 (209h39m55.929143729s) [upload=167956]: Waiting for error group managing segments...
Jul 2 02:51:29.009136933 (209h39m55.935431621s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.009959595 (209h39m55.936254298s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.010805328 (209h39m55.937100001s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.011547112 (209h39m55.9378418s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012145259 (209h39m55.938439932s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012854902 (209h39m55.939149581s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013365184 (209h39m55.939659859s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013906104 (209h39m55.940200783s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.014907872 (209h39m55.941202559s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015448646 (209h39m55.941743332s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015581494 (209h39m55.941876167s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015893548 (209h39m55.942188225s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017028456 (209h39m55.943323135s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017043139 (209h39m55.943337815s) [seg_pos={0 0}, upload=167956]: Segment reached optimal threshold of 13 pieces.
Jul 2 02:51:29.017178229 (209h39m55.943472905s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017310273 (209h39m55.94360495s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017394571 (209h39m55.943689253s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017409321 (209h39m55.943704012s) [seg_pos={0 0}, upload=167956]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 2 02:51:29.017486214 (209h39m55.943780908s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:51:29.017491827 (209h39m55.943786629s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:53:50.138713219 (209h42m17.065007921s) [upload=167956]: Done uploading segments...
Jul 2 02:53:50.138739023 (209h42m17.065033711s) [upload=167956]: Upload finished. err: metaclient: context deadline exceeded
Duration: 552.949002ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............................................X.............] duration:
462.649942ms
FAIL (2025-07-02 03:01:44 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
ece_id=4HWNWPU27EN6NDCHLIDTA7TR6SEDEGEEH374FY42E4S2YB64JIQA, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650643965 (209h38m42.57693864s) [noise=true , address=104.36.82.6:4873 , piece_id=U5UUFIN6D3CA65BQNOBGH2PXRXXUPJIHM4ICHMT2HGTNVYMYCQDQ, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650681786 (209h38m42.576976485s) [seg_pos={0 0}, upload=167939]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jul 2 02:50:15.650762048 (209h38m42.577056758s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:50:15.650767470 (209h38m42.577062168s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:50:15.720813246 (209h38m42.647107948s) [upload=167939]: Done uploading segments...
Jul 2 02:50:15.720854178 (209h38m42.647148871s) [upload=167939]: Upload finished. err: <nil>
Jul 2 02:51:28.868670051 (209h39m55.79496473s) [upload=167956]: Starting upload
Jul 2 02:51:28.868780851 (209h39m55.795075543s) [upload=167956]: Done starting upload
Jul 2 02:51:28.868884538 (209h39m55.795179225s) [upload=167956]: Uploading segments...
Jul 2 02:51:28.869381167 (209h39m55.795675855s) [upload=167956]: Got next segment. Inline: false
Jul 2 02:51:28.869486445 (209h39m55.795781135s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 2 02:51:28.869493196 (209h39m55.795787888s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 2 02:51:28.898707948 (209h39m55.825002664s) [seg_pos={0 0}, upload=167956]: Begin upload segment...
Jul 2 02:51:28.898787644 (209h39m55.825082342s) [seg_pos={0 0}, upload=167956]: Done begin upload segment.
Jul 2 02:51:28.898825319 (209h39m55.825120018s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.898864369 (209h39m55.825159066s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899423277 (209h39m55.825717966s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899547497 (209h39m55.825842195s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899793450 (209h39m55.82608814s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900152858 (209h39m55.826447544s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900210034 (209h39m55.826504727s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900634257 (209h39m55.826928949s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900678544 (209h39m55.826973233s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900990193 (209h39m55.827284883s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901196315 (209h39m55.827491002s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901407463 (209h39m55.827702159s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901732148 (209h39m55.828026843s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901817419 (209h39m55.828112106s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902270961 (209h39m55.828565648s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902808316 (209h39m55.829103005s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:29.002833067 (209h39m55.929127747s) [upload=167956]: Next returned nil segment
Jul 2 02:51:29.002849035 (209h39m55.929143729s) [upload=167956]: Waiting for error group managing segments...
Jul 2 02:51:29.009136933 (209h39m55.935431621s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.009959595 (209h39m55.936254298s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.010805328 (209h39m55.937100001s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.011547112 (209h39m55.9378418s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012145259 (209h39m55.938439932s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012854902 (209h39m55.939149581s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013365184 (209h39m55.939659859s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013906104 (209h39m55.940200783s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.014907872 (209h39m55.941202559s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015448646 (209h39m55.941743332s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015581494 (209h39m55.941876167s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015893548 (209h39m55.942188225s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017028456 (209h39m55.943323135s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017043139 (209h39m55.943337815s) [seg_pos={0 0}, upload=167956]: Segment reached optimal threshold of 13 pieces.
Jul 2 02:51:29.017178229 (209h39m55.943472905s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017310273 (209h39m55.94360495s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017394571 (209h39m55.943689253s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017409321 (209h39m55.943704012s) [seg_pos={0 0}, upload=167956]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 2 02:51:29.017486214 (209h39m55.943780908s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:51:29.017491827 (209h39m55.943786629s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:53:50.138713219 (209h42m17.065007921s) [upload=167956]: Done uploading segments...
Jul 2 02:53:50.138739023 (209h42m17.065033711s) [upload=167956]: Upload finished. err: metaclient: context deadline exceeded
Duration: 462.697541ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............] duration:
680.203864ms
FAIL (2025-07-02 03:01:24 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
ece_id=4HWNWPU27EN6NDCHLIDTA7TR6SEDEGEEH374FY42E4S2YB64JIQA, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650643965 (209h38m42.57693864s) [noise=true , address=104.36.82.6:4873 , piece_id=U5UUFIN6D3CA65BQNOBGH2PXRXXUPJIHM4ICHMT2HGTNVYMYCQDQ, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650681786 (209h38m42.576976485s) [seg_pos={0 0}, upload=167939]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jul 2 02:50:15.650762048 (209h38m42.577056758s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:50:15.650767470 (209h38m42.577062168s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:50:15.720813246 (209h38m42.647107948s) [upload=167939]: Done uploading segments...
Jul 2 02:50:15.720854178 (209h38m42.647148871s) [upload=167939]: Upload finished. err: <nil>
Jul 2 02:51:28.868670051 (209h39m55.79496473s) [upload=167956]: Starting upload
Jul 2 02:51:28.868780851 (209h39m55.795075543s) [upload=167956]: Done starting upload
Jul 2 02:51:28.868884538 (209h39m55.795179225s) [upload=167956]: Uploading segments...
Jul 2 02:51:28.869381167 (209h39m55.795675855s) [upload=167956]: Got next segment. Inline: false
Jul 2 02:51:28.869486445 (209h39m55.795781135s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 2 02:51:28.869493196 (209h39m55.795787888s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 2 02:51:28.898707948 (209h39m55.825002664s) [seg_pos={0 0}, upload=167956]: Begin upload segment...
Jul 2 02:51:28.898787644 (209h39m55.825082342s) [seg_pos={0 0}, upload=167956]: Done begin upload segment.
Jul 2 02:51:28.898825319 (209h39m55.825120018s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.898864369 (209h39m55.825159066s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899423277 (209h39m55.825717966s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899547497 (209h39m55.825842195s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899793450 (209h39m55.82608814s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900152858 (209h39m55.826447544s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900210034 (209h39m55.826504727s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900634257 (209h39m55.826928949s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900678544 (209h39m55.826973233s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900990193 (209h39m55.827284883s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901196315 (209h39m55.827491002s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901407463 (209h39m55.827702159s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901732148 (209h39m55.828026843s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901817419 (209h39m55.828112106s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902270961 (209h39m55.828565648s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902808316 (209h39m55.829103005s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:29.002833067 (209h39m55.929127747s) [upload=167956]: Next returned nil segment
Jul 2 02:51:29.002849035 (209h39m55.929143729s) [upload=167956]: Waiting for error group managing segments...
Jul 2 02:51:29.009136933 (209h39m55.935431621s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.009959595 (209h39m55.936254298s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.010805328 (209h39m55.937100001s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.011547112 (209h39m55.9378418s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012145259 (209h39m55.938439932s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012854902 (209h39m55.939149581s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013365184 (209h39m55.939659859s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013906104 (209h39m55.940200783s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.014907872 (209h39m55.941202559s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015448646 (209h39m55.941743332s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015581494 (209h39m55.941876167s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015893548 (209h39m55.942188225s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017028456 (209h39m55.943323135s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017043139 (209h39m55.943337815s) [seg_pos={0 0}, upload=167956]: Segment reached optimal threshold of 13 pieces.
Jul 2 02:51:29.017178229 (209h39m55.943472905s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017310273 (209h39m55.94360495s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017394571 (209h39m55.943689253s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017409321 (209h39m55.943704012s) [seg_pos={0 0}, upload=167956]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 2 02:51:29.017486214 (209h39m55.943780908s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:51:29.017491827 (209h39m55.943786629s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:53:50.138713219 (209h42m17.065007921s) [upload=167956]: Done uploading segments...
Jul 2 02:53:50.138739023 (209h42m17.065033711s) [upload=167956]: Upload finished. err: metaclient: context deadline exceeded
Duration: 680.239269ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........] duration:
442.195505ms
FAIL (2025-07-02 03:00:51 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
ece_id=4HWNWPU27EN6NDCHLIDTA7TR6SEDEGEEH374FY42E4S2YB64JIQA, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650643965 (209h38m42.57693864s) [noise=true , address=104.36.82.6:4873 , piece_id=U5UUFIN6D3CA65BQNOBGH2PXRXXUPJIHM4ICHMT2HGTNVYMYCQDQ, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650681786 (209h38m42.576976485s) [seg_pos={0 0}, upload=167939]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jul 2 02:50:15.650762048 (209h38m42.577056758s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:50:15.650767470 (209h38m42.577062168s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:50:15.720813246 (209h38m42.647107948s) [upload=167939]: Done uploading segments...
Jul 2 02:50:15.720854178 (209h38m42.647148871s) [upload=167939]: Upload finished. err: <nil>
Jul 2 02:51:28.868670051 (209h39m55.79496473s) [upload=167956]: Starting upload
Jul 2 02:51:28.868780851 (209h39m55.795075543s) [upload=167956]: Done starting upload
Jul 2 02:51:28.868884538 (209h39m55.795179225s) [upload=167956]: Uploading segments...
Jul 2 02:51:28.869381167 (209h39m55.795675855s) [upload=167956]: Got next segment. Inline: false
Jul 2 02:51:28.869486445 (209h39m55.795781135s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 2 02:51:28.869493196 (209h39m55.795787888s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 2 02:51:28.898707948 (209h39m55.825002664s) [seg_pos={0 0}, upload=167956]: Begin upload segment...
Jul 2 02:51:28.898787644 (209h39m55.825082342s) [seg_pos={0 0}, upload=167956]: Done begin upload segment.
Jul 2 02:51:28.898825319 (209h39m55.825120018s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.898864369 (209h39m55.825159066s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899423277 (209h39m55.825717966s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899547497 (209h39m55.825842195s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899793450 (209h39m55.82608814s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900152858 (209h39m55.826447544s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900210034 (209h39m55.826504727s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900634257 (209h39m55.826928949s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900678544 (209h39m55.826973233s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900990193 (209h39m55.827284883s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901196315 (209h39m55.827491002s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901407463 (209h39m55.827702159s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901732148 (209h39m55.828026843s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901817419 (209h39m55.828112106s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902270961 (209h39m55.828565648s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902808316 (209h39m55.829103005s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:29.002833067 (209h39m55.929127747s) [upload=167956]: Next returned nil segment
Jul 2 02:51:29.002849035 (209h39m55.929143729s) [upload=167956]: Waiting for error group managing segments...
Jul 2 02:51:29.009136933 (209h39m55.935431621s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.009959595 (209h39m55.936254298s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.010805328 (209h39m55.937100001s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.011547112 (209h39m55.9378418s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012145259 (209h39m55.938439932s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012854902 (209h39m55.939149581s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013365184 (209h39m55.939659859s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013906104 (209h39m55.940200783s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.014907872 (209h39m55.941202559s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015448646 (209h39m55.941743332s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015581494 (209h39m55.941876167s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015893548 (209h39m55.942188225s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017028456 (209h39m55.943323135s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017043139 (209h39m55.943337815s) [seg_pos={0 0}, upload=167956]: Segment reached optimal threshold of 13 pieces.
Jul 2 02:51:29.017178229 (209h39m55.943472905s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017310273 (209h39m55.94360495s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017394571 (209h39m55.943689253s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017409321 (209h39m55.943704012s) [seg_pos={0 0}, upload=167956]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 2 02:51:29.017486214 (209h39m55.943780908s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:51:29.017491827 (209h39m55.943786629s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:53:50.138713219 (209h42m17.065007921s) [upload=167956]: Done uploading segments...
Jul 2 02:53:50.138739023 (209h42m17.065033711s) [upload=167956]: Upload finished. err: metaclient: context deadline exceeded
Duration: 442.22196ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........] duration:
492.022234ms
FAIL (2025-07-02 03:00:31 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
ece_id=4HWNWPU27EN6NDCHLIDTA7TR6SEDEGEEH374FY42E4S2YB64JIQA, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650643965 (209h38m42.57693864s) [noise=true , address=104.36.82.6:4873 , piece_id=U5UUFIN6D3CA65BQNOBGH2PXRXXUPJIHM4ICHMT2HGTNVYMYCQDQ, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650681786 (209h38m42.576976485s) [seg_pos={0 0}, upload=167939]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jul 2 02:50:15.650762048 (209h38m42.577056758s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:50:15.650767470 (209h38m42.577062168s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:50:15.720813246 (209h38m42.647107948s) [upload=167939]: Done uploading segments...
Jul 2 02:50:15.720854178 (209h38m42.647148871s) [upload=167939]: Upload finished. err: <nil>
Jul 2 02:51:28.868670051 (209h39m55.79496473s) [upload=167956]: Starting upload
Jul 2 02:51:28.868780851 (209h39m55.795075543s) [upload=167956]: Done starting upload
Jul 2 02:51:28.868884538 (209h39m55.795179225s) [upload=167956]: Uploading segments...
Jul 2 02:51:28.869381167 (209h39m55.795675855s) [upload=167956]: Got next segment. Inline: false
Jul 2 02:51:28.869486445 (209h39m55.795781135s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 2 02:51:28.869493196 (209h39m55.795787888s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 2 02:51:28.898707948 (209h39m55.825002664s) [seg_pos={0 0}, upload=167956]: Begin upload segment...
Jul 2 02:51:28.898787644 (209h39m55.825082342s) [seg_pos={0 0}, upload=167956]: Done begin upload segment.
Jul 2 02:51:28.898825319 (209h39m55.825120018s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.898864369 (209h39m55.825159066s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899423277 (209h39m55.825717966s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899547497 (209h39m55.825842195s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899793450 (209h39m55.82608814s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900152858 (209h39m55.826447544s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900210034 (209h39m55.826504727s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900634257 (209h39m55.826928949s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900678544 (209h39m55.826973233s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900990193 (209h39m55.827284883s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901196315 (209h39m55.827491002s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901407463 (209h39m55.827702159s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901732148 (209h39m55.828026843s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901817419 (209h39m55.828112106s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902270961 (209h39m55.828565648s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902808316 (209h39m55.829103005s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:29.002833067 (209h39m55.929127747s) [upload=167956]: Next returned nil segment
Jul 2 02:51:29.002849035 (209h39m55.929143729s) [upload=167956]: Waiting for error group managing segments...
Jul 2 02:51:29.009136933 (209h39m55.935431621s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.009959595 (209h39m55.936254298s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.010805328 (209h39m55.937100001s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.011547112 (209h39m55.9378418s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012145259 (209h39m55.938439932s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012854902 (209h39m55.939149581s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013365184 (209h39m55.939659859s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013906104 (209h39m55.940200783s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.014907872 (209h39m55.941202559s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015448646 (209h39m55.941743332s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015581494 (209h39m55.941876167s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015893548 (209h39m55.942188225s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017028456 (209h39m55.943323135s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017043139 (209h39m55.943337815s) [seg_pos={0 0}, upload=167956]: Segment reached optimal threshold of 13 pieces.
Jul 2 02:51:29.017178229 (209h39m55.943472905s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017310273 (209h39m55.94360495s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017394571 (209h39m55.943689253s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017409321 (209h39m55.943704012s) [seg_pos={0 0}, upload=167956]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 2 02:51:29.017486214 (209h39m55.943780908s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:51:29.017491827 (209h39m55.943786629s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:53:50.138713219 (209h42m17.065007921s) [upload=167956]: Done uploading segments...
Jul 2 02:53:50.138739023 (209h42m17.065033711s) [upload=167956]: Upload finished. err: metaclient: context deadline exceeded
Duration: 492.052208ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
457.621037ms
FAIL (2025-07-02 02:59:46 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
ece_id=4HWNWPU27EN6NDCHLIDTA7TR6SEDEGEEH374FY42E4S2YB64JIQA, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650643965 (209h38m42.57693864s) [noise=true , address=104.36.82.6:4873 , piece_id=U5UUFIN6D3CA65BQNOBGH2PXRXXUPJIHM4ICHMT2HGTNVYMYCQDQ, seg_pos={0 0}, upload=167939]: Done uploading piece. err: <nil>
Jul 2 02:50:15.650681786 (209h38m42.576976485s) [seg_pos={0 0}, upload=167939]: Done waiting for segment. successful: 15 optimal: 13 errs: <nil>
Jul 2 02:50:15.650762048 (209h38m42.577056758s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:50:15.650767470 (209h38m42.577062168s) [upload=167939]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:50:15.720813246 (209h38m42.647107948s) [upload=167939]: Done uploading segments...
Jul 2 02:50:15.720854178 (209h38m42.647148871s) [upload=167939]: Upload finished. err: <nil>
Jul 2 02:51:28.868670051 (209h39m55.79496473s) [upload=167956]: Starting upload
Jul 2 02:51:28.868780851 (209h39m55.795075543s) [upload=167956]: Done starting upload
Jul 2 02:51:28.868884538 (209h39m55.795179225s) [upload=167956]: Uploading segments...
Jul 2 02:51:28.869381167 (209h39m55.795675855s) [upload=167956]: Got next segment. Inline: false
Jul 2 02:51:28.869486445 (209h39m55.795781135s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 2 02:51:28.869493196 (209h39m55.795787888s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 2 02:51:28.898707948 (209h39m55.825002664s) [seg_pos={0 0}, upload=167956]: Begin upload segment...
Jul 2 02:51:28.898787644 (209h39m55.825082342s) [seg_pos={0 0}, upload=167956]: Done begin upload segment.
Jul 2 02:51:28.898825319 (209h39m55.825120018s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.898864369 (209h39m55.825159066s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899423277 (209h39m55.825717966s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899547497 (209h39m55.825842195s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.899793450 (209h39m55.82608814s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900152858 (209h39m55.826447544s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900210034 (209h39m55.826504727s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900634257 (209h39m55.826928949s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900678544 (209h39m55.826973233s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.900990193 (209h39m55.827284883s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901196315 (209h39m55.827491002s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901407463 (209h39m55.827702159s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901732148 (209h39m55.828026843s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.901817419 (209h39m55.828112106s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902270961 (209h39m55.828565648s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:28.902808316 (209h39m55.829103005s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Uploading piece...
Jul 2 02:51:29.002833067 (209h39m55.929127747s) [upload=167956]: Next returned nil segment
Jul 2 02:51:29.002849035 (209h39m55.929143729s) [upload=167956]: Waiting for error group managing segments...
Jul 2 02:51:29.009136933 (209h39m55.935431621s) [noise=true , address=45.91.116.7:4011 , piece_id=OZPUOS5C2SXVMQ2UZO6GTOJGBNB44GP3FVDNEJ3TPV2S5AQQVKTA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.009959595 (209h39m55.936254298s) [noise=true , address=45.91.116.6:4973 , piece_id=J7CKQ2OY7GHL6AOAYQSFXV3OFGXFOXFECAX73DSMYN4I3BLWPLCQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.010805328 (209h39m55.937100001s) [noise=true , address=77.90.31.18:4807 , piece_id=SDJ5LDCGRMHFES4HBLA2YJHZBTBWCQUMZTJUKEAQCN6IV5EPS4RQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.011547112 (209h39m55.9378418s) [noise=true , address=77.90.31.19:4841 , piece_id=SPMT7Z5FJZEGSKMP2NRLVTQZYOWLRYHW2D3PUXNGBPUQEMSB5EXQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012145259 (209h39m55.938439932s) [noise=true , address=77.90.31.21:4879 , piece_id=XQJYDN6UJK4QH54DJRQQ4OC4AZXSKVOELEIWDKE4KVL2MXJSYQGA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.012854902 (209h39m55.939149581s) [noise=true , address=104.36.82.2:4732 , piece_id=R4KQGFFMSRGFAZCQIN3ABA4MBUKRVY7H6X7VLD7FDSUACSTEB3JA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013365184 (209h39m55.939659859s) [noise=true , address=104.36.82.16:4295 , piece_id=MDKAEJ6U4QKMYWVLISRRT4YCJDKCLDYM6UIW4NYAOAJC5C3HZYBA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.013906104 (209h39m55.940200783s) [noise=true , address=45.91.116.14:4189 , piece_id=W6QMHY4YNWVGK7JGDRNW4R2XHOLOA36LHEGE2572ECQXUS54SBLQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.014907872 (209h39m55.941202559s) [noise=true , address=104.36.82.4:4801 , piece_id=VKANDMNYUM2TWXEJIMVBP4VCUDQHMJUCKVXBDT2Y5HKWZH7RM6AA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015448646 (209h39m55.941743332s) [noise=true , address=104.36.82.16:4293 , piece_id=KQLDNUQZQNJANZLTPHSGBUROZQIUG6B3F46BH6SXXJYMRZ7JUIDQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015581494 (209h39m55.941876167s) [noise=true , address=104.36.82.11:4115 , piece_id=UZYHXFCH5RBH66TE7L7BB7JO4WTF2JMQQWOYQDANBF5WLNKHQWPQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.015893548 (209h39m55.942188225s) [noise=true , address=104.36.82.3:4767 , piece_id=SMXUO4BJEZGF6CNWUBR3COHII766QREWKBSBHZV36XAE6OKREIUQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017028456 (209h39m55.943323135s) [noise=true , address=104.36.82.12:4151 , piece_id=LDQKELV6DGYKJKK24FV7Y463CDUV37ODLSXXCYUDUXG3M372LGCA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: <nil>
Jul 2 02:51:29.017043139 (209h39m55.943337815s) [seg_pos={0 0}, upload=167956]: Segment reached optimal threshold of 13 pieces.
Jul 2 02:51:29.017178229 (209h39m55.943472905s) [noise=true , address=104.36.82.6:4874 , piece_id=Y2OV6DP3QYECFBZU2R2OHY5NAILXKQC2AD2F2PWTIMX63XSVBKFQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017310273 (209h39m55.94360495s) [noise=true , address=104.36.82.10:4078 , piece_id=K4U2MB7WNX62IPDYMSVTL2MCUBC6HHRH56Y2IFDJJ6SSNZO6ODNA, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017394571 (209h39m55.943689253s) [noise=true , address=104.36.82.13:4185 , piece_id=Q4J5PORHO4R3RSWFSCSZEFFT3BXFGBDOLT2S3ENFEV5PTEP5JUIQ, seg_pos={0 0}, upload=167956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN): protocol: expected piece hash; context canceled; EOF
Jul 2 02:51:29.017409321 (209h39m55.943704012s) [seg_pos={0 0}, upload=167956]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 2 02:51:29.017486214 (209h39m55.943780908s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 2 02:51:29.017491827 (209h39m55.943786629s) [upload=167956]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 2 02:53:50.138713219 (209h42m17.065007921s) [upload=167956]: Done uploading segments...
Jul 2 02:53:50.138739023 (209h42m17.065033711s) [upload=167956]: Upload finished. err: metaclient: context deadline exceeded
Duration: 457.651616ms
core
uplink
us-select-1
|