Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............................X..............................] duration:
500.185712ms
FAIL (2025-07-23 06:23:23 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
23 06:00:19.742341858 (103h10m1.590948949s) [noise=true , address=104.36.82.4:4804 , piece_id=SMCQKI5GIWKEHYLNSVUGHWQNBHP3S4C5RBB4E3K5CFMXE54Q4WAQ, seg_pos={0 0}, upload=82455]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g): protocol: expected piece hash; context canceled; EOF
Jul 23 06:00:19.742959346 (103h10m1.59156642s) [noise=true , address=77.90.31.16:4734 , piece_id=NFCPPB6P25YMO5UZRM4CQKMNRV5Q2FXMSR6M4VSTEERBMVNLYJJA, seg_pos={0 0}, upload=82455]: Done uploading piece. err: <nil>
Jul 23 06:00:19.743002232 (103h10m1.591609306s) [seg_pos={0 0}, upload=82455]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 23 06:00:19.743286680 (103h10m1.591893764s) [upload=82455]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 23 06:00:19.743293769 (103h10m1.591900841s) [upload=82455]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 23 06:00:19.815641707 (103h10m1.664248798s) [upload=82455]: Done uploading segments...
Jul 23 06:00:19.815666887 (103h10m1.664273954s) [upload=82455]: Upload finished. err: <nil>
Jul 23 06:01:08.401614022 (103h10m50.250221093s) [upload=82463]: Starting upload
Jul 23 06:01:08.401709823 (103h10m50.250316908s) [upload=82463]: Done starting upload
Jul 23 06:01:08.402138436 (103h10m50.250745505s) [upload=82463]: Uploading segments...
Jul 23 06:01:08.402189044 (103h10m50.250796137s) [upload=82463]: Got next segment. Inline: false
Jul 23 06:01:08.402231416 (103h10m50.250838487s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 23 06:01:08.402237731 (103h10m50.250844804s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 23 06:01:08.450301042 (103h10m50.298908146s) [seg_pos={0 0}, upload=82463]: Begin upload segment...
Jul 23 06:01:08.450364720 (103h10m50.298971798s) [seg_pos={0 0}, upload=82463]: Done begin upload segment.
Jul 23 06:01:08.450425015 (103h10m50.2990321s) [noise=true , address=45.91.116.5:4299 , piece_id=K3URA4ZO4R2CAGZLTO6YJTZQOZ6YIOB7XOJ3AQAVGDFIMVU63BIA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.450431954 (103h10m50.299039031s) [noise=true , address=45.91.116.7:4010 , piece_id=WVMOM2PPTVXY7NYI5UH25PFW6RODIUJWN73YB6FW6UOAMNY4C4SA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.450900413 (103h10m50.299507502s) [noise=true , address=77.90.31.17:4772 , piece_id=63V72ATK3KI3ADRC4N5HTRIXSEB3DRM7N73MVZNDPZJ7EVO3RZJA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.450963993 (103h10m50.299571083s) [noise=true , address=45.91.116.8:4048 , piece_id=AROG4HVUX7R6MHG3DT3N2FIQJBBEVVEVKQHSAGZCWMISWTFSQMAQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.451375091 (103h10m50.299982178s) [noise=true , address=45.91.116.13:4155 , piece_id=KMDEG3QMJNHRF7TETZAFVN3TOZWTQNW7IRTCVI3WLNE7NJAAAPCA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.451400410 (103h10m50.300007509s) [noise=true , address=77.90.31.12:4592 , piece_id=NAWAKPN4LUPCFZIXDIEPN2GKO3CSGX35LZNHI4QDCGYBYAMWBGOA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.451837258 (103h10m50.300444352s) [noise=true , address=77.90.31.17:4771 , piece_id=VCHLVD4PCT6SGJNCFA4LSEDIXSRMX27JH7VLUXDBM2PMIGZ54CNA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.452344092 (103h10m50.300951186s) [noise=true , address=45.91.116.4:4937 , piece_id=DZILXASXCQNDQGTAIYQIY4LCXHENKF6LLWHME5ILDWL3IZOSWMCQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.452773889 (103h10m50.301380982s) [noise=true , address=45.91.116.11:4119 , piece_id=LOMOQU77BVHAPGZ23ZY6CG63FZFQX2IG5QRY2P46WKSGB5AF5O3Q, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.453168858 (103h10m50.301775945s) [noise=true , address=45.91.116.5:4298 , piece_id=AIURH3V65NZD3QI3A5YGBJ222ANC4LTHEYFWTOFIC6SPVH3O4CKA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.453559736 (103h10m50.302166813s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQJDPJRZQEJHTED4UJBNSJ2CGAL6FPTLLIIRFTOVBUK65VSPLGA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.460854171 (103h10m50.309461265s) [noise=true , address=77.90.31.14:4661 , piece_id=MA3CRNAREYTT6ZDO7Q7QRSAOVQQ3EM3CHZTWYQYE6QOWNMYQBBQQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.462111720 (103h10m50.310718803s) [noise=true , address=45.91.116.12:4225 , piece_id=MFM7FUANQ2WU4GK46NPKKKFDUXSVVQ3MGK7GM3QBR7GMOD563PTQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.462819102 (103h10m50.311426189s) [noise=true , address=104.36.82.13:4186 , piece_id=O5BVVJBIWINWK2GXWWJDETED6T3Z462AAUQWCG6LGC7PVVFSHZBA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.463600977 (103h10m50.312208063s) [noise=true , address=45.91.116.7:4011 , piece_id=GKVDRTZ2TY2ESEKU2FKIJND4LXDMITNWQORE4Z47OQ454UZMM7BA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.464356397 (103h10m50.312963485s) [noise=true , address=77.90.31.13:4626 , piece_id=D63E7JGO5RRTEJGXVUMNEC3UYXHS45HBOB7S67VPCWLHJXNH7OSQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.587088382 (103h10m50.435695454s) [upload=82463]: Next returned nil segment
Jul 23 06:01:08.587107670 (103h10m50.435714757s) [upload=82463]: Waiting for error group managing segments...
Jul 23 06:01:08.609523184 (103h10m50.458130271s) [noise=true , address=77.90.31.14:4661 , piece_id=MA3CRNAREYTT6ZDO7Q7QRSAOVQQ3EM3CHZTWYQYE6QOWNMYQBBQQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.612539671 (103h10m50.46114674s) [noise=true , address=45.91.116.5:4299 , piece_id=K3URA4ZO4R2CAGZLTO6YJTZQOZ6YIOB7XOJ3AQAVGDFIMVU63BIA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.617758941 (103h10m50.466366025s) [noise=true , address=45.91.116.13:4155 , piece_id=KMDEG3QMJNHRF7TETZAFVN3TOZWTQNW7IRTCVI3WLNE7NJAAAPCA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.618513849 (103h10m50.467120922s) [noise=true , address=45.91.116.8:4048 , piece_id=AROG4HVUX7R6MHG3DT3N2FIQJBBEVVEVKQHSAGZCWMISWTFSQMAQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.624184177 (103h10m50.47279126s) [noise=true , address=45.91.116.7:4010 , piece_id=WVMOM2PPTVXY7NYI5UH25PFW6RODIUJWN73YB6FW6UOAMNY4C4SA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.625290831 (103h10m50.473897916s) [noise=true , address=45.91.116.7:4011 , piece_id=GKVDRTZ2TY2ESEKU2FKIJND4LXDMITNWQORE4Z47OQ454UZMM7BA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.628897649 (103h10m50.477504746s) [noise=true , address=45.91.116.5:4298 , piece_id=AIURH3V65NZD3QI3A5YGBJ222ANC4LTHEYFWTOFIC6SPVH3O4CKA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.629542622 (103h10m50.478149697s) [noise=true , address=45.91.116.12:4225 , piece_id=MFM7FUANQ2WU4GK46NPKKKFDUXSVVQ3MGK7GM3QBR7GMOD563PTQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.630003986 (103h10m50.478611073s) [noise=true , address=77.90.31.17:4772 , piece_id=63V72ATK3KI3ADRC4N5HTRIXSEB3DRM7N73MVZNDPZJ7EVO3RZJA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.630441959 (103h10m50.479049047s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQJDPJRZQEJHTED4UJBNSJ2CGAL6FPTLLIIRFTOVBUK65VSPLGA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.630941221 (103h10m50.479548305s) [noise=true , address=77.90.31.12:4592 , piece_id=NAWAKPN4LUPCFZIXDIEPN2GKO3CSGX35LZNHI4QDCGYBYAMWBGOA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.631378970 (103h10m50.47998605s) [noise=true , address=77.90.31.13:4626 , piece_id=D63E7JGO5RRTEJGXVUMNEC3UYXHS45HBOB7S67VPCWLHJXNH7OSQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.631840658 (103h10m50.480447733s) [noise=true , address=77.90.31.17:4771 , piece_id=VCHLVD4PCT6SGJNCFA4LSEDIXSRMX27JH7VLUXDBM2PMIGZ54CNA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.631850523 (103h10m50.480457598s) [seg_pos={0 0}, upload=82463]: Segment reached optimal threshold of 13 pieces.
Jul 23 06:01:08.631982956 (103h10m50.480590031s) [noise=true , address=104.36.82.13:4186 , piece_id=O5BVVJBIWINWK2GXWWJDETED6T3Z462AAUQWCG6LGC7PVVFSHZBA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF
Jul 23 06:01:08.632087803 (103h10m50.480694892s) [noise=true , address=45.91.116.4:4937 , piece_id=DZILXASXCQNDQGTAIYQIY4LCXHENKF6LLWHME5ILDWL3IZOSWMCQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF
Jul 23 06:01:08.633101069 (103h10m50.481708157s) [noise=true , address=45.91.116.11:4119 , piece_id=LOMOQU77BVHAPGZ23ZY6CG63FZFQX2IG5QRY2P46WKSGB5AF5O3Q, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.633132335 (103h10m50.481739428s) [seg_pos={0 0}, upload=82463]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 23 06:01:08.633221956 (103h10m50.481829049s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 23 06:01:08.633272252 (103h10m50.481879341s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 23 06:01:08.710605841 (103h10m50.559212933s) [upload=82463]: Done uploading segments...
Jul 23 06:01:08.710632085 (103h10m50.559239157s) [upload=82463]: Upload finished. err: <nil>
Duration: 500.214282ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [................................X...........................] duration:
975.089434ms
FAIL (2025-07-23 06:23:14 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
s) [upload=82448]: Got next segment. Inline: false
Jul 23 05:59:58.732247463 (103h9m40.580854536s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 23 05:59:58.732253833 (103h9m40.580860918s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 23 05:59:58.787057645 (103h9m40.635664731s) [seg_pos={0 0}, upload=82448]: Begin upload segment...
Jul 23 05:59:58.787102519 (103h9m40.635709584s) [seg_pos={0 0}, upload=82448]: Done begin upload segment.
Jul 23 05:59:58.787136403 (103h9m40.635743482s) [noise=true , address=77.90.31.12:4622 , piece_id=EZJAG75JRZTYILS7545KYAI37YOUTHHIOOSH26KIWMUQ4BCG556A, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787169354 (103h9m40.635776438s) [noise=true , address=79.127.205.235:4680 , piece_id=T3V7PMZXJOHWT77MQ52PUMFNAIAX4DLMTIIVL6XNVTNA565O5WCQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787544550 (103h9m40.636151681s) [noise=true , address=104.36.82.10:4105 , piece_id=GY72JPHTZOGEVFDG3FM5W5ZSYOPSY3BZXEME2Y37VVR2LR37SIUA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787598825 (103h9m40.636205915s) [noise=true , address=79.127.205.246:4280 , piece_id=NPBN75MFQYLCG4XKRJPLHJU5JCYBNQYIFQ7MKKB3JKXLT4TKF7AA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787946084 (103h9m40.636553167s) [noise=true , address=77.90.31.17:4799 , piece_id=RUTSDNZEXKJ5Y7WQBIVTMFUIJRYLL6DDNNJEXZAL5S7CIATWYHBQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788041731 (103h9m40.636648823s) [noise=true , address=77.90.31.11:4569 , piece_id=MTNLU22JHOYH6VSRMWLKQGSCDI5GIADPIXBVVSC2RIJLTHPMXTOA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788383218 (103h9m40.636990305s) [noise=true , address=104.36.82.42:4818 , piece_id=A6U7TDLTAZHE5IILHHCZKIHCQINHCJSWTBYNXJV7GLCWTIBNB2VA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788633554 (103h9m40.637240644s) [noise=true , address=79.127.219.51:4202 , piece_id=LJZ5A5HAHG4CGXT5J2WNSMXD7HCYSCBUEYGAGOES3NWIVV34BXEQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788735479 (103h9m40.637342568s) [noise=true , address=79.127.219.47:4040 , piece_id=BKWXBAVFXQ2RKG27WGPEAWG5UTHBVODW2BB4EGKHV6SN4EFI7KIQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789072444 (103h9m40.637679533s) [noise=true , address=77.90.31.15:4706 , piece_id=C3JOUPYB77PA2EHPSNPMDETERE724PBALBCSZ6MJCTZ6YRPUC2UA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789122202 (103h9m40.637729291s) [noise=true , address=104.36.82.9:4035 , piece_id=74BSWBLVWYQXPUI2PGV5NIJDSE6MSFKWK65TCVV6CNZKVQBQVNEA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789393961 (103h9m40.638001047s) [noise=true , address=45.91.116.10:4103 , piece_id=UGRDLKKFJRNCCDNDQTAY2LLOIS7CGI6KLNDZ7KJDOUNYWFNUAYQQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789591397 (103h9m40.638198479s) [noise=true , address=104.36.82.5:4870 , piece_id=AYTLB2OL7OAI5EO2ZE6LXIQLCGSGDXKDKT6V5HXIPAREK656MDPQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789716087 (103h9m40.638323176s) [noise=true , address=79.127.205.233:4645 , piece_id=6YAS5E7QYEZAJXB6HUY3R4PLHC4Z7EYTMSFWNFWD5AP7HKXOKTUQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789937996 (103h9m40.638545078s) [noise=true , address=104.36.82.53:4121 , piece_id=GJLXLW2A5RZUF4MBUP5AZAWFKTWGBDADIIWVIUBXPQPPGPKYZ4YA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.790288745 (103h9m40.638895835s) [noise=true , address=77.90.31.15:4708 , piece_id=67LX4NZJUKLYHRTSW3RKHHXZRKMGWJ2KV32FDA2KE3DY364CJ7OA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.790730467 (103h9m40.639337554s) [noise=true , address=45.91.116.14:4206 , piece_id=OVX63SL5OGJTT7HWDRIUV4BDVEKN2MD4ORUUK5M4JIEWX4J6VHFA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.791174576 (103h9m40.639781656s) [noise=true , address=45.91.116.13:4176 , piece_id=GOWXIBOORGZWFIBLMG46HQ4J632B3E6I3MVXNLTWPP2TY3IF2JUQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.898860128 (103h9m40.747467214s) [upload=82448]: Next returned nil segment
Jul 23 05:59:58.898881813 (103h9m40.747488906s) [upload=82448]: Waiting for error group managing segments...
Jul 23 05:59:58.905078785 (103h9m40.753685854s) [noise=true , address=45.91.116.14:4206 , piece_id=OVX63SL5OGJTT7HWDRIUV4BDVEKN2MD4ORUUK5M4JIEWX4J6VHFA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.906441982 (103h9m40.755049063s) [noise=true , address=77.90.31.15:4708 , piece_id=67LX4NZJUKLYHRTSW3RKHHXZRKMGWJ2KV32FDA2KE3DY364CJ7OA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.910115252 (103h9m40.758722337s) [noise=true , address=77.90.31.15:4706 , piece_id=C3JOUPYB77PA2EHPSNPMDETERE724PBALBCSZ6MJCTZ6YRPUC2UA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.910652695 (103h9m40.759259762s) [noise=true , address=104.36.82.9:4035 , piece_id=74BSWBLVWYQXPUI2PGV5NIJDSE6MSFKWK65TCVV6CNZKVQBQVNEA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.910672036 (103h9m40.759279117s) [noise=true , address=77.90.31.11:4569 , piece_id=MTNLU22JHOYH6VSRMWLKQGSCDI5GIADPIXBVVSC2RIJLTHPMXTOA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.911483080 (103h9m40.760090146s) [noise=true , address=45.91.116.13:4176 , piece_id=GOWXIBOORGZWFIBLMG46HQ4J632B3E6I3MVXNLTWPP2TY3IF2JUQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.911951970 (103h9m40.760559052s) [noise=true , address=45.91.116.10:4103 , piece_id=UGRDLKKFJRNCCDNDQTAY2LLOIS7CGI6KLNDZ7KJDOUNYWFNUAYQQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.912044937 (103h9m40.760652006s) [noise=true , address=77.90.31.12:4622 , piece_id=EZJAG75JRZTYILS7545KYAI37YOUTHHIOOSH26KIWMUQ4BCG556A, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.912649865 (103h9m40.761256931s) [noise=true , address=77.90.31.17:4799 , piece_id=RUTSDNZEXKJ5Y7WQBIVTMFUIJRYLL6DDNNJEXZAL5S7CIATWYHBQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.913487977 (103h9m40.76209505s) [noise=true , address=79.127.219.47:4040 , piece_id=BKWXBAVFXQ2RKG27WGPEAWG5UTHBVODW2BB4EGKHV6SN4EFI7KIQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.914486860 (103h9m40.763093927s) [noise=true , address=104.36.82.53:4121 , piece_id=GJLXLW2A5RZUF4MBUP5AZAWFKTWGBDADIIWVIUBXPQPPGPKYZ4YA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.918080770 (103h9m40.766687852s) [noise=true , address=104.36.82.5:4870 , piece_id=AYTLB2OL7OAI5EO2ZE6LXIQLCGSGDXKDKT6V5HXIPAREK656MDPQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.924012392 (103h9m40.772619461s) [noise=true , address=79.127.219.51:4202 , piece_id=LJZ5A5HAHG4CGXT5J2WNSMXD7HCYSCBUEYGAGOES3NWIVV34BXEQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.932892483 (103h9m40.78149955s) [noise=true , address=104.36.82.42:4818 , piece_id=A6U7TDLTAZHE5IILHHCZKIHCQINHCJSWTBYNXJV7GLCWTIBNB2VA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.932913099 (103h9m40.781520167s) [seg_pos={0 0}, upload=82448]: Segment reached optimal threshold of 14 pieces.
Jul 23 05:59:58.933104957 (103h9m40.781712037s) [noise=true , address=104.36.82.10:4105 , piece_id=GY72JPHTZOGEVFDG3FM5W5ZSYOPSY3BZXEME2Y37VVR2LR37SIUA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GhDrKMcDEg2RxamqKtiHphpgNdvtZYdHXrjzGsNJFjBFYeEen): protocol: expected piece hash; context canceled; EOF
Jul 23 05:59:58.933505265 (103h9m40.782112345s) [noise=true , address=79.127.205.233:4645 , piece_id=6YAS5E7QYEZAJXB6HUY3R4PLHC4Z7EYTMSFWNFWD5AP7HKXOKTUQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12NR52pweTqZNNMTyc1EHSbifDKB4awrnnb4megtZHwANSrR18T): protocol: read tcp 159.223.143.39:46566->79.127.205.233:4645: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 23 05:59:58.933568113 (103h9m40.782175194s) [noise=true , address=79.127.205.235:4680 , piece_id=T3V7PMZXJOHWT77MQ52PUMFNAIAX4DLMTIIVL6XNVTNA565O5WCQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GGvQQLoBcTniZST8Hz9h8eCLWbU4DrXn66DGy33iGLXaFgJLj): protocol: read tcp 159.223.143.39:45394->79.127.205.235:4680: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 23 05:59:58.933618915 (103h9m40.782225986s) [noise=true , address=79.127.205.246:4280 , piece_id=NPBN75MFQYLCG4XKRJPLHJU5JCYBNQYIFQ7MKKB3JKXLT4TKF7AA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rZDBvhrfNrYQB6h9MGhx1tNtQ7AH9ehUF4SfNMCJRpv4SKrDX): protocol: read tcp 159.223.143.39:45458->79.127.205.246:4280: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 23 05:59:58.933631191 (103h9m40.78223826s) [seg_pos={0 0}, upload=82448]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jul 23 05:59:58.933756265 (103h9m40.782363345s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 23 05:59:58.933762566 (103h9m40.782369645s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 23 05:59:59.025408726 (103h9m40.874015808s) [upload=82448]: Done uploading segments...
Jul 23 05:59:59.025427547 (103h9m40.874034627s) [upload=82448]: Upload finished. err: <nil>
Duration: 975.117213ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............................X.............................] duration:
487.922262ms
FAIL (2025-07-23 06:22:40 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
23 06:00:19.742341858 (103h10m1.590948949s) [noise=true , address=104.36.82.4:4804 , piece_id=SMCQKI5GIWKEHYLNSVUGHWQNBHP3S4C5RBB4E3K5CFMXE54Q4WAQ, seg_pos={0 0}, upload=82455]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g): protocol: expected piece hash; context canceled; EOF
Jul 23 06:00:19.742959346 (103h10m1.59156642s) [noise=true , address=77.90.31.16:4734 , piece_id=NFCPPB6P25YMO5UZRM4CQKMNRV5Q2FXMSR6M4VSTEERBMVNLYJJA, seg_pos={0 0}, upload=82455]: Done uploading piece. err: <nil>
Jul 23 06:00:19.743002232 (103h10m1.591609306s) [seg_pos={0 0}, upload=82455]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 23 06:00:19.743286680 (103h10m1.591893764s) [upload=82455]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 23 06:00:19.743293769 (103h10m1.591900841s) [upload=82455]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 23 06:00:19.815641707 (103h10m1.664248798s) [upload=82455]: Done uploading segments...
Jul 23 06:00:19.815666887 (103h10m1.664273954s) [upload=82455]: Upload finished. err: <nil>
Jul 23 06:01:08.401614022 (103h10m50.250221093s) [upload=82463]: Starting upload
Jul 23 06:01:08.401709823 (103h10m50.250316908s) [upload=82463]: Done starting upload
Jul 23 06:01:08.402138436 (103h10m50.250745505s) [upload=82463]: Uploading segments...
Jul 23 06:01:08.402189044 (103h10m50.250796137s) [upload=82463]: Got next segment. Inline: false
Jul 23 06:01:08.402231416 (103h10m50.250838487s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 23 06:01:08.402237731 (103h10m50.250844804s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 23 06:01:08.450301042 (103h10m50.298908146s) [seg_pos={0 0}, upload=82463]: Begin upload segment...
Jul 23 06:01:08.450364720 (103h10m50.298971798s) [seg_pos={0 0}, upload=82463]: Done begin upload segment.
Jul 23 06:01:08.450425015 (103h10m50.2990321s) [noise=true , address=45.91.116.5:4299 , piece_id=K3URA4ZO4R2CAGZLTO6YJTZQOZ6YIOB7XOJ3AQAVGDFIMVU63BIA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.450431954 (103h10m50.299039031s) [noise=true , address=45.91.116.7:4010 , piece_id=WVMOM2PPTVXY7NYI5UH25PFW6RODIUJWN73YB6FW6UOAMNY4C4SA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.450900413 (103h10m50.299507502s) [noise=true , address=77.90.31.17:4772 , piece_id=63V72ATK3KI3ADRC4N5HTRIXSEB3DRM7N73MVZNDPZJ7EVO3RZJA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.450963993 (103h10m50.299571083s) [noise=true , address=45.91.116.8:4048 , piece_id=AROG4HVUX7R6MHG3DT3N2FIQJBBEVVEVKQHSAGZCWMISWTFSQMAQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.451375091 (103h10m50.299982178s) [noise=true , address=45.91.116.13:4155 , piece_id=KMDEG3QMJNHRF7TETZAFVN3TOZWTQNW7IRTCVI3WLNE7NJAAAPCA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.451400410 (103h10m50.300007509s) [noise=true , address=77.90.31.12:4592 , piece_id=NAWAKPN4LUPCFZIXDIEPN2GKO3CSGX35LZNHI4QDCGYBYAMWBGOA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.451837258 (103h10m50.300444352s) [noise=true , address=77.90.31.17:4771 , piece_id=VCHLVD4PCT6SGJNCFA4LSEDIXSRMX27JH7VLUXDBM2PMIGZ54CNA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.452344092 (103h10m50.300951186s) [noise=true , address=45.91.116.4:4937 , piece_id=DZILXASXCQNDQGTAIYQIY4LCXHENKF6LLWHME5ILDWL3IZOSWMCQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.452773889 (103h10m50.301380982s) [noise=true , address=45.91.116.11:4119 , piece_id=LOMOQU77BVHAPGZ23ZY6CG63FZFQX2IG5QRY2P46WKSGB5AF5O3Q, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.453168858 (103h10m50.301775945s) [noise=true , address=45.91.116.5:4298 , piece_id=AIURH3V65NZD3QI3A5YGBJ222ANC4LTHEYFWTOFIC6SPVH3O4CKA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.453559736 (103h10m50.302166813s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQJDPJRZQEJHTED4UJBNSJ2CGAL6FPTLLIIRFTOVBUK65VSPLGA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.460854171 (103h10m50.309461265s) [noise=true , address=77.90.31.14:4661 , piece_id=MA3CRNAREYTT6ZDO7Q7QRSAOVQQ3EM3CHZTWYQYE6QOWNMYQBBQQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.462111720 (103h10m50.310718803s) [noise=true , address=45.91.116.12:4225 , piece_id=MFM7FUANQ2WU4GK46NPKKKFDUXSVVQ3MGK7GM3QBR7GMOD563PTQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.462819102 (103h10m50.311426189s) [noise=true , address=104.36.82.13:4186 , piece_id=O5BVVJBIWINWK2GXWWJDETED6T3Z462AAUQWCG6LGC7PVVFSHZBA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.463600977 (103h10m50.312208063s) [noise=true , address=45.91.116.7:4011 , piece_id=GKVDRTZ2TY2ESEKU2FKIJND4LXDMITNWQORE4Z47OQ454UZMM7BA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.464356397 (103h10m50.312963485s) [noise=true , address=77.90.31.13:4626 , piece_id=D63E7JGO5RRTEJGXVUMNEC3UYXHS45HBOB7S67VPCWLHJXNH7OSQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.587088382 (103h10m50.435695454s) [upload=82463]: Next returned nil segment
Jul 23 06:01:08.587107670 (103h10m50.435714757s) [upload=82463]: Waiting for error group managing segments...
Jul 23 06:01:08.609523184 (103h10m50.458130271s) [noise=true , address=77.90.31.14:4661 , piece_id=MA3CRNAREYTT6ZDO7Q7QRSAOVQQ3EM3CHZTWYQYE6QOWNMYQBBQQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.612539671 (103h10m50.46114674s) [noise=true , address=45.91.116.5:4299 , piece_id=K3URA4ZO4R2CAGZLTO6YJTZQOZ6YIOB7XOJ3AQAVGDFIMVU63BIA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.617758941 (103h10m50.466366025s) [noise=true , address=45.91.116.13:4155 , piece_id=KMDEG3QMJNHRF7TETZAFVN3TOZWTQNW7IRTCVI3WLNE7NJAAAPCA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.618513849 (103h10m50.467120922s) [noise=true , address=45.91.116.8:4048 , piece_id=AROG4HVUX7R6MHG3DT3N2FIQJBBEVVEVKQHSAGZCWMISWTFSQMAQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.624184177 (103h10m50.47279126s) [noise=true , address=45.91.116.7:4010 , piece_id=WVMOM2PPTVXY7NYI5UH25PFW6RODIUJWN73YB6FW6UOAMNY4C4SA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.625290831 (103h10m50.473897916s) [noise=true , address=45.91.116.7:4011 , piece_id=GKVDRTZ2TY2ESEKU2FKIJND4LXDMITNWQORE4Z47OQ454UZMM7BA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.628897649 (103h10m50.477504746s) [noise=true , address=45.91.116.5:4298 , piece_id=AIURH3V65NZD3QI3A5YGBJ222ANC4LTHEYFWTOFIC6SPVH3O4CKA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.629542622 (103h10m50.478149697s) [noise=true , address=45.91.116.12:4225 , piece_id=MFM7FUANQ2WU4GK46NPKKKFDUXSVVQ3MGK7GM3QBR7GMOD563PTQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.630003986 (103h10m50.478611073s) [noise=true , address=77.90.31.17:4772 , piece_id=63V72ATK3KI3ADRC4N5HTRIXSEB3DRM7N73MVZNDPZJ7EVO3RZJA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.630441959 (103h10m50.479049047s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQJDPJRZQEJHTED4UJBNSJ2CGAL6FPTLLIIRFTOVBUK65VSPLGA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.630941221 (103h10m50.479548305s) [noise=true , address=77.90.31.12:4592 , piece_id=NAWAKPN4LUPCFZIXDIEPN2GKO3CSGX35LZNHI4QDCGYBYAMWBGOA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.631378970 (103h10m50.47998605s) [noise=true , address=77.90.31.13:4626 , piece_id=D63E7JGO5RRTEJGXVUMNEC3UYXHS45HBOB7S67VPCWLHJXNH7OSQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.631840658 (103h10m50.480447733s) [noise=true , address=77.90.31.17:4771 , piece_id=VCHLVD4PCT6SGJNCFA4LSEDIXSRMX27JH7VLUXDBM2PMIGZ54CNA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.631850523 (103h10m50.480457598s) [seg_pos={0 0}, upload=82463]: Segment reached optimal threshold of 13 pieces.
Jul 23 06:01:08.631982956 (103h10m50.480590031s) [noise=true , address=104.36.82.13:4186 , piece_id=O5BVVJBIWINWK2GXWWJDETED6T3Z462AAUQWCG6LGC7PVVFSHZBA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF
Jul 23 06:01:08.632087803 (103h10m50.480694892s) [noise=true , address=45.91.116.4:4937 , piece_id=DZILXASXCQNDQGTAIYQIY4LCXHENKF6LLWHME5ILDWL3IZOSWMCQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF
Jul 23 06:01:08.633101069 (103h10m50.481708157s) [noise=true , address=45.91.116.11:4119 , piece_id=LOMOQU77BVHAPGZ23ZY6CG63FZFQX2IG5QRY2P46WKSGB5AF5O3Q, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.633132335 (103h10m50.481739428s) [seg_pos={0 0}, upload=82463]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 23 06:01:08.633221956 (103h10m50.481829049s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 23 06:01:08.633272252 (103h10m50.481879341s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 23 06:01:08.710605841 (103h10m50.559212933s) [upload=82463]: Done uploading segments...
Jul 23 06:01:08.710632085 (103h10m50.559239157s) [upload=82463]: Upload finished. err: <nil>
Duration: 487.953097ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.................................X..........................] duration:
788.02788ms
FAIL (2025-07-23 06:22:16 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
s) [upload=82448]: Got next segment. Inline: false
Jul 23 05:59:58.732247463 (103h9m40.580854536s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 23 05:59:58.732253833 (103h9m40.580860918s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 23 05:59:58.787057645 (103h9m40.635664731s) [seg_pos={0 0}, upload=82448]: Begin upload segment...
Jul 23 05:59:58.787102519 (103h9m40.635709584s) [seg_pos={0 0}, upload=82448]: Done begin upload segment.
Jul 23 05:59:58.787136403 (103h9m40.635743482s) [noise=true , address=77.90.31.12:4622 , piece_id=EZJAG75JRZTYILS7545KYAI37YOUTHHIOOSH26KIWMUQ4BCG556A, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787169354 (103h9m40.635776438s) [noise=true , address=79.127.205.235:4680 , piece_id=T3V7PMZXJOHWT77MQ52PUMFNAIAX4DLMTIIVL6XNVTNA565O5WCQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787544550 (103h9m40.636151681s) [noise=true , address=104.36.82.10:4105 , piece_id=GY72JPHTZOGEVFDG3FM5W5ZSYOPSY3BZXEME2Y37VVR2LR37SIUA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787598825 (103h9m40.636205915s) [noise=true , address=79.127.205.246:4280 , piece_id=NPBN75MFQYLCG4XKRJPLHJU5JCYBNQYIFQ7MKKB3JKXLT4TKF7AA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787946084 (103h9m40.636553167s) [noise=true , address=77.90.31.17:4799 , piece_id=RUTSDNZEXKJ5Y7WQBIVTMFUIJRYLL6DDNNJEXZAL5S7CIATWYHBQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788041731 (103h9m40.636648823s) [noise=true , address=77.90.31.11:4569 , piece_id=MTNLU22JHOYH6VSRMWLKQGSCDI5GIADPIXBVVSC2RIJLTHPMXTOA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788383218 (103h9m40.636990305s) [noise=true , address=104.36.82.42:4818 , piece_id=A6U7TDLTAZHE5IILHHCZKIHCQINHCJSWTBYNXJV7GLCWTIBNB2VA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788633554 (103h9m40.637240644s) [noise=true , address=79.127.219.51:4202 , piece_id=LJZ5A5HAHG4CGXT5J2WNSMXD7HCYSCBUEYGAGOES3NWIVV34BXEQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788735479 (103h9m40.637342568s) [noise=true , address=79.127.219.47:4040 , piece_id=BKWXBAVFXQ2RKG27WGPEAWG5UTHBVODW2BB4EGKHV6SN4EFI7KIQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789072444 (103h9m40.637679533s) [noise=true , address=77.90.31.15:4706 , piece_id=C3JOUPYB77PA2EHPSNPMDETERE724PBALBCSZ6MJCTZ6YRPUC2UA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789122202 (103h9m40.637729291s) [noise=true , address=104.36.82.9:4035 , piece_id=74BSWBLVWYQXPUI2PGV5NIJDSE6MSFKWK65TCVV6CNZKVQBQVNEA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789393961 (103h9m40.638001047s) [noise=true , address=45.91.116.10:4103 , piece_id=UGRDLKKFJRNCCDNDQTAY2LLOIS7CGI6KLNDZ7KJDOUNYWFNUAYQQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789591397 (103h9m40.638198479s) [noise=true , address=104.36.82.5:4870 , piece_id=AYTLB2OL7OAI5EO2ZE6LXIQLCGSGDXKDKT6V5HXIPAREK656MDPQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789716087 (103h9m40.638323176s) [noise=true , address=79.127.205.233:4645 , piece_id=6YAS5E7QYEZAJXB6HUY3R4PLHC4Z7EYTMSFWNFWD5AP7HKXOKTUQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789937996 (103h9m40.638545078s) [noise=true , address=104.36.82.53:4121 , piece_id=GJLXLW2A5RZUF4MBUP5AZAWFKTWGBDADIIWVIUBXPQPPGPKYZ4YA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.790288745 (103h9m40.638895835s) [noise=true , address=77.90.31.15:4708 , piece_id=67LX4NZJUKLYHRTSW3RKHHXZRKMGWJ2KV32FDA2KE3DY364CJ7OA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.790730467 (103h9m40.639337554s) [noise=true , address=45.91.116.14:4206 , piece_id=OVX63SL5OGJTT7HWDRIUV4BDVEKN2MD4ORUUK5M4JIEWX4J6VHFA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.791174576 (103h9m40.639781656s) [noise=true , address=45.91.116.13:4176 , piece_id=GOWXIBOORGZWFIBLMG46HQ4J632B3E6I3MVXNLTWPP2TY3IF2JUQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.898860128 (103h9m40.747467214s) [upload=82448]: Next returned nil segment
Jul 23 05:59:58.898881813 (103h9m40.747488906s) [upload=82448]: Waiting for error group managing segments...
Jul 23 05:59:58.905078785 (103h9m40.753685854s) [noise=true , address=45.91.116.14:4206 , piece_id=OVX63SL5OGJTT7HWDRIUV4BDVEKN2MD4ORUUK5M4JIEWX4J6VHFA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.906441982 (103h9m40.755049063s) [noise=true , address=77.90.31.15:4708 , piece_id=67LX4NZJUKLYHRTSW3RKHHXZRKMGWJ2KV32FDA2KE3DY364CJ7OA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.910115252 (103h9m40.758722337s) [noise=true , address=77.90.31.15:4706 , piece_id=C3JOUPYB77PA2EHPSNPMDETERE724PBALBCSZ6MJCTZ6YRPUC2UA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.910652695 (103h9m40.759259762s) [noise=true , address=104.36.82.9:4035 , piece_id=74BSWBLVWYQXPUI2PGV5NIJDSE6MSFKWK65TCVV6CNZKVQBQVNEA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.910672036 (103h9m40.759279117s) [noise=true , address=77.90.31.11:4569 , piece_id=MTNLU22JHOYH6VSRMWLKQGSCDI5GIADPIXBVVSC2RIJLTHPMXTOA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.911483080 (103h9m40.760090146s) [noise=true , address=45.91.116.13:4176 , piece_id=GOWXIBOORGZWFIBLMG46HQ4J632B3E6I3MVXNLTWPP2TY3IF2JUQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.911951970 (103h9m40.760559052s) [noise=true , address=45.91.116.10:4103 , piece_id=UGRDLKKFJRNCCDNDQTAY2LLOIS7CGI6KLNDZ7KJDOUNYWFNUAYQQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.912044937 (103h9m40.760652006s) [noise=true , address=77.90.31.12:4622 , piece_id=EZJAG75JRZTYILS7545KYAI37YOUTHHIOOSH26KIWMUQ4BCG556A, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.912649865 (103h9m40.761256931s) [noise=true , address=77.90.31.17:4799 , piece_id=RUTSDNZEXKJ5Y7WQBIVTMFUIJRYLL6DDNNJEXZAL5S7CIATWYHBQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.913487977 (103h9m40.76209505s) [noise=true , address=79.127.219.47:4040 , piece_id=BKWXBAVFXQ2RKG27WGPEAWG5UTHBVODW2BB4EGKHV6SN4EFI7KIQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.914486860 (103h9m40.763093927s) [noise=true , address=104.36.82.53:4121 , piece_id=GJLXLW2A5RZUF4MBUP5AZAWFKTWGBDADIIWVIUBXPQPPGPKYZ4YA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.918080770 (103h9m40.766687852s) [noise=true , address=104.36.82.5:4870 , piece_id=AYTLB2OL7OAI5EO2ZE6LXIQLCGSGDXKDKT6V5HXIPAREK656MDPQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.924012392 (103h9m40.772619461s) [noise=true , address=79.127.219.51:4202 , piece_id=LJZ5A5HAHG4CGXT5J2WNSMXD7HCYSCBUEYGAGOES3NWIVV34BXEQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.932892483 (103h9m40.78149955s) [noise=true , address=104.36.82.42:4818 , piece_id=A6U7TDLTAZHE5IILHHCZKIHCQINHCJSWTBYNXJV7GLCWTIBNB2VA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.932913099 (103h9m40.781520167s) [seg_pos={0 0}, upload=82448]: Segment reached optimal threshold of 14 pieces.
Jul 23 05:59:58.933104957 (103h9m40.781712037s) [noise=true , address=104.36.82.10:4105 , piece_id=GY72JPHTZOGEVFDG3FM5W5ZSYOPSY3BZXEME2Y37VVR2LR37SIUA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GhDrKMcDEg2RxamqKtiHphpgNdvtZYdHXrjzGsNJFjBFYeEen): protocol: expected piece hash; context canceled; EOF
Jul 23 05:59:58.933505265 (103h9m40.782112345s) [noise=true , address=79.127.205.233:4645 , piece_id=6YAS5E7QYEZAJXB6HUY3R4PLHC4Z7EYTMSFWNFWD5AP7HKXOKTUQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12NR52pweTqZNNMTyc1EHSbifDKB4awrnnb4megtZHwANSrR18T): protocol: read tcp 159.223.143.39:46566->79.127.205.233:4645: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 23 05:59:58.933568113 (103h9m40.782175194s) [noise=true , address=79.127.205.235:4680 , piece_id=T3V7PMZXJOHWT77MQ52PUMFNAIAX4DLMTIIVL6XNVTNA565O5WCQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GGvQQLoBcTniZST8Hz9h8eCLWbU4DrXn66DGy33iGLXaFgJLj): protocol: read tcp 159.223.143.39:45394->79.127.205.235:4680: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 23 05:59:58.933618915 (103h9m40.782225986s) [noise=true , address=79.127.205.246:4280 , piece_id=NPBN75MFQYLCG4XKRJPLHJU5JCYBNQYIFQ7MKKB3JKXLT4TKF7AA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rZDBvhrfNrYQB6h9MGhx1tNtQ7AH9ehUF4SfNMCJRpv4SKrDX): protocol: read tcp 159.223.143.39:45458->79.127.205.246:4280: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 23 05:59:58.933631191 (103h9m40.78223826s) [seg_pos={0 0}, upload=82448]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jul 23 05:59:58.933756265 (103h9m40.782363345s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 23 05:59:58.933762566 (103h9m40.782369645s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 23 05:59:59.025408726 (103h9m40.874015808s) [upload=82448]: Done uploading segments...
Jul 23 05:59:59.025427547 (103h9m40.874034627s) [upload=82448]: Upload finished. err: <nil>
Duration: 788.065444ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [..................................X.........................] duration:
979.023331ms
FAIL (2025-07-23 06:21:51 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
s) [upload=82448]: Got next segment. Inline: false
Jul 23 05:59:58.732247463 (103h9m40.580854536s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 23 05:59:58.732253833 (103h9m40.580860918s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 23 05:59:58.787057645 (103h9m40.635664731s) [seg_pos={0 0}, upload=82448]: Begin upload segment...
Jul 23 05:59:58.787102519 (103h9m40.635709584s) [seg_pos={0 0}, upload=82448]: Done begin upload segment.
Jul 23 05:59:58.787136403 (103h9m40.635743482s) [noise=true , address=77.90.31.12:4622 , piece_id=EZJAG75JRZTYILS7545KYAI37YOUTHHIOOSH26KIWMUQ4BCG556A, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787169354 (103h9m40.635776438s) [noise=true , address=79.127.205.235:4680 , piece_id=T3V7PMZXJOHWT77MQ52PUMFNAIAX4DLMTIIVL6XNVTNA565O5WCQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787544550 (103h9m40.636151681s) [noise=true , address=104.36.82.10:4105 , piece_id=GY72JPHTZOGEVFDG3FM5W5ZSYOPSY3BZXEME2Y37VVR2LR37SIUA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787598825 (103h9m40.636205915s) [noise=true , address=79.127.205.246:4280 , piece_id=NPBN75MFQYLCG4XKRJPLHJU5JCYBNQYIFQ7MKKB3JKXLT4TKF7AA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787946084 (103h9m40.636553167s) [noise=true , address=77.90.31.17:4799 , piece_id=RUTSDNZEXKJ5Y7WQBIVTMFUIJRYLL6DDNNJEXZAL5S7CIATWYHBQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788041731 (103h9m40.636648823s) [noise=true , address=77.90.31.11:4569 , piece_id=MTNLU22JHOYH6VSRMWLKQGSCDI5GIADPIXBVVSC2RIJLTHPMXTOA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788383218 (103h9m40.636990305s) [noise=true , address=104.36.82.42:4818 , piece_id=A6U7TDLTAZHE5IILHHCZKIHCQINHCJSWTBYNXJV7GLCWTIBNB2VA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788633554 (103h9m40.637240644s) [noise=true , address=79.127.219.51:4202 , piece_id=LJZ5A5HAHG4CGXT5J2WNSMXD7HCYSCBUEYGAGOES3NWIVV34BXEQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788735479 (103h9m40.637342568s) [noise=true , address=79.127.219.47:4040 , piece_id=BKWXBAVFXQ2RKG27WGPEAWG5UTHBVODW2BB4EGKHV6SN4EFI7KIQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789072444 (103h9m40.637679533s) [noise=true , address=77.90.31.15:4706 , piece_id=C3JOUPYB77PA2EHPSNPMDETERE724PBALBCSZ6MJCTZ6YRPUC2UA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789122202 (103h9m40.637729291s) [noise=true , address=104.36.82.9:4035 , piece_id=74BSWBLVWYQXPUI2PGV5NIJDSE6MSFKWK65TCVV6CNZKVQBQVNEA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789393961 (103h9m40.638001047s) [noise=true , address=45.91.116.10:4103 , piece_id=UGRDLKKFJRNCCDNDQTAY2LLOIS7CGI6KLNDZ7KJDOUNYWFNUAYQQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789591397 (103h9m40.638198479s) [noise=true , address=104.36.82.5:4870 , piece_id=AYTLB2OL7OAI5EO2ZE6LXIQLCGSGDXKDKT6V5HXIPAREK656MDPQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789716087 (103h9m40.638323176s) [noise=true , address=79.127.205.233:4645 , piece_id=6YAS5E7QYEZAJXB6HUY3R4PLHC4Z7EYTMSFWNFWD5AP7HKXOKTUQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789937996 (103h9m40.638545078s) [noise=true , address=104.36.82.53:4121 , piece_id=GJLXLW2A5RZUF4MBUP5AZAWFKTWGBDADIIWVIUBXPQPPGPKYZ4YA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.790288745 (103h9m40.638895835s) [noise=true , address=77.90.31.15:4708 , piece_id=67LX4NZJUKLYHRTSW3RKHHXZRKMGWJ2KV32FDA2KE3DY364CJ7OA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.790730467 (103h9m40.639337554s) [noise=true , address=45.91.116.14:4206 , piece_id=OVX63SL5OGJTT7HWDRIUV4BDVEKN2MD4ORUUK5M4JIEWX4J6VHFA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.791174576 (103h9m40.639781656s) [noise=true , address=45.91.116.13:4176 , piece_id=GOWXIBOORGZWFIBLMG46HQ4J632B3E6I3MVXNLTWPP2TY3IF2JUQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.898860128 (103h9m40.747467214s) [upload=82448]: Next returned nil segment
Jul 23 05:59:58.898881813 (103h9m40.747488906s) [upload=82448]: Waiting for error group managing segments...
Jul 23 05:59:58.905078785 (103h9m40.753685854s) [noise=true , address=45.91.116.14:4206 , piece_id=OVX63SL5OGJTT7HWDRIUV4BDVEKN2MD4ORUUK5M4JIEWX4J6VHFA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.906441982 (103h9m40.755049063s) [noise=true , address=77.90.31.15:4708 , piece_id=67LX4NZJUKLYHRTSW3RKHHXZRKMGWJ2KV32FDA2KE3DY364CJ7OA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.910115252 (103h9m40.758722337s) [noise=true , address=77.90.31.15:4706 , piece_id=C3JOUPYB77PA2EHPSNPMDETERE724PBALBCSZ6MJCTZ6YRPUC2UA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.910652695 (103h9m40.759259762s) [noise=true , address=104.36.82.9:4035 , piece_id=74BSWBLVWYQXPUI2PGV5NIJDSE6MSFKWK65TCVV6CNZKVQBQVNEA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.910672036 (103h9m40.759279117s) [noise=true , address=77.90.31.11:4569 , piece_id=MTNLU22JHOYH6VSRMWLKQGSCDI5GIADPIXBVVSC2RIJLTHPMXTOA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.911483080 (103h9m40.760090146s) [noise=true , address=45.91.116.13:4176 , piece_id=GOWXIBOORGZWFIBLMG46HQ4J632B3E6I3MVXNLTWPP2TY3IF2JUQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.911951970 (103h9m40.760559052s) [noise=true , address=45.91.116.10:4103 , piece_id=UGRDLKKFJRNCCDNDQTAY2LLOIS7CGI6KLNDZ7KJDOUNYWFNUAYQQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.912044937 (103h9m40.760652006s) [noise=true , address=77.90.31.12:4622 , piece_id=EZJAG75JRZTYILS7545KYAI37YOUTHHIOOSH26KIWMUQ4BCG556A, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.912649865 (103h9m40.761256931s) [noise=true , address=77.90.31.17:4799 , piece_id=RUTSDNZEXKJ5Y7WQBIVTMFUIJRYLL6DDNNJEXZAL5S7CIATWYHBQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.913487977 (103h9m40.76209505s) [noise=true , address=79.127.219.47:4040 , piece_id=BKWXBAVFXQ2RKG27WGPEAWG5UTHBVODW2BB4EGKHV6SN4EFI7KIQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.914486860 (103h9m40.763093927s) [noise=true , address=104.36.82.53:4121 , piece_id=GJLXLW2A5RZUF4MBUP5AZAWFKTWGBDADIIWVIUBXPQPPGPKYZ4YA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.918080770 (103h9m40.766687852s) [noise=true , address=104.36.82.5:4870 , piece_id=AYTLB2OL7OAI5EO2ZE6LXIQLCGSGDXKDKT6V5HXIPAREK656MDPQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.924012392 (103h9m40.772619461s) [noise=true , address=79.127.219.51:4202 , piece_id=LJZ5A5HAHG4CGXT5J2WNSMXD7HCYSCBUEYGAGOES3NWIVV34BXEQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.932892483 (103h9m40.78149955s) [noise=true , address=104.36.82.42:4818 , piece_id=A6U7TDLTAZHE5IILHHCZKIHCQINHCJSWTBYNXJV7GLCWTIBNB2VA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.932913099 (103h9m40.781520167s) [seg_pos={0 0}, upload=82448]: Segment reached optimal threshold of 14 pieces.
Jul 23 05:59:58.933104957 (103h9m40.781712037s) [noise=true , address=104.36.82.10:4105 , piece_id=GY72JPHTZOGEVFDG3FM5W5ZSYOPSY3BZXEME2Y37VVR2LR37SIUA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GhDrKMcDEg2RxamqKtiHphpgNdvtZYdHXrjzGsNJFjBFYeEen): protocol: expected piece hash; context canceled; EOF
Jul 23 05:59:58.933505265 (103h9m40.782112345s) [noise=true , address=79.127.205.233:4645 , piece_id=6YAS5E7QYEZAJXB6HUY3R4PLHC4Z7EYTMSFWNFWD5AP7HKXOKTUQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12NR52pweTqZNNMTyc1EHSbifDKB4awrnnb4megtZHwANSrR18T): protocol: read tcp 159.223.143.39:46566->79.127.205.233:4645: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 23 05:59:58.933568113 (103h9m40.782175194s) [noise=true , address=79.127.205.235:4680 , piece_id=T3V7PMZXJOHWT77MQ52PUMFNAIAX4DLMTIIVL6XNVTNA565O5WCQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GGvQQLoBcTniZST8Hz9h8eCLWbU4DrXn66DGy33iGLXaFgJLj): protocol: read tcp 159.223.143.39:45394->79.127.205.235:4680: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 23 05:59:58.933618915 (103h9m40.782225986s) [noise=true , address=79.127.205.246:4280 , piece_id=NPBN75MFQYLCG4XKRJPLHJU5JCYBNQYIFQ7MKKB3JKXLT4TKF7AA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rZDBvhrfNrYQB6h9MGhx1tNtQ7AH9ehUF4SfNMCJRpv4SKrDX): protocol: read tcp 159.223.143.39:45458->79.127.205.246:4280: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 23 05:59:58.933631191 (103h9m40.78223826s) [seg_pos={0 0}, upload=82448]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jul 23 05:59:58.933756265 (103h9m40.782363345s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 23 05:59:58.933762566 (103h9m40.782369645s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 23 05:59:59.025408726 (103h9m40.874015808s) [upload=82448]: Done uploading segments...
Jul 23 05:59:59.025427547 (103h9m40.874034627s) [upload=82448]: Upload finished. err: <nil>
Duration: 979.052706ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............................X............................] duration:
529.735557ms
FAIL (2025-07-23 06:21:33 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
23 06:00:19.742341858 (103h10m1.590948949s) [noise=true , address=104.36.82.4:4804 , piece_id=SMCQKI5GIWKEHYLNSVUGHWQNBHP3S4C5RBB4E3K5CFMXE54Q4WAQ, seg_pos={0 0}, upload=82455]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g): protocol: expected piece hash; context canceled; EOF
Jul 23 06:00:19.742959346 (103h10m1.59156642s) [noise=true , address=77.90.31.16:4734 , piece_id=NFCPPB6P25YMO5UZRM4CQKMNRV5Q2FXMSR6M4VSTEERBMVNLYJJA, seg_pos={0 0}, upload=82455]: Done uploading piece. err: <nil>
Jul 23 06:00:19.743002232 (103h10m1.591609306s) [seg_pos={0 0}, upload=82455]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 23 06:00:19.743286680 (103h10m1.591893764s) [upload=82455]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 23 06:00:19.743293769 (103h10m1.591900841s) [upload=82455]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 23 06:00:19.815641707 (103h10m1.664248798s) [upload=82455]: Done uploading segments...
Jul 23 06:00:19.815666887 (103h10m1.664273954s) [upload=82455]: Upload finished. err: <nil>
Jul 23 06:01:08.401614022 (103h10m50.250221093s) [upload=82463]: Starting upload
Jul 23 06:01:08.401709823 (103h10m50.250316908s) [upload=82463]: Done starting upload
Jul 23 06:01:08.402138436 (103h10m50.250745505s) [upload=82463]: Uploading segments...
Jul 23 06:01:08.402189044 (103h10m50.250796137s) [upload=82463]: Got next segment. Inline: false
Jul 23 06:01:08.402231416 (103h10m50.250838487s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 23 06:01:08.402237731 (103h10m50.250844804s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 23 06:01:08.450301042 (103h10m50.298908146s) [seg_pos={0 0}, upload=82463]: Begin upload segment...
Jul 23 06:01:08.450364720 (103h10m50.298971798s) [seg_pos={0 0}, upload=82463]: Done begin upload segment.
Jul 23 06:01:08.450425015 (103h10m50.2990321s) [noise=true , address=45.91.116.5:4299 , piece_id=K3URA4ZO4R2CAGZLTO6YJTZQOZ6YIOB7XOJ3AQAVGDFIMVU63BIA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.450431954 (103h10m50.299039031s) [noise=true , address=45.91.116.7:4010 , piece_id=WVMOM2PPTVXY7NYI5UH25PFW6RODIUJWN73YB6FW6UOAMNY4C4SA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.450900413 (103h10m50.299507502s) [noise=true , address=77.90.31.17:4772 , piece_id=63V72ATK3KI3ADRC4N5HTRIXSEB3DRM7N73MVZNDPZJ7EVO3RZJA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.450963993 (103h10m50.299571083s) [noise=true , address=45.91.116.8:4048 , piece_id=AROG4HVUX7R6MHG3DT3N2FIQJBBEVVEVKQHSAGZCWMISWTFSQMAQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.451375091 (103h10m50.299982178s) [noise=true , address=45.91.116.13:4155 , piece_id=KMDEG3QMJNHRF7TETZAFVN3TOZWTQNW7IRTCVI3WLNE7NJAAAPCA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.451400410 (103h10m50.300007509s) [noise=true , address=77.90.31.12:4592 , piece_id=NAWAKPN4LUPCFZIXDIEPN2GKO3CSGX35LZNHI4QDCGYBYAMWBGOA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.451837258 (103h10m50.300444352s) [noise=true , address=77.90.31.17:4771 , piece_id=VCHLVD4PCT6SGJNCFA4LSEDIXSRMX27JH7VLUXDBM2PMIGZ54CNA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.452344092 (103h10m50.300951186s) [noise=true , address=45.91.116.4:4937 , piece_id=DZILXASXCQNDQGTAIYQIY4LCXHENKF6LLWHME5ILDWL3IZOSWMCQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.452773889 (103h10m50.301380982s) [noise=true , address=45.91.116.11:4119 , piece_id=LOMOQU77BVHAPGZ23ZY6CG63FZFQX2IG5QRY2P46WKSGB5AF5O3Q, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.453168858 (103h10m50.301775945s) [noise=true , address=45.91.116.5:4298 , piece_id=AIURH3V65NZD3QI3A5YGBJ222ANC4LTHEYFWTOFIC6SPVH3O4CKA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.453559736 (103h10m50.302166813s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQJDPJRZQEJHTED4UJBNSJ2CGAL6FPTLLIIRFTOVBUK65VSPLGA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.460854171 (103h10m50.309461265s) [noise=true , address=77.90.31.14:4661 , piece_id=MA3CRNAREYTT6ZDO7Q7QRSAOVQQ3EM3CHZTWYQYE6QOWNMYQBBQQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.462111720 (103h10m50.310718803s) [noise=true , address=45.91.116.12:4225 , piece_id=MFM7FUANQ2WU4GK46NPKKKFDUXSVVQ3MGK7GM3QBR7GMOD563PTQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.462819102 (103h10m50.311426189s) [noise=true , address=104.36.82.13:4186 , piece_id=O5BVVJBIWINWK2GXWWJDETED6T3Z462AAUQWCG6LGC7PVVFSHZBA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.463600977 (103h10m50.312208063s) [noise=true , address=45.91.116.7:4011 , piece_id=GKVDRTZ2TY2ESEKU2FKIJND4LXDMITNWQORE4Z47OQ454UZMM7BA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.464356397 (103h10m50.312963485s) [noise=true , address=77.90.31.13:4626 , piece_id=D63E7JGO5RRTEJGXVUMNEC3UYXHS45HBOB7S67VPCWLHJXNH7OSQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.587088382 (103h10m50.435695454s) [upload=82463]: Next returned nil segment
Jul 23 06:01:08.587107670 (103h10m50.435714757s) [upload=82463]: Waiting for error group managing segments...
Jul 23 06:01:08.609523184 (103h10m50.458130271s) [noise=true , address=77.90.31.14:4661 , piece_id=MA3CRNAREYTT6ZDO7Q7QRSAOVQQ3EM3CHZTWYQYE6QOWNMYQBBQQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.612539671 (103h10m50.46114674s) [noise=true , address=45.91.116.5:4299 , piece_id=K3URA4ZO4R2CAGZLTO6YJTZQOZ6YIOB7XOJ3AQAVGDFIMVU63BIA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.617758941 (103h10m50.466366025s) [noise=true , address=45.91.116.13:4155 , piece_id=KMDEG3QMJNHRF7TETZAFVN3TOZWTQNW7IRTCVI3WLNE7NJAAAPCA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.618513849 (103h10m50.467120922s) [noise=true , address=45.91.116.8:4048 , piece_id=AROG4HVUX7R6MHG3DT3N2FIQJBBEVVEVKQHSAGZCWMISWTFSQMAQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.624184177 (103h10m50.47279126s) [noise=true , address=45.91.116.7:4010 , piece_id=WVMOM2PPTVXY7NYI5UH25PFW6RODIUJWN73YB6FW6UOAMNY4C4SA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.625290831 (103h10m50.473897916s) [noise=true , address=45.91.116.7:4011 , piece_id=GKVDRTZ2TY2ESEKU2FKIJND4LXDMITNWQORE4Z47OQ454UZMM7BA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.628897649 (103h10m50.477504746s) [noise=true , address=45.91.116.5:4298 , piece_id=AIURH3V65NZD3QI3A5YGBJ222ANC4LTHEYFWTOFIC6SPVH3O4CKA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.629542622 (103h10m50.478149697s) [noise=true , address=45.91.116.12:4225 , piece_id=MFM7FUANQ2WU4GK46NPKKKFDUXSVVQ3MGK7GM3QBR7GMOD563PTQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.630003986 (103h10m50.478611073s) [noise=true , address=77.90.31.17:4772 , piece_id=63V72ATK3KI3ADRC4N5HTRIXSEB3DRM7N73MVZNDPZJ7EVO3RZJA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.630441959 (103h10m50.479049047s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQJDPJRZQEJHTED4UJBNSJ2CGAL6FPTLLIIRFTOVBUK65VSPLGA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.630941221 (103h10m50.479548305s) [noise=true , address=77.90.31.12:4592 , piece_id=NAWAKPN4LUPCFZIXDIEPN2GKO3CSGX35LZNHI4QDCGYBYAMWBGOA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.631378970 (103h10m50.47998605s) [noise=true , address=77.90.31.13:4626 , piece_id=D63E7JGO5RRTEJGXVUMNEC3UYXHS45HBOB7S67VPCWLHJXNH7OSQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.631840658 (103h10m50.480447733s) [noise=true , address=77.90.31.17:4771 , piece_id=VCHLVD4PCT6SGJNCFA4LSEDIXSRMX27JH7VLUXDBM2PMIGZ54CNA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.631850523 (103h10m50.480457598s) [seg_pos={0 0}, upload=82463]: Segment reached optimal threshold of 13 pieces.
Jul 23 06:01:08.631982956 (103h10m50.480590031s) [noise=true , address=104.36.82.13:4186 , piece_id=O5BVVJBIWINWK2GXWWJDETED6T3Z462AAUQWCG6LGC7PVVFSHZBA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF
Jul 23 06:01:08.632087803 (103h10m50.480694892s) [noise=true , address=45.91.116.4:4937 , piece_id=DZILXASXCQNDQGTAIYQIY4LCXHENKF6LLWHME5ILDWL3IZOSWMCQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF
Jul 23 06:01:08.633101069 (103h10m50.481708157s) [noise=true , address=45.91.116.11:4119 , piece_id=LOMOQU77BVHAPGZ23ZY6CG63FZFQX2IG5QRY2P46WKSGB5AF5O3Q, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.633132335 (103h10m50.481739428s) [seg_pos={0 0}, upload=82463]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 23 06:01:08.633221956 (103h10m50.481829049s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 23 06:01:08.633272252 (103h10m50.481879341s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 23 06:01:08.710605841 (103h10m50.559212933s) [upload=82463]: Done uploading segments...
Jul 23 06:01:08.710632085 (103h10m50.559239157s) [upload=82463]: Upload finished. err: <nil>
Duration: 529.794693ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [...................................X........................] duration:
700.523604ms
FAIL (2025-07-23 06:21:01 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
s) [upload=82448]: Got next segment. Inline: false
Jul 23 05:59:58.732247463 (103h9m40.580854536s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 23 05:59:58.732253833 (103h9m40.580860918s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 23 05:59:58.787057645 (103h9m40.635664731s) [seg_pos={0 0}, upload=82448]: Begin upload segment...
Jul 23 05:59:58.787102519 (103h9m40.635709584s) [seg_pos={0 0}, upload=82448]: Done begin upload segment.
Jul 23 05:59:58.787136403 (103h9m40.635743482s) [noise=true , address=77.90.31.12:4622 , piece_id=EZJAG75JRZTYILS7545KYAI37YOUTHHIOOSH26KIWMUQ4BCG556A, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787169354 (103h9m40.635776438s) [noise=true , address=79.127.205.235:4680 , piece_id=T3V7PMZXJOHWT77MQ52PUMFNAIAX4DLMTIIVL6XNVTNA565O5WCQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787544550 (103h9m40.636151681s) [noise=true , address=104.36.82.10:4105 , piece_id=GY72JPHTZOGEVFDG3FM5W5ZSYOPSY3BZXEME2Y37VVR2LR37SIUA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787598825 (103h9m40.636205915s) [noise=true , address=79.127.205.246:4280 , piece_id=NPBN75MFQYLCG4XKRJPLHJU5JCYBNQYIFQ7MKKB3JKXLT4TKF7AA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787946084 (103h9m40.636553167s) [noise=true , address=77.90.31.17:4799 , piece_id=RUTSDNZEXKJ5Y7WQBIVTMFUIJRYLL6DDNNJEXZAL5S7CIATWYHBQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788041731 (103h9m40.636648823s) [noise=true , address=77.90.31.11:4569 , piece_id=MTNLU22JHOYH6VSRMWLKQGSCDI5GIADPIXBVVSC2RIJLTHPMXTOA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788383218 (103h9m40.636990305s) [noise=true , address=104.36.82.42:4818 , piece_id=A6U7TDLTAZHE5IILHHCZKIHCQINHCJSWTBYNXJV7GLCWTIBNB2VA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788633554 (103h9m40.637240644s) [noise=true , address=79.127.219.51:4202 , piece_id=LJZ5A5HAHG4CGXT5J2WNSMXD7HCYSCBUEYGAGOES3NWIVV34BXEQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788735479 (103h9m40.637342568s) [noise=true , address=79.127.219.47:4040 , piece_id=BKWXBAVFXQ2RKG27WGPEAWG5UTHBVODW2BB4EGKHV6SN4EFI7KIQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789072444 (103h9m40.637679533s) [noise=true , address=77.90.31.15:4706 , piece_id=C3JOUPYB77PA2EHPSNPMDETERE724PBALBCSZ6MJCTZ6YRPUC2UA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789122202 (103h9m40.637729291s) [noise=true , address=104.36.82.9:4035 , piece_id=74BSWBLVWYQXPUI2PGV5NIJDSE6MSFKWK65TCVV6CNZKVQBQVNEA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789393961 (103h9m40.638001047s) [noise=true , address=45.91.116.10:4103 , piece_id=UGRDLKKFJRNCCDNDQTAY2LLOIS7CGI6KLNDZ7KJDOUNYWFNUAYQQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789591397 (103h9m40.638198479s) [noise=true , address=104.36.82.5:4870 , piece_id=AYTLB2OL7OAI5EO2ZE6LXIQLCGSGDXKDKT6V5HXIPAREK656MDPQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789716087 (103h9m40.638323176s) [noise=true , address=79.127.205.233:4645 , piece_id=6YAS5E7QYEZAJXB6HUY3R4PLHC4Z7EYTMSFWNFWD5AP7HKXOKTUQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789937996 (103h9m40.638545078s) [noise=true , address=104.36.82.53:4121 , piece_id=GJLXLW2A5RZUF4MBUP5AZAWFKTWGBDADIIWVIUBXPQPPGPKYZ4YA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.790288745 (103h9m40.638895835s) [noise=true , address=77.90.31.15:4708 , piece_id=67LX4NZJUKLYHRTSW3RKHHXZRKMGWJ2KV32FDA2KE3DY364CJ7OA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.790730467 (103h9m40.639337554s) [noise=true , address=45.91.116.14:4206 , piece_id=OVX63SL5OGJTT7HWDRIUV4BDVEKN2MD4ORUUK5M4JIEWX4J6VHFA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.791174576 (103h9m40.639781656s) [noise=true , address=45.91.116.13:4176 , piece_id=GOWXIBOORGZWFIBLMG46HQ4J632B3E6I3MVXNLTWPP2TY3IF2JUQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.898860128 (103h9m40.747467214s) [upload=82448]: Next returned nil segment
Jul 23 05:59:58.898881813 (103h9m40.747488906s) [upload=82448]: Waiting for error group managing segments...
Jul 23 05:59:58.905078785 (103h9m40.753685854s) [noise=true , address=45.91.116.14:4206 , piece_id=OVX63SL5OGJTT7HWDRIUV4BDVEKN2MD4ORUUK5M4JIEWX4J6VHFA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.906441982 (103h9m40.755049063s) [noise=true , address=77.90.31.15:4708 , piece_id=67LX4NZJUKLYHRTSW3RKHHXZRKMGWJ2KV32FDA2KE3DY364CJ7OA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.910115252 (103h9m40.758722337s) [noise=true , address=77.90.31.15:4706 , piece_id=C3JOUPYB77PA2EHPSNPMDETERE724PBALBCSZ6MJCTZ6YRPUC2UA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.910652695 (103h9m40.759259762s) [noise=true , address=104.36.82.9:4035 , piece_id=74BSWBLVWYQXPUI2PGV5NIJDSE6MSFKWK65TCVV6CNZKVQBQVNEA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.910672036 (103h9m40.759279117s) [noise=true , address=77.90.31.11:4569 , piece_id=MTNLU22JHOYH6VSRMWLKQGSCDI5GIADPIXBVVSC2RIJLTHPMXTOA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.911483080 (103h9m40.760090146s) [noise=true , address=45.91.116.13:4176 , piece_id=GOWXIBOORGZWFIBLMG46HQ4J632B3E6I3MVXNLTWPP2TY3IF2JUQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.911951970 (103h9m40.760559052s) [noise=true , address=45.91.116.10:4103 , piece_id=UGRDLKKFJRNCCDNDQTAY2LLOIS7CGI6KLNDZ7KJDOUNYWFNUAYQQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.912044937 (103h9m40.760652006s) [noise=true , address=77.90.31.12:4622 , piece_id=EZJAG75JRZTYILS7545KYAI37YOUTHHIOOSH26KIWMUQ4BCG556A, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.912649865 (103h9m40.761256931s) [noise=true , address=77.90.31.17:4799 , piece_id=RUTSDNZEXKJ5Y7WQBIVTMFUIJRYLL6DDNNJEXZAL5S7CIATWYHBQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.913487977 (103h9m40.76209505s) [noise=true , address=79.127.219.47:4040 , piece_id=BKWXBAVFXQ2RKG27WGPEAWG5UTHBVODW2BB4EGKHV6SN4EFI7KIQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.914486860 (103h9m40.763093927s) [noise=true , address=104.36.82.53:4121 , piece_id=GJLXLW2A5RZUF4MBUP5AZAWFKTWGBDADIIWVIUBXPQPPGPKYZ4YA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.918080770 (103h9m40.766687852s) [noise=true , address=104.36.82.5:4870 , piece_id=AYTLB2OL7OAI5EO2ZE6LXIQLCGSGDXKDKT6V5HXIPAREK656MDPQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.924012392 (103h9m40.772619461s) [noise=true , address=79.127.219.51:4202 , piece_id=LJZ5A5HAHG4CGXT5J2WNSMXD7HCYSCBUEYGAGOES3NWIVV34BXEQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.932892483 (103h9m40.78149955s) [noise=true , address=104.36.82.42:4818 , piece_id=A6U7TDLTAZHE5IILHHCZKIHCQINHCJSWTBYNXJV7GLCWTIBNB2VA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.932913099 (103h9m40.781520167s) [seg_pos={0 0}, upload=82448]: Segment reached optimal threshold of 14 pieces.
Jul 23 05:59:58.933104957 (103h9m40.781712037s) [noise=true , address=104.36.82.10:4105 , piece_id=GY72JPHTZOGEVFDG3FM5W5ZSYOPSY3BZXEME2Y37VVR2LR37SIUA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GhDrKMcDEg2RxamqKtiHphpgNdvtZYdHXrjzGsNJFjBFYeEen): protocol: expected piece hash; context canceled; EOF
Jul 23 05:59:58.933505265 (103h9m40.782112345s) [noise=true , address=79.127.205.233:4645 , piece_id=6YAS5E7QYEZAJXB6HUY3R4PLHC4Z7EYTMSFWNFWD5AP7HKXOKTUQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12NR52pweTqZNNMTyc1EHSbifDKB4awrnnb4megtZHwANSrR18T): protocol: read tcp 159.223.143.39:46566->79.127.205.233:4645: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 23 05:59:58.933568113 (103h9m40.782175194s) [noise=true , address=79.127.205.235:4680 , piece_id=T3V7PMZXJOHWT77MQ52PUMFNAIAX4DLMTIIVL6XNVTNA565O5WCQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GGvQQLoBcTniZST8Hz9h8eCLWbU4DrXn66DGy33iGLXaFgJLj): protocol: read tcp 159.223.143.39:45394->79.127.205.235:4680: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 23 05:59:58.933618915 (103h9m40.782225986s) [noise=true , address=79.127.205.246:4280 , piece_id=NPBN75MFQYLCG4XKRJPLHJU5JCYBNQYIFQ7MKKB3JKXLT4TKF7AA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rZDBvhrfNrYQB6h9MGhx1tNtQ7AH9ehUF4SfNMCJRpv4SKrDX): protocol: read tcp 159.223.143.39:45458->79.127.205.246:4280: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 23 05:59:58.933631191 (103h9m40.78223826s) [seg_pos={0 0}, upload=82448]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jul 23 05:59:58.933756265 (103h9m40.782363345s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 23 05:59:58.933762566 (103h9m40.782369645s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 23 05:59:59.025408726 (103h9m40.874015808s) [upload=82448]: Done uploading segments...
Jul 23 05:59:59.025427547 (103h9m40.874034627s) [upload=82448]: Upload finished. err: <nil>
Duration: 700.66504ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [................................X...........................] duration:
488.936915ms
FAIL (2025-07-23 06:20:33 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
23 06:00:19.742341858 (103h10m1.590948949s) [noise=true , address=104.36.82.4:4804 , piece_id=SMCQKI5GIWKEHYLNSVUGHWQNBHP3S4C5RBB4E3K5CFMXE54Q4WAQ, seg_pos={0 0}, upload=82455]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g): protocol: expected piece hash; context canceled; EOF
Jul 23 06:00:19.742959346 (103h10m1.59156642s) [noise=true , address=77.90.31.16:4734 , piece_id=NFCPPB6P25YMO5UZRM4CQKMNRV5Q2FXMSR6M4VSTEERBMVNLYJJA, seg_pos={0 0}, upload=82455]: Done uploading piece. err: <nil>
Jul 23 06:00:19.743002232 (103h10m1.591609306s) [seg_pos={0 0}, upload=82455]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 23 06:00:19.743286680 (103h10m1.591893764s) [upload=82455]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 23 06:00:19.743293769 (103h10m1.591900841s) [upload=82455]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 23 06:00:19.815641707 (103h10m1.664248798s) [upload=82455]: Done uploading segments...
Jul 23 06:00:19.815666887 (103h10m1.664273954s) [upload=82455]: Upload finished. err: <nil>
Jul 23 06:01:08.401614022 (103h10m50.250221093s) [upload=82463]: Starting upload
Jul 23 06:01:08.401709823 (103h10m50.250316908s) [upload=82463]: Done starting upload
Jul 23 06:01:08.402138436 (103h10m50.250745505s) [upload=82463]: Uploading segments...
Jul 23 06:01:08.402189044 (103h10m50.250796137s) [upload=82463]: Got next segment. Inline: false
Jul 23 06:01:08.402231416 (103h10m50.250838487s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 23 06:01:08.402237731 (103h10m50.250844804s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 23 06:01:08.450301042 (103h10m50.298908146s) [seg_pos={0 0}, upload=82463]: Begin upload segment...
Jul 23 06:01:08.450364720 (103h10m50.298971798s) [seg_pos={0 0}, upload=82463]: Done begin upload segment.
Jul 23 06:01:08.450425015 (103h10m50.2990321s) [noise=true , address=45.91.116.5:4299 , piece_id=K3URA4ZO4R2CAGZLTO6YJTZQOZ6YIOB7XOJ3AQAVGDFIMVU63BIA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.450431954 (103h10m50.299039031s) [noise=true , address=45.91.116.7:4010 , piece_id=WVMOM2PPTVXY7NYI5UH25PFW6RODIUJWN73YB6FW6UOAMNY4C4SA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.450900413 (103h10m50.299507502s) [noise=true , address=77.90.31.17:4772 , piece_id=63V72ATK3KI3ADRC4N5HTRIXSEB3DRM7N73MVZNDPZJ7EVO3RZJA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.450963993 (103h10m50.299571083s) [noise=true , address=45.91.116.8:4048 , piece_id=AROG4HVUX7R6MHG3DT3N2FIQJBBEVVEVKQHSAGZCWMISWTFSQMAQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.451375091 (103h10m50.299982178s) [noise=true , address=45.91.116.13:4155 , piece_id=KMDEG3QMJNHRF7TETZAFVN3TOZWTQNW7IRTCVI3WLNE7NJAAAPCA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.451400410 (103h10m50.300007509s) [noise=true , address=77.90.31.12:4592 , piece_id=NAWAKPN4LUPCFZIXDIEPN2GKO3CSGX35LZNHI4QDCGYBYAMWBGOA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.451837258 (103h10m50.300444352s) [noise=true , address=77.90.31.17:4771 , piece_id=VCHLVD4PCT6SGJNCFA4LSEDIXSRMX27JH7VLUXDBM2PMIGZ54CNA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.452344092 (103h10m50.300951186s) [noise=true , address=45.91.116.4:4937 , piece_id=DZILXASXCQNDQGTAIYQIY4LCXHENKF6LLWHME5ILDWL3IZOSWMCQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.452773889 (103h10m50.301380982s) [noise=true , address=45.91.116.11:4119 , piece_id=LOMOQU77BVHAPGZ23ZY6CG63FZFQX2IG5QRY2P46WKSGB5AF5O3Q, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.453168858 (103h10m50.301775945s) [noise=true , address=45.91.116.5:4298 , piece_id=AIURH3V65NZD3QI3A5YGBJ222ANC4LTHEYFWTOFIC6SPVH3O4CKA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.453559736 (103h10m50.302166813s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQJDPJRZQEJHTED4UJBNSJ2CGAL6FPTLLIIRFTOVBUK65VSPLGA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.460854171 (103h10m50.309461265s) [noise=true , address=77.90.31.14:4661 , piece_id=MA3CRNAREYTT6ZDO7Q7QRSAOVQQ3EM3CHZTWYQYE6QOWNMYQBBQQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.462111720 (103h10m50.310718803s) [noise=true , address=45.91.116.12:4225 , piece_id=MFM7FUANQ2WU4GK46NPKKKFDUXSVVQ3MGK7GM3QBR7GMOD563PTQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.462819102 (103h10m50.311426189s) [noise=true , address=104.36.82.13:4186 , piece_id=O5BVVJBIWINWK2GXWWJDETED6T3Z462AAUQWCG6LGC7PVVFSHZBA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.463600977 (103h10m50.312208063s) [noise=true , address=45.91.116.7:4011 , piece_id=GKVDRTZ2TY2ESEKU2FKIJND4LXDMITNWQORE4Z47OQ454UZMM7BA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.464356397 (103h10m50.312963485s) [noise=true , address=77.90.31.13:4626 , piece_id=D63E7JGO5RRTEJGXVUMNEC3UYXHS45HBOB7S67VPCWLHJXNH7OSQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.587088382 (103h10m50.435695454s) [upload=82463]: Next returned nil segment
Jul 23 06:01:08.587107670 (103h10m50.435714757s) [upload=82463]: Waiting for error group managing segments...
Jul 23 06:01:08.609523184 (103h10m50.458130271s) [noise=true , address=77.90.31.14:4661 , piece_id=MA3CRNAREYTT6ZDO7Q7QRSAOVQQ3EM3CHZTWYQYE6QOWNMYQBBQQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.612539671 (103h10m50.46114674s) [noise=true , address=45.91.116.5:4299 , piece_id=K3URA4ZO4R2CAGZLTO6YJTZQOZ6YIOB7XOJ3AQAVGDFIMVU63BIA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.617758941 (103h10m50.466366025s) [noise=true , address=45.91.116.13:4155 , piece_id=KMDEG3QMJNHRF7TETZAFVN3TOZWTQNW7IRTCVI3WLNE7NJAAAPCA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.618513849 (103h10m50.467120922s) [noise=true , address=45.91.116.8:4048 , piece_id=AROG4HVUX7R6MHG3DT3N2FIQJBBEVVEVKQHSAGZCWMISWTFSQMAQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.624184177 (103h10m50.47279126s) [noise=true , address=45.91.116.7:4010 , piece_id=WVMOM2PPTVXY7NYI5UH25PFW6RODIUJWN73YB6FW6UOAMNY4C4SA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.625290831 (103h10m50.473897916s) [noise=true , address=45.91.116.7:4011 , piece_id=GKVDRTZ2TY2ESEKU2FKIJND4LXDMITNWQORE4Z47OQ454UZMM7BA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.628897649 (103h10m50.477504746s) [noise=true , address=45.91.116.5:4298 , piece_id=AIURH3V65NZD3QI3A5YGBJ222ANC4LTHEYFWTOFIC6SPVH3O4CKA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.629542622 (103h10m50.478149697s) [noise=true , address=45.91.116.12:4225 , piece_id=MFM7FUANQ2WU4GK46NPKKKFDUXSVVQ3MGK7GM3QBR7GMOD563PTQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.630003986 (103h10m50.478611073s) [noise=true , address=77.90.31.17:4772 , piece_id=63V72ATK3KI3ADRC4N5HTRIXSEB3DRM7N73MVZNDPZJ7EVO3RZJA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.630441959 (103h10m50.479049047s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQJDPJRZQEJHTED4UJBNSJ2CGAL6FPTLLIIRFTOVBUK65VSPLGA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.630941221 (103h10m50.479548305s) [noise=true , address=77.90.31.12:4592 , piece_id=NAWAKPN4LUPCFZIXDIEPN2GKO3CSGX35LZNHI4QDCGYBYAMWBGOA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.631378970 (103h10m50.47998605s) [noise=true , address=77.90.31.13:4626 , piece_id=D63E7JGO5RRTEJGXVUMNEC3UYXHS45HBOB7S67VPCWLHJXNH7OSQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.631840658 (103h10m50.480447733s) [noise=true , address=77.90.31.17:4771 , piece_id=VCHLVD4PCT6SGJNCFA4LSEDIXSRMX27JH7VLUXDBM2PMIGZ54CNA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.631850523 (103h10m50.480457598s) [seg_pos={0 0}, upload=82463]: Segment reached optimal threshold of 13 pieces.
Jul 23 06:01:08.631982956 (103h10m50.480590031s) [noise=true , address=104.36.82.13:4186 , piece_id=O5BVVJBIWINWK2GXWWJDETED6T3Z462AAUQWCG6LGC7PVVFSHZBA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF
Jul 23 06:01:08.632087803 (103h10m50.480694892s) [noise=true , address=45.91.116.4:4937 , piece_id=DZILXASXCQNDQGTAIYQIY4LCXHENKF6LLWHME5ILDWL3IZOSWMCQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF
Jul 23 06:01:08.633101069 (103h10m50.481708157s) [noise=true , address=45.91.116.11:4119 , piece_id=LOMOQU77BVHAPGZ23ZY6CG63FZFQX2IG5QRY2P46WKSGB5AF5O3Q, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.633132335 (103h10m50.481739428s) [seg_pos={0 0}, upload=82463]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 23 06:01:08.633221956 (103h10m50.481829049s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 23 06:01:08.633272252 (103h10m50.481879341s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 23 06:01:08.710605841 (103h10m50.559212933s) [upload=82463]: Done uploading segments...
Jul 23 06:01:08.710632085 (103h10m50.559239157s) [upload=82463]: Upload finished. err: <nil>
Duration: 489.022137ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [....................................X.......................] duration:
845.838404ms
FAIL (2025-07-23 06:20:14 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
s) [upload=82448]: Got next segment. Inline: false
Jul 23 05:59:58.732247463 (103h9m40.580854536s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 23 05:59:58.732253833 (103h9m40.580860918s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 23 05:59:58.787057645 (103h9m40.635664731s) [seg_pos={0 0}, upload=82448]: Begin upload segment...
Jul 23 05:59:58.787102519 (103h9m40.635709584s) [seg_pos={0 0}, upload=82448]: Done begin upload segment.
Jul 23 05:59:58.787136403 (103h9m40.635743482s) [noise=true , address=77.90.31.12:4622 , piece_id=EZJAG75JRZTYILS7545KYAI37YOUTHHIOOSH26KIWMUQ4BCG556A, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787169354 (103h9m40.635776438s) [noise=true , address=79.127.205.235:4680 , piece_id=T3V7PMZXJOHWT77MQ52PUMFNAIAX4DLMTIIVL6XNVTNA565O5WCQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787544550 (103h9m40.636151681s) [noise=true , address=104.36.82.10:4105 , piece_id=GY72JPHTZOGEVFDG3FM5W5ZSYOPSY3BZXEME2Y37VVR2LR37SIUA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787598825 (103h9m40.636205915s) [noise=true , address=79.127.205.246:4280 , piece_id=NPBN75MFQYLCG4XKRJPLHJU5JCYBNQYIFQ7MKKB3JKXLT4TKF7AA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787946084 (103h9m40.636553167s) [noise=true , address=77.90.31.17:4799 , piece_id=RUTSDNZEXKJ5Y7WQBIVTMFUIJRYLL6DDNNJEXZAL5S7CIATWYHBQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788041731 (103h9m40.636648823s) [noise=true , address=77.90.31.11:4569 , piece_id=MTNLU22JHOYH6VSRMWLKQGSCDI5GIADPIXBVVSC2RIJLTHPMXTOA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788383218 (103h9m40.636990305s) [noise=true , address=104.36.82.42:4818 , piece_id=A6U7TDLTAZHE5IILHHCZKIHCQINHCJSWTBYNXJV7GLCWTIBNB2VA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788633554 (103h9m40.637240644s) [noise=true , address=79.127.219.51:4202 , piece_id=LJZ5A5HAHG4CGXT5J2WNSMXD7HCYSCBUEYGAGOES3NWIVV34BXEQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788735479 (103h9m40.637342568s) [noise=true , address=79.127.219.47:4040 , piece_id=BKWXBAVFXQ2RKG27WGPEAWG5UTHBVODW2BB4EGKHV6SN4EFI7KIQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789072444 (103h9m40.637679533s) [noise=true , address=77.90.31.15:4706 , piece_id=C3JOUPYB77PA2EHPSNPMDETERE724PBALBCSZ6MJCTZ6YRPUC2UA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789122202 (103h9m40.637729291s) [noise=true , address=104.36.82.9:4035 , piece_id=74BSWBLVWYQXPUI2PGV5NIJDSE6MSFKWK65TCVV6CNZKVQBQVNEA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789393961 (103h9m40.638001047s) [noise=true , address=45.91.116.10:4103 , piece_id=UGRDLKKFJRNCCDNDQTAY2LLOIS7CGI6KLNDZ7KJDOUNYWFNUAYQQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789591397 (103h9m40.638198479s) [noise=true , address=104.36.82.5:4870 , piece_id=AYTLB2OL7OAI5EO2ZE6LXIQLCGSGDXKDKT6V5HXIPAREK656MDPQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789716087 (103h9m40.638323176s) [noise=true , address=79.127.205.233:4645 , piece_id=6YAS5E7QYEZAJXB6HUY3R4PLHC4Z7EYTMSFWNFWD5AP7HKXOKTUQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789937996 (103h9m40.638545078s) [noise=true , address=104.36.82.53:4121 , piece_id=GJLXLW2A5RZUF4MBUP5AZAWFKTWGBDADIIWVIUBXPQPPGPKYZ4YA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.790288745 (103h9m40.638895835s) [noise=true , address=77.90.31.15:4708 , piece_id=67LX4NZJUKLYHRTSW3RKHHXZRKMGWJ2KV32FDA2KE3DY364CJ7OA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.790730467 (103h9m40.639337554s) [noise=true , address=45.91.116.14:4206 , piece_id=OVX63SL5OGJTT7HWDRIUV4BDVEKN2MD4ORUUK5M4JIEWX4J6VHFA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.791174576 (103h9m40.639781656s) [noise=true , address=45.91.116.13:4176 , piece_id=GOWXIBOORGZWFIBLMG46HQ4J632B3E6I3MVXNLTWPP2TY3IF2JUQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.898860128 (103h9m40.747467214s) [upload=82448]: Next returned nil segment
Jul 23 05:59:58.898881813 (103h9m40.747488906s) [upload=82448]: Waiting for error group managing segments...
Jul 23 05:59:58.905078785 (103h9m40.753685854s) [noise=true , address=45.91.116.14:4206 , piece_id=OVX63SL5OGJTT7HWDRIUV4BDVEKN2MD4ORUUK5M4JIEWX4J6VHFA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.906441982 (103h9m40.755049063s) [noise=true , address=77.90.31.15:4708 , piece_id=67LX4NZJUKLYHRTSW3RKHHXZRKMGWJ2KV32FDA2KE3DY364CJ7OA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.910115252 (103h9m40.758722337s) [noise=true , address=77.90.31.15:4706 , piece_id=C3JOUPYB77PA2EHPSNPMDETERE724PBALBCSZ6MJCTZ6YRPUC2UA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.910652695 (103h9m40.759259762s) [noise=true , address=104.36.82.9:4035 , piece_id=74BSWBLVWYQXPUI2PGV5NIJDSE6MSFKWK65TCVV6CNZKVQBQVNEA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.910672036 (103h9m40.759279117s) [noise=true , address=77.90.31.11:4569 , piece_id=MTNLU22JHOYH6VSRMWLKQGSCDI5GIADPIXBVVSC2RIJLTHPMXTOA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.911483080 (103h9m40.760090146s) [noise=true , address=45.91.116.13:4176 , piece_id=GOWXIBOORGZWFIBLMG46HQ4J632B3E6I3MVXNLTWPP2TY3IF2JUQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.911951970 (103h9m40.760559052s) [noise=true , address=45.91.116.10:4103 , piece_id=UGRDLKKFJRNCCDNDQTAY2LLOIS7CGI6KLNDZ7KJDOUNYWFNUAYQQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.912044937 (103h9m40.760652006s) [noise=true , address=77.90.31.12:4622 , piece_id=EZJAG75JRZTYILS7545KYAI37YOUTHHIOOSH26KIWMUQ4BCG556A, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.912649865 (103h9m40.761256931s) [noise=true , address=77.90.31.17:4799 , piece_id=RUTSDNZEXKJ5Y7WQBIVTMFUIJRYLL6DDNNJEXZAL5S7CIATWYHBQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.913487977 (103h9m40.76209505s) [noise=true , address=79.127.219.47:4040 , piece_id=BKWXBAVFXQ2RKG27WGPEAWG5UTHBVODW2BB4EGKHV6SN4EFI7KIQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.914486860 (103h9m40.763093927s) [noise=true , address=104.36.82.53:4121 , piece_id=GJLXLW2A5RZUF4MBUP5AZAWFKTWGBDADIIWVIUBXPQPPGPKYZ4YA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.918080770 (103h9m40.766687852s) [noise=true , address=104.36.82.5:4870 , piece_id=AYTLB2OL7OAI5EO2ZE6LXIQLCGSGDXKDKT6V5HXIPAREK656MDPQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.924012392 (103h9m40.772619461s) [noise=true , address=79.127.219.51:4202 , piece_id=LJZ5A5HAHG4CGXT5J2WNSMXD7HCYSCBUEYGAGOES3NWIVV34BXEQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.932892483 (103h9m40.78149955s) [noise=true , address=104.36.82.42:4818 , piece_id=A6U7TDLTAZHE5IILHHCZKIHCQINHCJSWTBYNXJV7GLCWTIBNB2VA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.932913099 (103h9m40.781520167s) [seg_pos={0 0}, upload=82448]: Segment reached optimal threshold of 14 pieces.
Jul 23 05:59:58.933104957 (103h9m40.781712037s) [noise=true , address=104.36.82.10:4105 , piece_id=GY72JPHTZOGEVFDG3FM5W5ZSYOPSY3BZXEME2Y37VVR2LR37SIUA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GhDrKMcDEg2RxamqKtiHphpgNdvtZYdHXrjzGsNJFjBFYeEen): protocol: expected piece hash; context canceled; EOF
Jul 23 05:59:58.933505265 (103h9m40.782112345s) [noise=true , address=79.127.205.233:4645 , piece_id=6YAS5E7QYEZAJXB6HUY3R4PLHC4Z7EYTMSFWNFWD5AP7HKXOKTUQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12NR52pweTqZNNMTyc1EHSbifDKB4awrnnb4megtZHwANSrR18T): protocol: read tcp 159.223.143.39:46566->79.127.205.233:4645: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 23 05:59:58.933568113 (103h9m40.782175194s) [noise=true , address=79.127.205.235:4680 , piece_id=T3V7PMZXJOHWT77MQ52PUMFNAIAX4DLMTIIVL6XNVTNA565O5WCQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GGvQQLoBcTniZST8Hz9h8eCLWbU4DrXn66DGy33iGLXaFgJLj): protocol: read tcp 159.223.143.39:45394->79.127.205.235:4680: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 23 05:59:58.933618915 (103h9m40.782225986s) [noise=true , address=79.127.205.246:4280 , piece_id=NPBN75MFQYLCG4XKRJPLHJU5JCYBNQYIFQ7MKKB3JKXLT4TKF7AA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rZDBvhrfNrYQB6h9MGhx1tNtQ7AH9ehUF4SfNMCJRpv4SKrDX): protocol: read tcp 159.223.143.39:45458->79.127.205.246:4280: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 23 05:59:58.933631191 (103h9m40.78223826s) [seg_pos={0 0}, upload=82448]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jul 23 05:59:58.933756265 (103h9m40.782363345s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 23 05:59:58.933762566 (103h9m40.782369645s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 23 05:59:59.025408726 (103h9m40.874015808s) [upload=82448]: Done uploading segments...
Jul 23 05:59:59.025427547 (103h9m40.874034627s) [upload=82448]: Upload finished. err: <nil>
Duration: 845.874087ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.....................................X......................] duration:
925.322155ms
FAIL (2025-07-23 06:19:30 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
s) [upload=82448]: Got next segment. Inline: false
Jul 23 05:59:58.732247463 (103h9m40.580854536s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 23 05:59:58.732253833 (103h9m40.580860918s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 23 05:59:58.787057645 (103h9m40.635664731s) [seg_pos={0 0}, upload=82448]: Begin upload segment...
Jul 23 05:59:58.787102519 (103h9m40.635709584s) [seg_pos={0 0}, upload=82448]: Done begin upload segment.
Jul 23 05:59:58.787136403 (103h9m40.635743482s) [noise=true , address=77.90.31.12:4622 , piece_id=EZJAG75JRZTYILS7545KYAI37YOUTHHIOOSH26KIWMUQ4BCG556A, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787169354 (103h9m40.635776438s) [noise=true , address=79.127.205.235:4680 , piece_id=T3V7PMZXJOHWT77MQ52PUMFNAIAX4DLMTIIVL6XNVTNA565O5WCQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787544550 (103h9m40.636151681s) [noise=true , address=104.36.82.10:4105 , piece_id=GY72JPHTZOGEVFDG3FM5W5ZSYOPSY3BZXEME2Y37VVR2LR37SIUA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787598825 (103h9m40.636205915s) [noise=true , address=79.127.205.246:4280 , piece_id=NPBN75MFQYLCG4XKRJPLHJU5JCYBNQYIFQ7MKKB3JKXLT4TKF7AA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787946084 (103h9m40.636553167s) [noise=true , address=77.90.31.17:4799 , piece_id=RUTSDNZEXKJ5Y7WQBIVTMFUIJRYLL6DDNNJEXZAL5S7CIATWYHBQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788041731 (103h9m40.636648823s) [noise=true , address=77.90.31.11:4569 , piece_id=MTNLU22JHOYH6VSRMWLKQGSCDI5GIADPIXBVVSC2RIJLTHPMXTOA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788383218 (103h9m40.636990305s) [noise=true , address=104.36.82.42:4818 , piece_id=A6U7TDLTAZHE5IILHHCZKIHCQINHCJSWTBYNXJV7GLCWTIBNB2VA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788633554 (103h9m40.637240644s) [noise=true , address=79.127.219.51:4202 , piece_id=LJZ5A5HAHG4CGXT5J2WNSMXD7HCYSCBUEYGAGOES3NWIVV34BXEQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788735479 (103h9m40.637342568s) [noise=true , address=79.127.219.47:4040 , piece_id=BKWXBAVFXQ2RKG27WGPEAWG5UTHBVODW2BB4EGKHV6SN4EFI7KIQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789072444 (103h9m40.637679533s) [noise=true , address=77.90.31.15:4706 , piece_id=C3JOUPYB77PA2EHPSNPMDETERE724PBALBCSZ6MJCTZ6YRPUC2UA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789122202 (103h9m40.637729291s) [noise=true , address=104.36.82.9:4035 , piece_id=74BSWBLVWYQXPUI2PGV5NIJDSE6MSFKWK65TCVV6CNZKVQBQVNEA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789393961 (103h9m40.638001047s) [noise=true , address=45.91.116.10:4103 , piece_id=UGRDLKKFJRNCCDNDQTAY2LLOIS7CGI6KLNDZ7KJDOUNYWFNUAYQQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789591397 (103h9m40.638198479s) [noise=true , address=104.36.82.5:4870 , piece_id=AYTLB2OL7OAI5EO2ZE6LXIQLCGSGDXKDKT6V5HXIPAREK656MDPQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789716087 (103h9m40.638323176s) [noise=true , address=79.127.205.233:4645 , piece_id=6YAS5E7QYEZAJXB6HUY3R4PLHC4Z7EYTMSFWNFWD5AP7HKXOKTUQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789937996 (103h9m40.638545078s) [noise=true , address=104.36.82.53:4121 , piece_id=GJLXLW2A5RZUF4MBUP5AZAWFKTWGBDADIIWVIUBXPQPPGPKYZ4YA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.790288745 (103h9m40.638895835s) [noise=true , address=77.90.31.15:4708 , piece_id=67LX4NZJUKLYHRTSW3RKHHXZRKMGWJ2KV32FDA2KE3DY364CJ7OA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.790730467 (103h9m40.639337554s) [noise=true , address=45.91.116.14:4206 , piece_id=OVX63SL5OGJTT7HWDRIUV4BDVEKN2MD4ORUUK5M4JIEWX4J6VHFA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.791174576 (103h9m40.639781656s) [noise=true , address=45.91.116.13:4176 , piece_id=GOWXIBOORGZWFIBLMG46HQ4J632B3E6I3MVXNLTWPP2TY3IF2JUQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.898860128 (103h9m40.747467214s) [upload=82448]: Next returned nil segment
Jul 23 05:59:58.898881813 (103h9m40.747488906s) [upload=82448]: Waiting for error group managing segments...
Jul 23 05:59:58.905078785 (103h9m40.753685854s) [noise=true , address=45.91.116.14:4206 , piece_id=OVX63SL5OGJTT7HWDRIUV4BDVEKN2MD4ORUUK5M4JIEWX4J6VHFA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.906441982 (103h9m40.755049063s) [noise=true , address=77.90.31.15:4708 , piece_id=67LX4NZJUKLYHRTSW3RKHHXZRKMGWJ2KV32FDA2KE3DY364CJ7OA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.910115252 (103h9m40.758722337s) [noise=true , address=77.90.31.15:4706 , piece_id=C3JOUPYB77PA2EHPSNPMDETERE724PBALBCSZ6MJCTZ6YRPUC2UA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.910652695 (103h9m40.759259762s) [noise=true , address=104.36.82.9:4035 , piece_id=74BSWBLVWYQXPUI2PGV5NIJDSE6MSFKWK65TCVV6CNZKVQBQVNEA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.910672036 (103h9m40.759279117s) [noise=true , address=77.90.31.11:4569 , piece_id=MTNLU22JHOYH6VSRMWLKQGSCDI5GIADPIXBVVSC2RIJLTHPMXTOA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.911483080 (103h9m40.760090146s) [noise=true , address=45.91.116.13:4176 , piece_id=GOWXIBOORGZWFIBLMG46HQ4J632B3E6I3MVXNLTWPP2TY3IF2JUQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.911951970 (103h9m40.760559052s) [noise=true , address=45.91.116.10:4103 , piece_id=UGRDLKKFJRNCCDNDQTAY2LLOIS7CGI6KLNDZ7KJDOUNYWFNUAYQQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.912044937 (103h9m40.760652006s) [noise=true , address=77.90.31.12:4622 , piece_id=EZJAG75JRZTYILS7545KYAI37YOUTHHIOOSH26KIWMUQ4BCG556A, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.912649865 (103h9m40.761256931s) [noise=true , address=77.90.31.17:4799 , piece_id=RUTSDNZEXKJ5Y7WQBIVTMFUIJRYLL6DDNNJEXZAL5S7CIATWYHBQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.913487977 (103h9m40.76209505s) [noise=true , address=79.127.219.47:4040 , piece_id=BKWXBAVFXQ2RKG27WGPEAWG5UTHBVODW2BB4EGKHV6SN4EFI7KIQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.914486860 (103h9m40.763093927s) [noise=true , address=104.36.82.53:4121 , piece_id=GJLXLW2A5RZUF4MBUP5AZAWFKTWGBDADIIWVIUBXPQPPGPKYZ4YA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.918080770 (103h9m40.766687852s) [noise=true , address=104.36.82.5:4870 , piece_id=AYTLB2OL7OAI5EO2ZE6LXIQLCGSGDXKDKT6V5HXIPAREK656MDPQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.924012392 (103h9m40.772619461s) [noise=true , address=79.127.219.51:4202 , piece_id=LJZ5A5HAHG4CGXT5J2WNSMXD7HCYSCBUEYGAGOES3NWIVV34BXEQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.932892483 (103h9m40.78149955s) [noise=true , address=104.36.82.42:4818 , piece_id=A6U7TDLTAZHE5IILHHCZKIHCQINHCJSWTBYNXJV7GLCWTIBNB2VA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.932913099 (103h9m40.781520167s) [seg_pos={0 0}, upload=82448]: Segment reached optimal threshold of 14 pieces.
Jul 23 05:59:58.933104957 (103h9m40.781712037s) [noise=true , address=104.36.82.10:4105 , piece_id=GY72JPHTZOGEVFDG3FM5W5ZSYOPSY3BZXEME2Y37VVR2LR37SIUA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GhDrKMcDEg2RxamqKtiHphpgNdvtZYdHXrjzGsNJFjBFYeEen): protocol: expected piece hash; context canceled; EOF
Jul 23 05:59:58.933505265 (103h9m40.782112345s) [noise=true , address=79.127.205.233:4645 , piece_id=6YAS5E7QYEZAJXB6HUY3R4PLHC4Z7EYTMSFWNFWD5AP7HKXOKTUQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12NR52pweTqZNNMTyc1EHSbifDKB4awrnnb4megtZHwANSrR18T): protocol: read tcp 159.223.143.39:46566->79.127.205.233:4645: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 23 05:59:58.933568113 (103h9m40.782175194s) [noise=true , address=79.127.205.235:4680 , piece_id=T3V7PMZXJOHWT77MQ52PUMFNAIAX4DLMTIIVL6XNVTNA565O5WCQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GGvQQLoBcTniZST8Hz9h8eCLWbU4DrXn66DGy33iGLXaFgJLj): protocol: read tcp 159.223.143.39:45394->79.127.205.235:4680: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 23 05:59:58.933618915 (103h9m40.782225986s) [noise=true , address=79.127.205.246:4280 , piece_id=NPBN75MFQYLCG4XKRJPLHJU5JCYBNQYIFQ7MKKB3JKXLT4TKF7AA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rZDBvhrfNrYQB6h9MGhx1tNtQ7AH9ehUF4SfNMCJRpv4SKrDX): protocol: read tcp 159.223.143.39:45458->79.127.205.246:4280: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 23 05:59:58.933631191 (103h9m40.78223826s) [seg_pos={0 0}, upload=82448]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jul 23 05:59:58.933756265 (103h9m40.782363345s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 23 05:59:58.933762566 (103h9m40.782369645s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 23 05:59:59.025408726 (103h9m40.874015808s) [upload=82448]: Done uploading segments...
Jul 23 05:59:59.025427547 (103h9m40.874034627s) [upload=82448]: Upload finished. err: <nil>
Duration: 925.377952ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................................X..........................] duration:
565.82487ms
FAIL (2025-07-23 06:19:26 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
23 06:00:19.742341858 (103h10m1.590948949s) [noise=true , address=104.36.82.4:4804 , piece_id=SMCQKI5GIWKEHYLNSVUGHWQNBHP3S4C5RBB4E3K5CFMXE54Q4WAQ, seg_pos={0 0}, upload=82455]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g): protocol: expected piece hash; context canceled; EOF
Jul 23 06:00:19.742959346 (103h10m1.59156642s) [noise=true , address=77.90.31.16:4734 , piece_id=NFCPPB6P25YMO5UZRM4CQKMNRV5Q2FXMSR6M4VSTEERBMVNLYJJA, seg_pos={0 0}, upload=82455]: Done uploading piece. err: <nil>
Jul 23 06:00:19.743002232 (103h10m1.591609306s) [seg_pos={0 0}, upload=82455]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 23 06:00:19.743286680 (103h10m1.591893764s) [upload=82455]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 23 06:00:19.743293769 (103h10m1.591900841s) [upload=82455]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 23 06:00:19.815641707 (103h10m1.664248798s) [upload=82455]: Done uploading segments...
Jul 23 06:00:19.815666887 (103h10m1.664273954s) [upload=82455]: Upload finished. err: <nil>
Jul 23 06:01:08.401614022 (103h10m50.250221093s) [upload=82463]: Starting upload
Jul 23 06:01:08.401709823 (103h10m50.250316908s) [upload=82463]: Done starting upload
Jul 23 06:01:08.402138436 (103h10m50.250745505s) [upload=82463]: Uploading segments...
Jul 23 06:01:08.402189044 (103h10m50.250796137s) [upload=82463]: Got next segment. Inline: false
Jul 23 06:01:08.402231416 (103h10m50.250838487s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 23 06:01:08.402237731 (103h10m50.250844804s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 23 06:01:08.450301042 (103h10m50.298908146s) [seg_pos={0 0}, upload=82463]: Begin upload segment...
Jul 23 06:01:08.450364720 (103h10m50.298971798s) [seg_pos={0 0}, upload=82463]: Done begin upload segment.
Jul 23 06:01:08.450425015 (103h10m50.2990321s) [noise=true , address=45.91.116.5:4299 , piece_id=K3URA4ZO4R2CAGZLTO6YJTZQOZ6YIOB7XOJ3AQAVGDFIMVU63BIA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.450431954 (103h10m50.299039031s) [noise=true , address=45.91.116.7:4010 , piece_id=WVMOM2PPTVXY7NYI5UH25PFW6RODIUJWN73YB6FW6UOAMNY4C4SA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.450900413 (103h10m50.299507502s) [noise=true , address=77.90.31.17:4772 , piece_id=63V72ATK3KI3ADRC4N5HTRIXSEB3DRM7N73MVZNDPZJ7EVO3RZJA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.450963993 (103h10m50.299571083s) [noise=true , address=45.91.116.8:4048 , piece_id=AROG4HVUX7R6MHG3DT3N2FIQJBBEVVEVKQHSAGZCWMISWTFSQMAQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.451375091 (103h10m50.299982178s) [noise=true , address=45.91.116.13:4155 , piece_id=KMDEG3QMJNHRF7TETZAFVN3TOZWTQNW7IRTCVI3WLNE7NJAAAPCA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.451400410 (103h10m50.300007509s) [noise=true , address=77.90.31.12:4592 , piece_id=NAWAKPN4LUPCFZIXDIEPN2GKO3CSGX35LZNHI4QDCGYBYAMWBGOA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.451837258 (103h10m50.300444352s) [noise=true , address=77.90.31.17:4771 , piece_id=VCHLVD4PCT6SGJNCFA4LSEDIXSRMX27JH7VLUXDBM2PMIGZ54CNA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.452344092 (103h10m50.300951186s) [noise=true , address=45.91.116.4:4937 , piece_id=DZILXASXCQNDQGTAIYQIY4LCXHENKF6LLWHME5ILDWL3IZOSWMCQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.452773889 (103h10m50.301380982s) [noise=true , address=45.91.116.11:4119 , piece_id=LOMOQU77BVHAPGZ23ZY6CG63FZFQX2IG5QRY2P46WKSGB5AF5O3Q, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.453168858 (103h10m50.301775945s) [noise=true , address=45.91.116.5:4298 , piece_id=AIURH3V65NZD3QI3A5YGBJ222ANC4LTHEYFWTOFIC6SPVH3O4CKA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.453559736 (103h10m50.302166813s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQJDPJRZQEJHTED4UJBNSJ2CGAL6FPTLLIIRFTOVBUK65VSPLGA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.460854171 (103h10m50.309461265s) [noise=true , address=77.90.31.14:4661 , piece_id=MA3CRNAREYTT6ZDO7Q7QRSAOVQQ3EM3CHZTWYQYE6QOWNMYQBBQQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.462111720 (103h10m50.310718803s) [noise=true , address=45.91.116.12:4225 , piece_id=MFM7FUANQ2WU4GK46NPKKKFDUXSVVQ3MGK7GM3QBR7GMOD563PTQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.462819102 (103h10m50.311426189s) [noise=true , address=104.36.82.13:4186 , piece_id=O5BVVJBIWINWK2GXWWJDETED6T3Z462AAUQWCG6LGC7PVVFSHZBA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.463600977 (103h10m50.312208063s) [noise=true , address=45.91.116.7:4011 , piece_id=GKVDRTZ2TY2ESEKU2FKIJND4LXDMITNWQORE4Z47OQ454UZMM7BA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.464356397 (103h10m50.312963485s) [noise=true , address=77.90.31.13:4626 , piece_id=D63E7JGO5RRTEJGXVUMNEC3UYXHS45HBOB7S67VPCWLHJXNH7OSQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.587088382 (103h10m50.435695454s) [upload=82463]: Next returned nil segment
Jul 23 06:01:08.587107670 (103h10m50.435714757s) [upload=82463]: Waiting for error group managing segments...
Jul 23 06:01:08.609523184 (103h10m50.458130271s) [noise=true , address=77.90.31.14:4661 , piece_id=MA3CRNAREYTT6ZDO7Q7QRSAOVQQ3EM3CHZTWYQYE6QOWNMYQBBQQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.612539671 (103h10m50.46114674s) [noise=true , address=45.91.116.5:4299 , piece_id=K3URA4ZO4R2CAGZLTO6YJTZQOZ6YIOB7XOJ3AQAVGDFIMVU63BIA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.617758941 (103h10m50.466366025s) [noise=true , address=45.91.116.13:4155 , piece_id=KMDEG3QMJNHRF7TETZAFVN3TOZWTQNW7IRTCVI3WLNE7NJAAAPCA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.618513849 (103h10m50.467120922s) [noise=true , address=45.91.116.8:4048 , piece_id=AROG4HVUX7R6MHG3DT3N2FIQJBBEVVEVKQHSAGZCWMISWTFSQMAQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.624184177 (103h10m50.47279126s) [noise=true , address=45.91.116.7:4010 , piece_id=WVMOM2PPTVXY7NYI5UH25PFW6RODIUJWN73YB6FW6UOAMNY4C4SA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.625290831 (103h10m50.473897916s) [noise=true , address=45.91.116.7:4011 , piece_id=GKVDRTZ2TY2ESEKU2FKIJND4LXDMITNWQORE4Z47OQ454UZMM7BA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.628897649 (103h10m50.477504746s) [noise=true , address=45.91.116.5:4298 , piece_id=AIURH3V65NZD3QI3A5YGBJ222ANC4LTHEYFWTOFIC6SPVH3O4CKA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.629542622 (103h10m50.478149697s) [noise=true , address=45.91.116.12:4225 , piece_id=MFM7FUANQ2WU4GK46NPKKKFDUXSVVQ3MGK7GM3QBR7GMOD563PTQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.630003986 (103h10m50.478611073s) [noise=true , address=77.90.31.17:4772 , piece_id=63V72ATK3KI3ADRC4N5HTRIXSEB3DRM7N73MVZNDPZJ7EVO3RZJA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.630441959 (103h10m50.479049047s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQJDPJRZQEJHTED4UJBNSJ2CGAL6FPTLLIIRFTOVBUK65VSPLGA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.630941221 (103h10m50.479548305s) [noise=true , address=77.90.31.12:4592 , piece_id=NAWAKPN4LUPCFZIXDIEPN2GKO3CSGX35LZNHI4QDCGYBYAMWBGOA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.631378970 (103h10m50.47998605s) [noise=true , address=77.90.31.13:4626 , piece_id=D63E7JGO5RRTEJGXVUMNEC3UYXHS45HBOB7S67VPCWLHJXNH7OSQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.631840658 (103h10m50.480447733s) [noise=true , address=77.90.31.17:4771 , piece_id=VCHLVD4PCT6SGJNCFA4LSEDIXSRMX27JH7VLUXDBM2PMIGZ54CNA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.631850523 (103h10m50.480457598s) [seg_pos={0 0}, upload=82463]: Segment reached optimal threshold of 13 pieces.
Jul 23 06:01:08.631982956 (103h10m50.480590031s) [noise=true , address=104.36.82.13:4186 , piece_id=O5BVVJBIWINWK2GXWWJDETED6T3Z462AAUQWCG6LGC7PVVFSHZBA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF
Jul 23 06:01:08.632087803 (103h10m50.480694892s) [noise=true , address=45.91.116.4:4937 , piece_id=DZILXASXCQNDQGTAIYQIY4LCXHENKF6LLWHME5ILDWL3IZOSWMCQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF
Jul 23 06:01:08.633101069 (103h10m50.481708157s) [noise=true , address=45.91.116.11:4119 , piece_id=LOMOQU77BVHAPGZ23ZY6CG63FZFQX2IG5QRY2P46WKSGB5AF5O3Q, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.633132335 (103h10m50.481739428s) [seg_pos={0 0}, upload=82463]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 23 06:01:08.633221956 (103h10m50.481829049s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 23 06:01:08.633272252 (103h10m50.481879341s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 23 06:01:08.710605841 (103h10m50.559212933s) [upload=82463]: Done uploading segments...
Jul 23 06:01:08.710632085 (103h10m50.559239157s) [upload=82463]: Upload finished. err: <nil>
Duration: 565.857062ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................................X.........................] duration:
419.229094ms
FAIL (2025-07-23 06:19:10 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
23 06:00:19.742341858 (103h10m1.590948949s) [noise=true , address=104.36.82.4:4804 , piece_id=SMCQKI5GIWKEHYLNSVUGHWQNBHP3S4C5RBB4E3K5CFMXE54Q4WAQ, seg_pos={0 0}, upload=82455]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g): protocol: expected piece hash; context canceled; EOF
Jul 23 06:00:19.742959346 (103h10m1.59156642s) [noise=true , address=77.90.31.16:4734 , piece_id=NFCPPB6P25YMO5UZRM4CQKMNRV5Q2FXMSR6M4VSTEERBMVNLYJJA, seg_pos={0 0}, upload=82455]: Done uploading piece. err: <nil>
Jul 23 06:00:19.743002232 (103h10m1.591609306s) [seg_pos={0 0}, upload=82455]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 23 06:00:19.743286680 (103h10m1.591893764s) [upload=82455]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 23 06:00:19.743293769 (103h10m1.591900841s) [upload=82455]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 23 06:00:19.815641707 (103h10m1.664248798s) [upload=82455]: Done uploading segments...
Jul 23 06:00:19.815666887 (103h10m1.664273954s) [upload=82455]: Upload finished. err: <nil>
Jul 23 06:01:08.401614022 (103h10m50.250221093s) [upload=82463]: Starting upload
Jul 23 06:01:08.401709823 (103h10m50.250316908s) [upload=82463]: Done starting upload
Jul 23 06:01:08.402138436 (103h10m50.250745505s) [upload=82463]: Uploading segments...
Jul 23 06:01:08.402189044 (103h10m50.250796137s) [upload=82463]: Got next segment. Inline: false
Jul 23 06:01:08.402231416 (103h10m50.250838487s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 23 06:01:08.402237731 (103h10m50.250844804s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 23 06:01:08.450301042 (103h10m50.298908146s) [seg_pos={0 0}, upload=82463]: Begin upload segment...
Jul 23 06:01:08.450364720 (103h10m50.298971798s) [seg_pos={0 0}, upload=82463]: Done begin upload segment.
Jul 23 06:01:08.450425015 (103h10m50.2990321s) [noise=true , address=45.91.116.5:4299 , piece_id=K3URA4ZO4R2CAGZLTO6YJTZQOZ6YIOB7XOJ3AQAVGDFIMVU63BIA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.450431954 (103h10m50.299039031s) [noise=true , address=45.91.116.7:4010 , piece_id=WVMOM2PPTVXY7NYI5UH25PFW6RODIUJWN73YB6FW6UOAMNY4C4SA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.450900413 (103h10m50.299507502s) [noise=true , address=77.90.31.17:4772 , piece_id=63V72ATK3KI3ADRC4N5HTRIXSEB3DRM7N73MVZNDPZJ7EVO3RZJA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.450963993 (103h10m50.299571083s) [noise=true , address=45.91.116.8:4048 , piece_id=AROG4HVUX7R6MHG3DT3N2FIQJBBEVVEVKQHSAGZCWMISWTFSQMAQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.451375091 (103h10m50.299982178s) [noise=true , address=45.91.116.13:4155 , piece_id=KMDEG3QMJNHRF7TETZAFVN3TOZWTQNW7IRTCVI3WLNE7NJAAAPCA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.451400410 (103h10m50.300007509s) [noise=true , address=77.90.31.12:4592 , piece_id=NAWAKPN4LUPCFZIXDIEPN2GKO3CSGX35LZNHI4QDCGYBYAMWBGOA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.451837258 (103h10m50.300444352s) [noise=true , address=77.90.31.17:4771 , piece_id=VCHLVD4PCT6SGJNCFA4LSEDIXSRMX27JH7VLUXDBM2PMIGZ54CNA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.452344092 (103h10m50.300951186s) [noise=true , address=45.91.116.4:4937 , piece_id=DZILXASXCQNDQGTAIYQIY4LCXHENKF6LLWHME5ILDWL3IZOSWMCQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.452773889 (103h10m50.301380982s) [noise=true , address=45.91.116.11:4119 , piece_id=LOMOQU77BVHAPGZ23ZY6CG63FZFQX2IG5QRY2P46WKSGB5AF5O3Q, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.453168858 (103h10m50.301775945s) [noise=true , address=45.91.116.5:4298 , piece_id=AIURH3V65NZD3QI3A5YGBJ222ANC4LTHEYFWTOFIC6SPVH3O4CKA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.453559736 (103h10m50.302166813s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQJDPJRZQEJHTED4UJBNSJ2CGAL6FPTLLIIRFTOVBUK65VSPLGA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.460854171 (103h10m50.309461265s) [noise=true , address=77.90.31.14:4661 , piece_id=MA3CRNAREYTT6ZDO7Q7QRSAOVQQ3EM3CHZTWYQYE6QOWNMYQBBQQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.462111720 (103h10m50.310718803s) [noise=true , address=45.91.116.12:4225 , piece_id=MFM7FUANQ2WU4GK46NPKKKFDUXSVVQ3MGK7GM3QBR7GMOD563PTQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.462819102 (103h10m50.311426189s) [noise=true , address=104.36.82.13:4186 , piece_id=O5BVVJBIWINWK2GXWWJDETED6T3Z462AAUQWCG6LGC7PVVFSHZBA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.463600977 (103h10m50.312208063s) [noise=true , address=45.91.116.7:4011 , piece_id=GKVDRTZ2TY2ESEKU2FKIJND4LXDMITNWQORE4Z47OQ454UZMM7BA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.464356397 (103h10m50.312963485s) [noise=true , address=77.90.31.13:4626 , piece_id=D63E7JGO5RRTEJGXVUMNEC3UYXHS45HBOB7S67VPCWLHJXNH7OSQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.587088382 (103h10m50.435695454s) [upload=82463]: Next returned nil segment
Jul 23 06:01:08.587107670 (103h10m50.435714757s) [upload=82463]: Waiting for error group managing segments...
Jul 23 06:01:08.609523184 (103h10m50.458130271s) [noise=true , address=77.90.31.14:4661 , piece_id=MA3CRNAREYTT6ZDO7Q7QRSAOVQQ3EM3CHZTWYQYE6QOWNMYQBBQQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.612539671 (103h10m50.46114674s) [noise=true , address=45.91.116.5:4299 , piece_id=K3URA4ZO4R2CAGZLTO6YJTZQOZ6YIOB7XOJ3AQAVGDFIMVU63BIA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.617758941 (103h10m50.466366025s) [noise=true , address=45.91.116.13:4155 , piece_id=KMDEG3QMJNHRF7TETZAFVN3TOZWTQNW7IRTCVI3WLNE7NJAAAPCA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.618513849 (103h10m50.467120922s) [noise=true , address=45.91.116.8:4048 , piece_id=AROG4HVUX7R6MHG3DT3N2FIQJBBEVVEVKQHSAGZCWMISWTFSQMAQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.624184177 (103h10m50.47279126s) [noise=true , address=45.91.116.7:4010 , piece_id=WVMOM2PPTVXY7NYI5UH25PFW6RODIUJWN73YB6FW6UOAMNY4C4SA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.625290831 (103h10m50.473897916s) [noise=true , address=45.91.116.7:4011 , piece_id=GKVDRTZ2TY2ESEKU2FKIJND4LXDMITNWQORE4Z47OQ454UZMM7BA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.628897649 (103h10m50.477504746s) [noise=true , address=45.91.116.5:4298 , piece_id=AIURH3V65NZD3QI3A5YGBJ222ANC4LTHEYFWTOFIC6SPVH3O4CKA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.629542622 (103h10m50.478149697s) [noise=true , address=45.91.116.12:4225 , piece_id=MFM7FUANQ2WU4GK46NPKKKFDUXSVVQ3MGK7GM3QBR7GMOD563PTQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.630003986 (103h10m50.478611073s) [noise=true , address=77.90.31.17:4772 , piece_id=63V72ATK3KI3ADRC4N5HTRIXSEB3DRM7N73MVZNDPZJ7EVO3RZJA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.630441959 (103h10m50.479049047s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQJDPJRZQEJHTED4UJBNSJ2CGAL6FPTLLIIRFTOVBUK65VSPLGA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.630941221 (103h10m50.479548305s) [noise=true , address=77.90.31.12:4592 , piece_id=NAWAKPN4LUPCFZIXDIEPN2GKO3CSGX35LZNHI4QDCGYBYAMWBGOA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.631378970 (103h10m50.47998605s) [noise=true , address=77.90.31.13:4626 , piece_id=D63E7JGO5RRTEJGXVUMNEC3UYXHS45HBOB7S67VPCWLHJXNH7OSQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.631840658 (103h10m50.480447733s) [noise=true , address=77.90.31.17:4771 , piece_id=VCHLVD4PCT6SGJNCFA4LSEDIXSRMX27JH7VLUXDBM2PMIGZ54CNA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.631850523 (103h10m50.480457598s) [seg_pos={0 0}, upload=82463]: Segment reached optimal threshold of 13 pieces.
Jul 23 06:01:08.631982956 (103h10m50.480590031s) [noise=true , address=104.36.82.13:4186 , piece_id=O5BVVJBIWINWK2GXWWJDETED6T3Z462AAUQWCG6LGC7PVVFSHZBA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF
Jul 23 06:01:08.632087803 (103h10m50.480694892s) [noise=true , address=45.91.116.4:4937 , piece_id=DZILXASXCQNDQGTAIYQIY4LCXHENKF6LLWHME5ILDWL3IZOSWMCQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF
Jul 23 06:01:08.633101069 (103h10m50.481708157s) [noise=true , address=45.91.116.11:4119 , piece_id=LOMOQU77BVHAPGZ23ZY6CG63FZFQX2IG5QRY2P46WKSGB5AF5O3Q, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.633132335 (103h10m50.481739428s) [seg_pos={0 0}, upload=82463]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 23 06:01:08.633221956 (103h10m50.481829049s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 23 06:01:08.633272252 (103h10m50.481879341s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 23 06:01:08.710605841 (103h10m50.559212933s) [upload=82463]: Done uploading segments...
Jul 23 06:01:08.710632085 (103h10m50.559239157s) [upload=82463]: Upload finished. err: <nil>
Duration: 419.262947ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [......................................X.....................] duration:
758.123259ms
FAIL (2025-07-23 06:18:16 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
s) [upload=82448]: Got next segment. Inline: false
Jul 23 05:59:58.732247463 (103h9m40.580854536s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 23 05:59:58.732253833 (103h9m40.580860918s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 23 05:59:58.787057645 (103h9m40.635664731s) [seg_pos={0 0}, upload=82448]: Begin upload segment...
Jul 23 05:59:58.787102519 (103h9m40.635709584s) [seg_pos={0 0}, upload=82448]: Done begin upload segment.
Jul 23 05:59:58.787136403 (103h9m40.635743482s) [noise=true , address=77.90.31.12:4622 , piece_id=EZJAG75JRZTYILS7545KYAI37YOUTHHIOOSH26KIWMUQ4BCG556A, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787169354 (103h9m40.635776438s) [noise=true , address=79.127.205.235:4680 , piece_id=T3V7PMZXJOHWT77MQ52PUMFNAIAX4DLMTIIVL6XNVTNA565O5WCQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787544550 (103h9m40.636151681s) [noise=true , address=104.36.82.10:4105 , piece_id=GY72JPHTZOGEVFDG3FM5W5ZSYOPSY3BZXEME2Y37VVR2LR37SIUA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787598825 (103h9m40.636205915s) [noise=true , address=79.127.205.246:4280 , piece_id=NPBN75MFQYLCG4XKRJPLHJU5JCYBNQYIFQ7MKKB3JKXLT4TKF7AA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787946084 (103h9m40.636553167s) [noise=true , address=77.90.31.17:4799 , piece_id=RUTSDNZEXKJ5Y7WQBIVTMFUIJRYLL6DDNNJEXZAL5S7CIATWYHBQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788041731 (103h9m40.636648823s) [noise=true , address=77.90.31.11:4569 , piece_id=MTNLU22JHOYH6VSRMWLKQGSCDI5GIADPIXBVVSC2RIJLTHPMXTOA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788383218 (103h9m40.636990305s) [noise=true , address=104.36.82.42:4818 , piece_id=A6U7TDLTAZHE5IILHHCZKIHCQINHCJSWTBYNXJV7GLCWTIBNB2VA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788633554 (103h9m40.637240644s) [noise=true , address=79.127.219.51:4202 , piece_id=LJZ5A5HAHG4CGXT5J2WNSMXD7HCYSCBUEYGAGOES3NWIVV34BXEQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788735479 (103h9m40.637342568s) [noise=true , address=79.127.219.47:4040 , piece_id=BKWXBAVFXQ2RKG27WGPEAWG5UTHBVODW2BB4EGKHV6SN4EFI7KIQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789072444 (103h9m40.637679533s) [noise=true , address=77.90.31.15:4706 , piece_id=C3JOUPYB77PA2EHPSNPMDETERE724PBALBCSZ6MJCTZ6YRPUC2UA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789122202 (103h9m40.637729291s) [noise=true , address=104.36.82.9:4035 , piece_id=74BSWBLVWYQXPUI2PGV5NIJDSE6MSFKWK65TCVV6CNZKVQBQVNEA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789393961 (103h9m40.638001047s) [noise=true , address=45.91.116.10:4103 , piece_id=UGRDLKKFJRNCCDNDQTAY2LLOIS7CGI6KLNDZ7KJDOUNYWFNUAYQQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789591397 (103h9m40.638198479s) [noise=true , address=104.36.82.5:4870 , piece_id=AYTLB2OL7OAI5EO2ZE6LXIQLCGSGDXKDKT6V5HXIPAREK656MDPQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789716087 (103h9m40.638323176s) [noise=true , address=79.127.205.233:4645 , piece_id=6YAS5E7QYEZAJXB6HUY3R4PLHC4Z7EYTMSFWNFWD5AP7HKXOKTUQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789937996 (103h9m40.638545078s) [noise=true , address=104.36.82.53:4121 , piece_id=GJLXLW2A5RZUF4MBUP5AZAWFKTWGBDADIIWVIUBXPQPPGPKYZ4YA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.790288745 (103h9m40.638895835s) [noise=true , address=77.90.31.15:4708 , piece_id=67LX4NZJUKLYHRTSW3RKHHXZRKMGWJ2KV32FDA2KE3DY364CJ7OA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.790730467 (103h9m40.639337554s) [noise=true , address=45.91.116.14:4206 , piece_id=OVX63SL5OGJTT7HWDRIUV4BDVEKN2MD4ORUUK5M4JIEWX4J6VHFA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.791174576 (103h9m40.639781656s) [noise=true , address=45.91.116.13:4176 , piece_id=GOWXIBOORGZWFIBLMG46HQ4J632B3E6I3MVXNLTWPP2TY3IF2JUQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.898860128 (103h9m40.747467214s) [upload=82448]: Next returned nil segment
Jul 23 05:59:58.898881813 (103h9m40.747488906s) [upload=82448]: Waiting for error group managing segments...
Jul 23 05:59:58.905078785 (103h9m40.753685854s) [noise=true , address=45.91.116.14:4206 , piece_id=OVX63SL5OGJTT7HWDRIUV4BDVEKN2MD4ORUUK5M4JIEWX4J6VHFA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.906441982 (103h9m40.755049063s) [noise=true , address=77.90.31.15:4708 , piece_id=67LX4NZJUKLYHRTSW3RKHHXZRKMGWJ2KV32FDA2KE3DY364CJ7OA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.910115252 (103h9m40.758722337s) [noise=true , address=77.90.31.15:4706 , piece_id=C3JOUPYB77PA2EHPSNPMDETERE724PBALBCSZ6MJCTZ6YRPUC2UA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.910652695 (103h9m40.759259762s) [noise=true , address=104.36.82.9:4035 , piece_id=74BSWBLVWYQXPUI2PGV5NIJDSE6MSFKWK65TCVV6CNZKVQBQVNEA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.910672036 (103h9m40.759279117s) [noise=true , address=77.90.31.11:4569 , piece_id=MTNLU22JHOYH6VSRMWLKQGSCDI5GIADPIXBVVSC2RIJLTHPMXTOA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.911483080 (103h9m40.760090146s) [noise=true , address=45.91.116.13:4176 , piece_id=GOWXIBOORGZWFIBLMG46HQ4J632B3E6I3MVXNLTWPP2TY3IF2JUQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.911951970 (103h9m40.760559052s) [noise=true , address=45.91.116.10:4103 , piece_id=UGRDLKKFJRNCCDNDQTAY2LLOIS7CGI6KLNDZ7KJDOUNYWFNUAYQQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.912044937 (103h9m40.760652006s) [noise=true , address=77.90.31.12:4622 , piece_id=EZJAG75JRZTYILS7545KYAI37YOUTHHIOOSH26KIWMUQ4BCG556A, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.912649865 (103h9m40.761256931s) [noise=true , address=77.90.31.17:4799 , piece_id=RUTSDNZEXKJ5Y7WQBIVTMFUIJRYLL6DDNNJEXZAL5S7CIATWYHBQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.913487977 (103h9m40.76209505s) [noise=true , address=79.127.219.47:4040 , piece_id=BKWXBAVFXQ2RKG27WGPEAWG5UTHBVODW2BB4EGKHV6SN4EFI7KIQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.914486860 (103h9m40.763093927s) [noise=true , address=104.36.82.53:4121 , piece_id=GJLXLW2A5RZUF4MBUP5AZAWFKTWGBDADIIWVIUBXPQPPGPKYZ4YA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.918080770 (103h9m40.766687852s) [noise=true , address=104.36.82.5:4870 , piece_id=AYTLB2OL7OAI5EO2ZE6LXIQLCGSGDXKDKT6V5HXIPAREK656MDPQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.924012392 (103h9m40.772619461s) [noise=true , address=79.127.219.51:4202 , piece_id=LJZ5A5HAHG4CGXT5J2WNSMXD7HCYSCBUEYGAGOES3NWIVV34BXEQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.932892483 (103h9m40.78149955s) [noise=true , address=104.36.82.42:4818 , piece_id=A6U7TDLTAZHE5IILHHCZKIHCQINHCJSWTBYNXJV7GLCWTIBNB2VA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.932913099 (103h9m40.781520167s) [seg_pos={0 0}, upload=82448]: Segment reached optimal threshold of 14 pieces.
Jul 23 05:59:58.933104957 (103h9m40.781712037s) [noise=true , address=104.36.82.10:4105 , piece_id=GY72JPHTZOGEVFDG3FM5W5ZSYOPSY3BZXEME2Y37VVR2LR37SIUA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GhDrKMcDEg2RxamqKtiHphpgNdvtZYdHXrjzGsNJFjBFYeEen): protocol: expected piece hash; context canceled; EOF
Jul 23 05:59:58.933505265 (103h9m40.782112345s) [noise=true , address=79.127.205.233:4645 , piece_id=6YAS5E7QYEZAJXB6HUY3R4PLHC4Z7EYTMSFWNFWD5AP7HKXOKTUQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12NR52pweTqZNNMTyc1EHSbifDKB4awrnnb4megtZHwANSrR18T): protocol: read tcp 159.223.143.39:46566->79.127.205.233:4645: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 23 05:59:58.933568113 (103h9m40.782175194s) [noise=true , address=79.127.205.235:4680 , piece_id=T3V7PMZXJOHWT77MQ52PUMFNAIAX4DLMTIIVL6XNVTNA565O5WCQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GGvQQLoBcTniZST8Hz9h8eCLWbU4DrXn66DGy33iGLXaFgJLj): protocol: read tcp 159.223.143.39:45394->79.127.205.235:4680: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 23 05:59:58.933618915 (103h9m40.782225986s) [noise=true , address=79.127.205.246:4280 , piece_id=NPBN75MFQYLCG4XKRJPLHJU5JCYBNQYIFQ7MKKB3JKXLT4TKF7AA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rZDBvhrfNrYQB6h9MGhx1tNtQ7AH9ehUF4SfNMCJRpv4SKrDX): protocol: read tcp 159.223.143.39:45458->79.127.205.246:4280: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 23 05:59:58.933631191 (103h9m40.78223826s) [seg_pos={0 0}, upload=82448]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jul 23 05:59:58.933756265 (103h9m40.782363345s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 23 05:59:58.933762566 (103h9m40.782369645s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 23 05:59:59.025408726 (103h9m40.874015808s) [upload=82448]: Done uploading segments...
Jul 23 05:59:59.025427547 (103h9m40.874034627s) [upload=82448]: Upload finished. err: <nil>
Duration: 758.154115ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................................X........................] duration:
639.693681ms
FAIL (2025-07-23 06:18:11 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
23 06:00:19.742341858 (103h10m1.590948949s) [noise=true , address=104.36.82.4:4804 , piece_id=SMCQKI5GIWKEHYLNSVUGHWQNBHP3S4C5RBB4E3K5CFMXE54Q4WAQ, seg_pos={0 0}, upload=82455]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g): protocol: expected piece hash; context canceled; EOF
Jul 23 06:00:19.742959346 (103h10m1.59156642s) [noise=true , address=77.90.31.16:4734 , piece_id=NFCPPB6P25YMO5UZRM4CQKMNRV5Q2FXMSR6M4VSTEERBMVNLYJJA, seg_pos={0 0}, upload=82455]: Done uploading piece. err: <nil>
Jul 23 06:00:19.743002232 (103h10m1.591609306s) [seg_pos={0 0}, upload=82455]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 23 06:00:19.743286680 (103h10m1.591893764s) [upload=82455]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 23 06:00:19.743293769 (103h10m1.591900841s) [upload=82455]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 23 06:00:19.815641707 (103h10m1.664248798s) [upload=82455]: Done uploading segments...
Jul 23 06:00:19.815666887 (103h10m1.664273954s) [upload=82455]: Upload finished. err: <nil>
Jul 23 06:01:08.401614022 (103h10m50.250221093s) [upload=82463]: Starting upload
Jul 23 06:01:08.401709823 (103h10m50.250316908s) [upload=82463]: Done starting upload
Jul 23 06:01:08.402138436 (103h10m50.250745505s) [upload=82463]: Uploading segments...
Jul 23 06:01:08.402189044 (103h10m50.250796137s) [upload=82463]: Got next segment. Inline: false
Jul 23 06:01:08.402231416 (103h10m50.250838487s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 23 06:01:08.402237731 (103h10m50.250844804s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 23 06:01:08.450301042 (103h10m50.298908146s) [seg_pos={0 0}, upload=82463]: Begin upload segment...
Jul 23 06:01:08.450364720 (103h10m50.298971798s) [seg_pos={0 0}, upload=82463]: Done begin upload segment.
Jul 23 06:01:08.450425015 (103h10m50.2990321s) [noise=true , address=45.91.116.5:4299 , piece_id=K3URA4ZO4R2CAGZLTO6YJTZQOZ6YIOB7XOJ3AQAVGDFIMVU63BIA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.450431954 (103h10m50.299039031s) [noise=true , address=45.91.116.7:4010 , piece_id=WVMOM2PPTVXY7NYI5UH25PFW6RODIUJWN73YB6FW6UOAMNY4C4SA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.450900413 (103h10m50.299507502s) [noise=true , address=77.90.31.17:4772 , piece_id=63V72ATK3KI3ADRC4N5HTRIXSEB3DRM7N73MVZNDPZJ7EVO3RZJA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.450963993 (103h10m50.299571083s) [noise=true , address=45.91.116.8:4048 , piece_id=AROG4HVUX7R6MHG3DT3N2FIQJBBEVVEVKQHSAGZCWMISWTFSQMAQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.451375091 (103h10m50.299982178s) [noise=true , address=45.91.116.13:4155 , piece_id=KMDEG3QMJNHRF7TETZAFVN3TOZWTQNW7IRTCVI3WLNE7NJAAAPCA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.451400410 (103h10m50.300007509s) [noise=true , address=77.90.31.12:4592 , piece_id=NAWAKPN4LUPCFZIXDIEPN2GKO3CSGX35LZNHI4QDCGYBYAMWBGOA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.451837258 (103h10m50.300444352s) [noise=true , address=77.90.31.17:4771 , piece_id=VCHLVD4PCT6SGJNCFA4LSEDIXSRMX27JH7VLUXDBM2PMIGZ54CNA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.452344092 (103h10m50.300951186s) [noise=true , address=45.91.116.4:4937 , piece_id=DZILXASXCQNDQGTAIYQIY4LCXHENKF6LLWHME5ILDWL3IZOSWMCQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.452773889 (103h10m50.301380982s) [noise=true , address=45.91.116.11:4119 , piece_id=LOMOQU77BVHAPGZ23ZY6CG63FZFQX2IG5QRY2P46WKSGB5AF5O3Q, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.453168858 (103h10m50.301775945s) [noise=true , address=45.91.116.5:4298 , piece_id=AIURH3V65NZD3QI3A5YGBJ222ANC4LTHEYFWTOFIC6SPVH3O4CKA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.453559736 (103h10m50.302166813s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQJDPJRZQEJHTED4UJBNSJ2CGAL6FPTLLIIRFTOVBUK65VSPLGA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.460854171 (103h10m50.309461265s) [noise=true , address=77.90.31.14:4661 , piece_id=MA3CRNAREYTT6ZDO7Q7QRSAOVQQ3EM3CHZTWYQYE6QOWNMYQBBQQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.462111720 (103h10m50.310718803s) [noise=true , address=45.91.116.12:4225 , piece_id=MFM7FUANQ2WU4GK46NPKKKFDUXSVVQ3MGK7GM3QBR7GMOD563PTQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.462819102 (103h10m50.311426189s) [noise=true , address=104.36.82.13:4186 , piece_id=O5BVVJBIWINWK2GXWWJDETED6T3Z462AAUQWCG6LGC7PVVFSHZBA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.463600977 (103h10m50.312208063s) [noise=true , address=45.91.116.7:4011 , piece_id=GKVDRTZ2TY2ESEKU2FKIJND4LXDMITNWQORE4Z47OQ454UZMM7BA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.464356397 (103h10m50.312963485s) [noise=true , address=77.90.31.13:4626 , piece_id=D63E7JGO5RRTEJGXVUMNEC3UYXHS45HBOB7S67VPCWLHJXNH7OSQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.587088382 (103h10m50.435695454s) [upload=82463]: Next returned nil segment
Jul 23 06:01:08.587107670 (103h10m50.435714757s) [upload=82463]: Waiting for error group managing segments...
Jul 23 06:01:08.609523184 (103h10m50.458130271s) [noise=true , address=77.90.31.14:4661 , piece_id=MA3CRNAREYTT6ZDO7Q7QRSAOVQQ3EM3CHZTWYQYE6QOWNMYQBBQQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.612539671 (103h10m50.46114674s) [noise=true , address=45.91.116.5:4299 , piece_id=K3URA4ZO4R2CAGZLTO6YJTZQOZ6YIOB7XOJ3AQAVGDFIMVU63BIA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.617758941 (103h10m50.466366025s) [noise=true , address=45.91.116.13:4155 , piece_id=KMDEG3QMJNHRF7TETZAFVN3TOZWTQNW7IRTCVI3WLNE7NJAAAPCA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.618513849 (103h10m50.467120922s) [noise=true , address=45.91.116.8:4048 , piece_id=AROG4HVUX7R6MHG3DT3N2FIQJBBEVVEVKQHSAGZCWMISWTFSQMAQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.624184177 (103h10m50.47279126s) [noise=true , address=45.91.116.7:4010 , piece_id=WVMOM2PPTVXY7NYI5UH25PFW6RODIUJWN73YB6FW6UOAMNY4C4SA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.625290831 (103h10m50.473897916s) [noise=true , address=45.91.116.7:4011 , piece_id=GKVDRTZ2TY2ESEKU2FKIJND4LXDMITNWQORE4Z47OQ454UZMM7BA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.628897649 (103h10m50.477504746s) [noise=true , address=45.91.116.5:4298 , piece_id=AIURH3V65NZD3QI3A5YGBJ222ANC4LTHEYFWTOFIC6SPVH3O4CKA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.629542622 (103h10m50.478149697s) [noise=true , address=45.91.116.12:4225 , piece_id=MFM7FUANQ2WU4GK46NPKKKFDUXSVVQ3MGK7GM3QBR7GMOD563PTQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.630003986 (103h10m50.478611073s) [noise=true , address=77.90.31.17:4772 , piece_id=63V72ATK3KI3ADRC4N5HTRIXSEB3DRM7N73MVZNDPZJ7EVO3RZJA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.630441959 (103h10m50.479049047s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQJDPJRZQEJHTED4UJBNSJ2CGAL6FPTLLIIRFTOVBUK65VSPLGA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.630941221 (103h10m50.479548305s) [noise=true , address=77.90.31.12:4592 , piece_id=NAWAKPN4LUPCFZIXDIEPN2GKO3CSGX35LZNHI4QDCGYBYAMWBGOA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.631378970 (103h10m50.47998605s) [noise=true , address=77.90.31.13:4626 , piece_id=D63E7JGO5RRTEJGXVUMNEC3UYXHS45HBOB7S67VPCWLHJXNH7OSQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.631840658 (103h10m50.480447733s) [noise=true , address=77.90.31.17:4771 , piece_id=VCHLVD4PCT6SGJNCFA4LSEDIXSRMX27JH7VLUXDBM2PMIGZ54CNA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.631850523 (103h10m50.480457598s) [seg_pos={0 0}, upload=82463]: Segment reached optimal threshold of 13 pieces.
Jul 23 06:01:08.631982956 (103h10m50.480590031s) [noise=true , address=104.36.82.13:4186 , piece_id=O5BVVJBIWINWK2GXWWJDETED6T3Z462AAUQWCG6LGC7PVVFSHZBA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF
Jul 23 06:01:08.632087803 (103h10m50.480694892s) [noise=true , address=45.91.116.4:4937 , piece_id=DZILXASXCQNDQGTAIYQIY4LCXHENKF6LLWHME5ILDWL3IZOSWMCQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF
Jul 23 06:01:08.633101069 (103h10m50.481708157s) [noise=true , address=45.91.116.11:4119 , piece_id=LOMOQU77BVHAPGZ23ZY6CG63FZFQX2IG5QRY2P46WKSGB5AF5O3Q, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.633132335 (103h10m50.481739428s) [seg_pos={0 0}, upload=82463]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 23 06:01:08.633221956 (103h10m50.481829049s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 23 06:01:08.633272252 (103h10m50.481879341s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 23 06:01:08.710605841 (103h10m50.559212933s) [upload=82463]: Done uploading segments...
Jul 23 06:01:08.710632085 (103h10m50.559239157s) [upload=82463]: Upload finished. err: <nil>
Duration: 639.746302ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................................X.......................] duration:
605.35408ms
FAIL (2025-07-23 06:17:52 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
23 06:00:19.742341858 (103h10m1.590948949s) [noise=true , address=104.36.82.4:4804 , piece_id=SMCQKI5GIWKEHYLNSVUGHWQNBHP3S4C5RBB4E3K5CFMXE54Q4WAQ, seg_pos={0 0}, upload=82455]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g): protocol: expected piece hash; context canceled; EOF
Jul 23 06:00:19.742959346 (103h10m1.59156642s) [noise=true , address=77.90.31.16:4734 , piece_id=NFCPPB6P25YMO5UZRM4CQKMNRV5Q2FXMSR6M4VSTEERBMVNLYJJA, seg_pos={0 0}, upload=82455]: Done uploading piece. err: <nil>
Jul 23 06:00:19.743002232 (103h10m1.591609306s) [seg_pos={0 0}, upload=82455]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 23 06:00:19.743286680 (103h10m1.591893764s) [upload=82455]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 23 06:00:19.743293769 (103h10m1.591900841s) [upload=82455]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 23 06:00:19.815641707 (103h10m1.664248798s) [upload=82455]: Done uploading segments...
Jul 23 06:00:19.815666887 (103h10m1.664273954s) [upload=82455]: Upload finished. err: <nil>
Jul 23 06:01:08.401614022 (103h10m50.250221093s) [upload=82463]: Starting upload
Jul 23 06:01:08.401709823 (103h10m50.250316908s) [upload=82463]: Done starting upload
Jul 23 06:01:08.402138436 (103h10m50.250745505s) [upload=82463]: Uploading segments...
Jul 23 06:01:08.402189044 (103h10m50.250796137s) [upload=82463]: Got next segment. Inline: false
Jul 23 06:01:08.402231416 (103h10m50.250838487s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 23 06:01:08.402237731 (103h10m50.250844804s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 23 06:01:08.450301042 (103h10m50.298908146s) [seg_pos={0 0}, upload=82463]: Begin upload segment...
Jul 23 06:01:08.450364720 (103h10m50.298971798s) [seg_pos={0 0}, upload=82463]: Done begin upload segment.
Jul 23 06:01:08.450425015 (103h10m50.2990321s) [noise=true , address=45.91.116.5:4299 , piece_id=K3URA4ZO4R2CAGZLTO6YJTZQOZ6YIOB7XOJ3AQAVGDFIMVU63BIA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.450431954 (103h10m50.299039031s) [noise=true , address=45.91.116.7:4010 , piece_id=WVMOM2PPTVXY7NYI5UH25PFW6RODIUJWN73YB6FW6UOAMNY4C4SA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.450900413 (103h10m50.299507502s) [noise=true , address=77.90.31.17:4772 , piece_id=63V72ATK3KI3ADRC4N5HTRIXSEB3DRM7N73MVZNDPZJ7EVO3RZJA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.450963993 (103h10m50.299571083s) [noise=true , address=45.91.116.8:4048 , piece_id=AROG4HVUX7R6MHG3DT3N2FIQJBBEVVEVKQHSAGZCWMISWTFSQMAQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.451375091 (103h10m50.299982178s) [noise=true , address=45.91.116.13:4155 , piece_id=KMDEG3QMJNHRF7TETZAFVN3TOZWTQNW7IRTCVI3WLNE7NJAAAPCA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.451400410 (103h10m50.300007509s) [noise=true , address=77.90.31.12:4592 , piece_id=NAWAKPN4LUPCFZIXDIEPN2GKO3CSGX35LZNHI4QDCGYBYAMWBGOA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.451837258 (103h10m50.300444352s) [noise=true , address=77.90.31.17:4771 , piece_id=VCHLVD4PCT6SGJNCFA4LSEDIXSRMX27JH7VLUXDBM2PMIGZ54CNA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.452344092 (103h10m50.300951186s) [noise=true , address=45.91.116.4:4937 , piece_id=DZILXASXCQNDQGTAIYQIY4LCXHENKF6LLWHME5ILDWL3IZOSWMCQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.452773889 (103h10m50.301380982s) [noise=true , address=45.91.116.11:4119 , piece_id=LOMOQU77BVHAPGZ23ZY6CG63FZFQX2IG5QRY2P46WKSGB5AF5O3Q, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.453168858 (103h10m50.301775945s) [noise=true , address=45.91.116.5:4298 , piece_id=AIURH3V65NZD3QI3A5YGBJ222ANC4LTHEYFWTOFIC6SPVH3O4CKA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.453559736 (103h10m50.302166813s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQJDPJRZQEJHTED4UJBNSJ2CGAL6FPTLLIIRFTOVBUK65VSPLGA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.460854171 (103h10m50.309461265s) [noise=true , address=77.90.31.14:4661 , piece_id=MA3CRNAREYTT6ZDO7Q7QRSAOVQQ3EM3CHZTWYQYE6QOWNMYQBBQQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.462111720 (103h10m50.310718803s) [noise=true , address=45.91.116.12:4225 , piece_id=MFM7FUANQ2WU4GK46NPKKKFDUXSVVQ3MGK7GM3QBR7GMOD563PTQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.462819102 (103h10m50.311426189s) [noise=true , address=104.36.82.13:4186 , piece_id=O5BVVJBIWINWK2GXWWJDETED6T3Z462AAUQWCG6LGC7PVVFSHZBA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.463600977 (103h10m50.312208063s) [noise=true , address=45.91.116.7:4011 , piece_id=GKVDRTZ2TY2ESEKU2FKIJND4LXDMITNWQORE4Z47OQ454UZMM7BA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.464356397 (103h10m50.312963485s) [noise=true , address=77.90.31.13:4626 , piece_id=D63E7JGO5RRTEJGXVUMNEC3UYXHS45HBOB7S67VPCWLHJXNH7OSQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.587088382 (103h10m50.435695454s) [upload=82463]: Next returned nil segment
Jul 23 06:01:08.587107670 (103h10m50.435714757s) [upload=82463]: Waiting for error group managing segments...
Jul 23 06:01:08.609523184 (103h10m50.458130271s) [noise=true , address=77.90.31.14:4661 , piece_id=MA3CRNAREYTT6ZDO7Q7QRSAOVQQ3EM3CHZTWYQYE6QOWNMYQBBQQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.612539671 (103h10m50.46114674s) [noise=true , address=45.91.116.5:4299 , piece_id=K3URA4ZO4R2CAGZLTO6YJTZQOZ6YIOB7XOJ3AQAVGDFIMVU63BIA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.617758941 (103h10m50.466366025s) [noise=true , address=45.91.116.13:4155 , piece_id=KMDEG3QMJNHRF7TETZAFVN3TOZWTQNW7IRTCVI3WLNE7NJAAAPCA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.618513849 (103h10m50.467120922s) [noise=true , address=45.91.116.8:4048 , piece_id=AROG4HVUX7R6MHG3DT3N2FIQJBBEVVEVKQHSAGZCWMISWTFSQMAQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.624184177 (103h10m50.47279126s) [noise=true , address=45.91.116.7:4010 , piece_id=WVMOM2PPTVXY7NYI5UH25PFW6RODIUJWN73YB6FW6UOAMNY4C4SA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.625290831 (103h10m50.473897916s) [noise=true , address=45.91.116.7:4011 , piece_id=GKVDRTZ2TY2ESEKU2FKIJND4LXDMITNWQORE4Z47OQ454UZMM7BA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.628897649 (103h10m50.477504746s) [noise=true , address=45.91.116.5:4298 , piece_id=AIURH3V65NZD3QI3A5YGBJ222ANC4LTHEYFWTOFIC6SPVH3O4CKA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.629542622 (103h10m50.478149697s) [noise=true , address=45.91.116.12:4225 , piece_id=MFM7FUANQ2WU4GK46NPKKKFDUXSVVQ3MGK7GM3QBR7GMOD563PTQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.630003986 (103h10m50.478611073s) [noise=true , address=77.90.31.17:4772 , piece_id=63V72ATK3KI3ADRC4N5HTRIXSEB3DRM7N73MVZNDPZJ7EVO3RZJA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.630441959 (103h10m50.479049047s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQJDPJRZQEJHTED4UJBNSJ2CGAL6FPTLLIIRFTOVBUK65VSPLGA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.630941221 (103h10m50.479548305s) [noise=true , address=77.90.31.12:4592 , piece_id=NAWAKPN4LUPCFZIXDIEPN2GKO3CSGX35LZNHI4QDCGYBYAMWBGOA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.631378970 (103h10m50.47998605s) [noise=true , address=77.90.31.13:4626 , piece_id=D63E7JGO5RRTEJGXVUMNEC3UYXHS45HBOB7S67VPCWLHJXNH7OSQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.631840658 (103h10m50.480447733s) [noise=true , address=77.90.31.17:4771 , piece_id=VCHLVD4PCT6SGJNCFA4LSEDIXSRMX27JH7VLUXDBM2PMIGZ54CNA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.631850523 (103h10m50.480457598s) [seg_pos={0 0}, upload=82463]: Segment reached optimal threshold of 13 pieces.
Jul 23 06:01:08.631982956 (103h10m50.480590031s) [noise=true , address=104.36.82.13:4186 , piece_id=O5BVVJBIWINWK2GXWWJDETED6T3Z462AAUQWCG6LGC7PVVFSHZBA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF
Jul 23 06:01:08.632087803 (103h10m50.480694892s) [noise=true , address=45.91.116.4:4937 , piece_id=DZILXASXCQNDQGTAIYQIY4LCXHENKF6LLWHME5ILDWL3IZOSWMCQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF
Jul 23 06:01:08.633101069 (103h10m50.481708157s) [noise=true , address=45.91.116.11:4119 , piece_id=LOMOQU77BVHAPGZ23ZY6CG63FZFQX2IG5QRY2P46WKSGB5AF5O3Q, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.633132335 (103h10m50.481739428s) [seg_pos={0 0}, upload=82463]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 23 06:01:08.633221956 (103h10m50.481829049s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 23 06:01:08.633272252 (103h10m50.481879341s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 23 06:01:08.710605841 (103h10m50.559212933s) [upload=82463]: Done uploading segments...
Jul 23 06:01:08.710632085 (103h10m50.559239157s) [upload=82463]: Upload finished. err: <nil>
Duration: 605.3838ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.......................................X....................] duration:
510.870723ms
FAIL (2025-07-23 06:17:36 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
s) [upload=82448]: Got next segment. Inline: false
Jul 23 05:59:58.732247463 (103h9m40.580854536s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 23 05:59:58.732253833 (103h9m40.580860918s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 23 05:59:58.787057645 (103h9m40.635664731s) [seg_pos={0 0}, upload=82448]: Begin upload segment...
Jul 23 05:59:58.787102519 (103h9m40.635709584s) [seg_pos={0 0}, upload=82448]: Done begin upload segment.
Jul 23 05:59:58.787136403 (103h9m40.635743482s) [noise=true , address=77.90.31.12:4622 , piece_id=EZJAG75JRZTYILS7545KYAI37YOUTHHIOOSH26KIWMUQ4BCG556A, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787169354 (103h9m40.635776438s) [noise=true , address=79.127.205.235:4680 , piece_id=T3V7PMZXJOHWT77MQ52PUMFNAIAX4DLMTIIVL6XNVTNA565O5WCQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787544550 (103h9m40.636151681s) [noise=true , address=104.36.82.10:4105 , piece_id=GY72JPHTZOGEVFDG3FM5W5ZSYOPSY3BZXEME2Y37VVR2LR37SIUA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787598825 (103h9m40.636205915s) [noise=true , address=79.127.205.246:4280 , piece_id=NPBN75MFQYLCG4XKRJPLHJU5JCYBNQYIFQ7MKKB3JKXLT4TKF7AA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787946084 (103h9m40.636553167s) [noise=true , address=77.90.31.17:4799 , piece_id=RUTSDNZEXKJ5Y7WQBIVTMFUIJRYLL6DDNNJEXZAL5S7CIATWYHBQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788041731 (103h9m40.636648823s) [noise=true , address=77.90.31.11:4569 , piece_id=MTNLU22JHOYH6VSRMWLKQGSCDI5GIADPIXBVVSC2RIJLTHPMXTOA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788383218 (103h9m40.636990305s) [noise=true , address=104.36.82.42:4818 , piece_id=A6U7TDLTAZHE5IILHHCZKIHCQINHCJSWTBYNXJV7GLCWTIBNB2VA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788633554 (103h9m40.637240644s) [noise=true , address=79.127.219.51:4202 , piece_id=LJZ5A5HAHG4CGXT5J2WNSMXD7HCYSCBUEYGAGOES3NWIVV34BXEQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788735479 (103h9m40.637342568s) [noise=true , address=79.127.219.47:4040 , piece_id=BKWXBAVFXQ2RKG27WGPEAWG5UTHBVODW2BB4EGKHV6SN4EFI7KIQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789072444 (103h9m40.637679533s) [noise=true , address=77.90.31.15:4706 , piece_id=C3JOUPYB77PA2EHPSNPMDETERE724PBALBCSZ6MJCTZ6YRPUC2UA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789122202 (103h9m40.637729291s) [noise=true , address=104.36.82.9:4035 , piece_id=74BSWBLVWYQXPUI2PGV5NIJDSE6MSFKWK65TCVV6CNZKVQBQVNEA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789393961 (103h9m40.638001047s) [noise=true , address=45.91.116.10:4103 , piece_id=UGRDLKKFJRNCCDNDQTAY2LLOIS7CGI6KLNDZ7KJDOUNYWFNUAYQQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789591397 (103h9m40.638198479s) [noise=true , address=104.36.82.5:4870 , piece_id=AYTLB2OL7OAI5EO2ZE6LXIQLCGSGDXKDKT6V5HXIPAREK656MDPQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789716087 (103h9m40.638323176s) [noise=true , address=79.127.205.233:4645 , piece_id=6YAS5E7QYEZAJXB6HUY3R4PLHC4Z7EYTMSFWNFWD5AP7HKXOKTUQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789937996 (103h9m40.638545078s) [noise=true , address=104.36.82.53:4121 , piece_id=GJLXLW2A5RZUF4MBUP5AZAWFKTWGBDADIIWVIUBXPQPPGPKYZ4YA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.790288745 (103h9m40.638895835s) [noise=true , address=77.90.31.15:4708 , piece_id=67LX4NZJUKLYHRTSW3RKHHXZRKMGWJ2KV32FDA2KE3DY364CJ7OA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.790730467 (103h9m40.639337554s) [noise=true , address=45.91.116.14:4206 , piece_id=OVX63SL5OGJTT7HWDRIUV4BDVEKN2MD4ORUUK5M4JIEWX4J6VHFA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.791174576 (103h9m40.639781656s) [noise=true , address=45.91.116.13:4176 , piece_id=GOWXIBOORGZWFIBLMG46HQ4J632B3E6I3MVXNLTWPP2TY3IF2JUQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.898860128 (103h9m40.747467214s) [upload=82448]: Next returned nil segment
Jul 23 05:59:58.898881813 (103h9m40.747488906s) [upload=82448]: Waiting for error group managing segments...
Jul 23 05:59:58.905078785 (103h9m40.753685854s) [noise=true , address=45.91.116.14:4206 , piece_id=OVX63SL5OGJTT7HWDRIUV4BDVEKN2MD4ORUUK5M4JIEWX4J6VHFA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.906441982 (103h9m40.755049063s) [noise=true , address=77.90.31.15:4708 , piece_id=67LX4NZJUKLYHRTSW3RKHHXZRKMGWJ2KV32FDA2KE3DY364CJ7OA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.910115252 (103h9m40.758722337s) [noise=true , address=77.90.31.15:4706 , piece_id=C3JOUPYB77PA2EHPSNPMDETERE724PBALBCSZ6MJCTZ6YRPUC2UA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.910652695 (103h9m40.759259762s) [noise=true , address=104.36.82.9:4035 , piece_id=74BSWBLVWYQXPUI2PGV5NIJDSE6MSFKWK65TCVV6CNZKVQBQVNEA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.910672036 (103h9m40.759279117s) [noise=true , address=77.90.31.11:4569 , piece_id=MTNLU22JHOYH6VSRMWLKQGSCDI5GIADPIXBVVSC2RIJLTHPMXTOA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.911483080 (103h9m40.760090146s) [noise=true , address=45.91.116.13:4176 , piece_id=GOWXIBOORGZWFIBLMG46HQ4J632B3E6I3MVXNLTWPP2TY3IF2JUQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.911951970 (103h9m40.760559052s) [noise=true , address=45.91.116.10:4103 , piece_id=UGRDLKKFJRNCCDNDQTAY2LLOIS7CGI6KLNDZ7KJDOUNYWFNUAYQQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.912044937 (103h9m40.760652006s) [noise=true , address=77.90.31.12:4622 , piece_id=EZJAG75JRZTYILS7545KYAI37YOUTHHIOOSH26KIWMUQ4BCG556A, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.912649865 (103h9m40.761256931s) [noise=true , address=77.90.31.17:4799 , piece_id=RUTSDNZEXKJ5Y7WQBIVTMFUIJRYLL6DDNNJEXZAL5S7CIATWYHBQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.913487977 (103h9m40.76209505s) [noise=true , address=79.127.219.47:4040 , piece_id=BKWXBAVFXQ2RKG27WGPEAWG5UTHBVODW2BB4EGKHV6SN4EFI7KIQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.914486860 (103h9m40.763093927s) [noise=true , address=104.36.82.53:4121 , piece_id=GJLXLW2A5RZUF4MBUP5AZAWFKTWGBDADIIWVIUBXPQPPGPKYZ4YA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.918080770 (103h9m40.766687852s) [noise=true , address=104.36.82.5:4870 , piece_id=AYTLB2OL7OAI5EO2ZE6LXIQLCGSGDXKDKT6V5HXIPAREK656MDPQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.924012392 (103h9m40.772619461s) [noise=true , address=79.127.219.51:4202 , piece_id=LJZ5A5HAHG4CGXT5J2WNSMXD7HCYSCBUEYGAGOES3NWIVV34BXEQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.932892483 (103h9m40.78149955s) [noise=true , address=104.36.82.42:4818 , piece_id=A6U7TDLTAZHE5IILHHCZKIHCQINHCJSWTBYNXJV7GLCWTIBNB2VA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.932913099 (103h9m40.781520167s) [seg_pos={0 0}, upload=82448]: Segment reached optimal threshold of 14 pieces.
Jul 23 05:59:58.933104957 (103h9m40.781712037s) [noise=true , address=104.36.82.10:4105 , piece_id=GY72JPHTZOGEVFDG3FM5W5ZSYOPSY3BZXEME2Y37VVR2LR37SIUA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GhDrKMcDEg2RxamqKtiHphpgNdvtZYdHXrjzGsNJFjBFYeEen): protocol: expected piece hash; context canceled; EOF
Jul 23 05:59:58.933505265 (103h9m40.782112345s) [noise=true , address=79.127.205.233:4645 , piece_id=6YAS5E7QYEZAJXB6HUY3R4PLHC4Z7EYTMSFWNFWD5AP7HKXOKTUQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12NR52pweTqZNNMTyc1EHSbifDKB4awrnnb4megtZHwANSrR18T): protocol: read tcp 159.223.143.39:46566->79.127.205.233:4645: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 23 05:59:58.933568113 (103h9m40.782175194s) [noise=true , address=79.127.205.235:4680 , piece_id=T3V7PMZXJOHWT77MQ52PUMFNAIAX4DLMTIIVL6XNVTNA565O5WCQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GGvQQLoBcTniZST8Hz9h8eCLWbU4DrXn66DGy33iGLXaFgJLj): protocol: read tcp 159.223.143.39:45394->79.127.205.235:4680: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 23 05:59:58.933618915 (103h9m40.782225986s) [noise=true , address=79.127.205.246:4280 , piece_id=NPBN75MFQYLCG4XKRJPLHJU5JCYBNQYIFQ7MKKB3JKXLT4TKF7AA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rZDBvhrfNrYQB6h9MGhx1tNtQ7AH9ehUF4SfNMCJRpv4SKrDX): protocol: read tcp 159.223.143.39:45458->79.127.205.246:4280: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 23 05:59:58.933631191 (103h9m40.78223826s) [seg_pos={0 0}, upload=82448]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jul 23 05:59:58.933756265 (103h9m40.782363345s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 23 05:59:58.933762566 (103h9m40.782369645s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 23 05:59:59.025408726 (103h9m40.874015808s) [upload=82448]: Done uploading segments...
Jul 23 05:59:59.025427547 (103h9m40.874034627s) [upload=82448]: Upload finished. err: <nil>
Duration: 510.910563ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................................X......................] duration:
576.632305ms
FAIL (2025-07-23 06:16:56 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
23 06:00:19.742341858 (103h10m1.590948949s) [noise=true , address=104.36.82.4:4804 , piece_id=SMCQKI5GIWKEHYLNSVUGHWQNBHP3S4C5RBB4E3K5CFMXE54Q4WAQ, seg_pos={0 0}, upload=82455]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g): protocol: expected piece hash; context canceled; EOF
Jul 23 06:00:19.742959346 (103h10m1.59156642s) [noise=true , address=77.90.31.16:4734 , piece_id=NFCPPB6P25YMO5UZRM4CQKMNRV5Q2FXMSR6M4VSTEERBMVNLYJJA, seg_pos={0 0}, upload=82455]: Done uploading piece. err: <nil>
Jul 23 06:00:19.743002232 (103h10m1.591609306s) [seg_pos={0 0}, upload=82455]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 23 06:00:19.743286680 (103h10m1.591893764s) [upload=82455]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 23 06:00:19.743293769 (103h10m1.591900841s) [upload=82455]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 23 06:00:19.815641707 (103h10m1.664248798s) [upload=82455]: Done uploading segments...
Jul 23 06:00:19.815666887 (103h10m1.664273954s) [upload=82455]: Upload finished. err: <nil>
Jul 23 06:01:08.401614022 (103h10m50.250221093s) [upload=82463]: Starting upload
Jul 23 06:01:08.401709823 (103h10m50.250316908s) [upload=82463]: Done starting upload
Jul 23 06:01:08.402138436 (103h10m50.250745505s) [upload=82463]: Uploading segments...
Jul 23 06:01:08.402189044 (103h10m50.250796137s) [upload=82463]: Got next segment. Inline: false
Jul 23 06:01:08.402231416 (103h10m50.250838487s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 23 06:01:08.402237731 (103h10m50.250844804s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 23 06:01:08.450301042 (103h10m50.298908146s) [seg_pos={0 0}, upload=82463]: Begin upload segment...
Jul 23 06:01:08.450364720 (103h10m50.298971798s) [seg_pos={0 0}, upload=82463]: Done begin upload segment.
Jul 23 06:01:08.450425015 (103h10m50.2990321s) [noise=true , address=45.91.116.5:4299 , piece_id=K3URA4ZO4R2CAGZLTO6YJTZQOZ6YIOB7XOJ3AQAVGDFIMVU63BIA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.450431954 (103h10m50.299039031s) [noise=true , address=45.91.116.7:4010 , piece_id=WVMOM2PPTVXY7NYI5UH25PFW6RODIUJWN73YB6FW6UOAMNY4C4SA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.450900413 (103h10m50.299507502s) [noise=true , address=77.90.31.17:4772 , piece_id=63V72ATK3KI3ADRC4N5HTRIXSEB3DRM7N73MVZNDPZJ7EVO3RZJA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.450963993 (103h10m50.299571083s) [noise=true , address=45.91.116.8:4048 , piece_id=AROG4HVUX7R6MHG3DT3N2FIQJBBEVVEVKQHSAGZCWMISWTFSQMAQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.451375091 (103h10m50.299982178s) [noise=true , address=45.91.116.13:4155 , piece_id=KMDEG3QMJNHRF7TETZAFVN3TOZWTQNW7IRTCVI3WLNE7NJAAAPCA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.451400410 (103h10m50.300007509s) [noise=true , address=77.90.31.12:4592 , piece_id=NAWAKPN4LUPCFZIXDIEPN2GKO3CSGX35LZNHI4QDCGYBYAMWBGOA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.451837258 (103h10m50.300444352s) [noise=true , address=77.90.31.17:4771 , piece_id=VCHLVD4PCT6SGJNCFA4LSEDIXSRMX27JH7VLUXDBM2PMIGZ54CNA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.452344092 (103h10m50.300951186s) [noise=true , address=45.91.116.4:4937 , piece_id=DZILXASXCQNDQGTAIYQIY4LCXHENKF6LLWHME5ILDWL3IZOSWMCQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.452773889 (103h10m50.301380982s) [noise=true , address=45.91.116.11:4119 , piece_id=LOMOQU77BVHAPGZ23ZY6CG63FZFQX2IG5QRY2P46WKSGB5AF5O3Q, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.453168858 (103h10m50.301775945s) [noise=true , address=45.91.116.5:4298 , piece_id=AIURH3V65NZD3QI3A5YGBJ222ANC4LTHEYFWTOFIC6SPVH3O4CKA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.453559736 (103h10m50.302166813s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQJDPJRZQEJHTED4UJBNSJ2CGAL6FPTLLIIRFTOVBUK65VSPLGA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.460854171 (103h10m50.309461265s) [noise=true , address=77.90.31.14:4661 , piece_id=MA3CRNAREYTT6ZDO7Q7QRSAOVQQ3EM3CHZTWYQYE6QOWNMYQBBQQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.462111720 (103h10m50.310718803s) [noise=true , address=45.91.116.12:4225 , piece_id=MFM7FUANQ2WU4GK46NPKKKFDUXSVVQ3MGK7GM3QBR7GMOD563PTQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.462819102 (103h10m50.311426189s) [noise=true , address=104.36.82.13:4186 , piece_id=O5BVVJBIWINWK2GXWWJDETED6T3Z462AAUQWCG6LGC7PVVFSHZBA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.463600977 (103h10m50.312208063s) [noise=true , address=45.91.116.7:4011 , piece_id=GKVDRTZ2TY2ESEKU2FKIJND4LXDMITNWQORE4Z47OQ454UZMM7BA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.464356397 (103h10m50.312963485s) [noise=true , address=77.90.31.13:4626 , piece_id=D63E7JGO5RRTEJGXVUMNEC3UYXHS45HBOB7S67VPCWLHJXNH7OSQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.587088382 (103h10m50.435695454s) [upload=82463]: Next returned nil segment
Jul 23 06:01:08.587107670 (103h10m50.435714757s) [upload=82463]: Waiting for error group managing segments...
Jul 23 06:01:08.609523184 (103h10m50.458130271s) [noise=true , address=77.90.31.14:4661 , piece_id=MA3CRNAREYTT6ZDO7Q7QRSAOVQQ3EM3CHZTWYQYE6QOWNMYQBBQQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.612539671 (103h10m50.46114674s) [noise=true , address=45.91.116.5:4299 , piece_id=K3URA4ZO4R2CAGZLTO6YJTZQOZ6YIOB7XOJ3AQAVGDFIMVU63BIA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.617758941 (103h10m50.466366025s) [noise=true , address=45.91.116.13:4155 , piece_id=KMDEG3QMJNHRF7TETZAFVN3TOZWTQNW7IRTCVI3WLNE7NJAAAPCA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.618513849 (103h10m50.467120922s) [noise=true , address=45.91.116.8:4048 , piece_id=AROG4HVUX7R6MHG3DT3N2FIQJBBEVVEVKQHSAGZCWMISWTFSQMAQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.624184177 (103h10m50.47279126s) [noise=true , address=45.91.116.7:4010 , piece_id=WVMOM2PPTVXY7NYI5UH25PFW6RODIUJWN73YB6FW6UOAMNY4C4SA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.625290831 (103h10m50.473897916s) [noise=true , address=45.91.116.7:4011 , piece_id=GKVDRTZ2TY2ESEKU2FKIJND4LXDMITNWQORE4Z47OQ454UZMM7BA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.628897649 (103h10m50.477504746s) [noise=true , address=45.91.116.5:4298 , piece_id=AIURH3V65NZD3QI3A5YGBJ222ANC4LTHEYFWTOFIC6SPVH3O4CKA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.629542622 (103h10m50.478149697s) [noise=true , address=45.91.116.12:4225 , piece_id=MFM7FUANQ2WU4GK46NPKKKFDUXSVVQ3MGK7GM3QBR7GMOD563PTQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.630003986 (103h10m50.478611073s) [noise=true , address=77.90.31.17:4772 , piece_id=63V72ATK3KI3ADRC4N5HTRIXSEB3DRM7N73MVZNDPZJ7EVO3RZJA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.630441959 (103h10m50.479049047s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQJDPJRZQEJHTED4UJBNSJ2CGAL6FPTLLIIRFTOVBUK65VSPLGA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.630941221 (103h10m50.479548305s) [noise=true , address=77.90.31.12:4592 , piece_id=NAWAKPN4LUPCFZIXDIEPN2GKO3CSGX35LZNHI4QDCGYBYAMWBGOA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.631378970 (103h10m50.47998605s) [noise=true , address=77.90.31.13:4626 , piece_id=D63E7JGO5RRTEJGXVUMNEC3UYXHS45HBOB7S67VPCWLHJXNH7OSQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.631840658 (103h10m50.480447733s) [noise=true , address=77.90.31.17:4771 , piece_id=VCHLVD4PCT6SGJNCFA4LSEDIXSRMX27JH7VLUXDBM2PMIGZ54CNA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.631850523 (103h10m50.480457598s) [seg_pos={0 0}, upload=82463]: Segment reached optimal threshold of 13 pieces.
Jul 23 06:01:08.631982956 (103h10m50.480590031s) [noise=true , address=104.36.82.13:4186 , piece_id=O5BVVJBIWINWK2GXWWJDETED6T3Z462AAUQWCG6LGC7PVVFSHZBA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF
Jul 23 06:01:08.632087803 (103h10m50.480694892s) [noise=true , address=45.91.116.4:4937 , piece_id=DZILXASXCQNDQGTAIYQIY4LCXHENKF6LLWHME5ILDWL3IZOSWMCQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF
Jul 23 06:01:08.633101069 (103h10m50.481708157s) [noise=true , address=45.91.116.11:4119 , piece_id=LOMOQU77BVHAPGZ23ZY6CG63FZFQX2IG5QRY2P46WKSGB5AF5O3Q, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.633132335 (103h10m50.481739428s) [seg_pos={0 0}, upload=82463]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 23 06:01:08.633221956 (103h10m50.481829049s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 23 06:01:08.633272252 (103h10m50.481879341s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 23 06:01:08.710605841 (103h10m50.559212933s) [upload=82463]: Done uploading segments...
Jul 23 06:01:08.710632085 (103h10m50.559239157s) [upload=82463]: Upload finished. err: <nil>
Duration: 576.681613ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................................X.....................] duration:
550.792163ms
FAIL (2025-07-23 06:16:33 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
23 06:00:19.742341858 (103h10m1.590948949s) [noise=true , address=104.36.82.4:4804 , piece_id=SMCQKI5GIWKEHYLNSVUGHWQNBHP3S4C5RBB4E3K5CFMXE54Q4WAQ, seg_pos={0 0}, upload=82455]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g): protocol: expected piece hash; context canceled; EOF
Jul 23 06:00:19.742959346 (103h10m1.59156642s) [noise=true , address=77.90.31.16:4734 , piece_id=NFCPPB6P25YMO5UZRM4CQKMNRV5Q2FXMSR6M4VSTEERBMVNLYJJA, seg_pos={0 0}, upload=82455]: Done uploading piece. err: <nil>
Jul 23 06:00:19.743002232 (103h10m1.591609306s) [seg_pos={0 0}, upload=82455]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 23 06:00:19.743286680 (103h10m1.591893764s) [upload=82455]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 23 06:00:19.743293769 (103h10m1.591900841s) [upload=82455]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 23 06:00:19.815641707 (103h10m1.664248798s) [upload=82455]: Done uploading segments...
Jul 23 06:00:19.815666887 (103h10m1.664273954s) [upload=82455]: Upload finished. err: <nil>
Jul 23 06:01:08.401614022 (103h10m50.250221093s) [upload=82463]: Starting upload
Jul 23 06:01:08.401709823 (103h10m50.250316908s) [upload=82463]: Done starting upload
Jul 23 06:01:08.402138436 (103h10m50.250745505s) [upload=82463]: Uploading segments...
Jul 23 06:01:08.402189044 (103h10m50.250796137s) [upload=82463]: Got next segment. Inline: false
Jul 23 06:01:08.402231416 (103h10m50.250838487s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 23 06:01:08.402237731 (103h10m50.250844804s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 23 06:01:08.450301042 (103h10m50.298908146s) [seg_pos={0 0}, upload=82463]: Begin upload segment...
Jul 23 06:01:08.450364720 (103h10m50.298971798s) [seg_pos={0 0}, upload=82463]: Done begin upload segment.
Jul 23 06:01:08.450425015 (103h10m50.2990321s) [noise=true , address=45.91.116.5:4299 , piece_id=K3URA4ZO4R2CAGZLTO6YJTZQOZ6YIOB7XOJ3AQAVGDFIMVU63BIA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.450431954 (103h10m50.299039031s) [noise=true , address=45.91.116.7:4010 , piece_id=WVMOM2PPTVXY7NYI5UH25PFW6RODIUJWN73YB6FW6UOAMNY4C4SA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.450900413 (103h10m50.299507502s) [noise=true , address=77.90.31.17:4772 , piece_id=63V72ATK3KI3ADRC4N5HTRIXSEB3DRM7N73MVZNDPZJ7EVO3RZJA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.450963993 (103h10m50.299571083s) [noise=true , address=45.91.116.8:4048 , piece_id=AROG4HVUX7R6MHG3DT3N2FIQJBBEVVEVKQHSAGZCWMISWTFSQMAQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.451375091 (103h10m50.299982178s) [noise=true , address=45.91.116.13:4155 , piece_id=KMDEG3QMJNHRF7TETZAFVN3TOZWTQNW7IRTCVI3WLNE7NJAAAPCA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.451400410 (103h10m50.300007509s) [noise=true , address=77.90.31.12:4592 , piece_id=NAWAKPN4LUPCFZIXDIEPN2GKO3CSGX35LZNHI4QDCGYBYAMWBGOA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.451837258 (103h10m50.300444352s) [noise=true , address=77.90.31.17:4771 , piece_id=VCHLVD4PCT6SGJNCFA4LSEDIXSRMX27JH7VLUXDBM2PMIGZ54CNA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.452344092 (103h10m50.300951186s) [noise=true , address=45.91.116.4:4937 , piece_id=DZILXASXCQNDQGTAIYQIY4LCXHENKF6LLWHME5ILDWL3IZOSWMCQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.452773889 (103h10m50.301380982s) [noise=true , address=45.91.116.11:4119 , piece_id=LOMOQU77BVHAPGZ23ZY6CG63FZFQX2IG5QRY2P46WKSGB5AF5O3Q, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.453168858 (103h10m50.301775945s) [noise=true , address=45.91.116.5:4298 , piece_id=AIURH3V65NZD3QI3A5YGBJ222ANC4LTHEYFWTOFIC6SPVH3O4CKA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.453559736 (103h10m50.302166813s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQJDPJRZQEJHTED4UJBNSJ2CGAL6FPTLLIIRFTOVBUK65VSPLGA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.460854171 (103h10m50.309461265s) [noise=true , address=77.90.31.14:4661 , piece_id=MA3CRNAREYTT6ZDO7Q7QRSAOVQQ3EM3CHZTWYQYE6QOWNMYQBBQQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.462111720 (103h10m50.310718803s) [noise=true , address=45.91.116.12:4225 , piece_id=MFM7FUANQ2WU4GK46NPKKKFDUXSVVQ3MGK7GM3QBR7GMOD563PTQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.462819102 (103h10m50.311426189s) [noise=true , address=104.36.82.13:4186 , piece_id=O5BVVJBIWINWK2GXWWJDETED6T3Z462AAUQWCG6LGC7PVVFSHZBA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.463600977 (103h10m50.312208063s) [noise=true , address=45.91.116.7:4011 , piece_id=GKVDRTZ2TY2ESEKU2FKIJND4LXDMITNWQORE4Z47OQ454UZMM7BA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.464356397 (103h10m50.312963485s) [noise=true , address=77.90.31.13:4626 , piece_id=D63E7JGO5RRTEJGXVUMNEC3UYXHS45HBOB7S67VPCWLHJXNH7OSQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.587088382 (103h10m50.435695454s) [upload=82463]: Next returned nil segment
Jul 23 06:01:08.587107670 (103h10m50.435714757s) [upload=82463]: Waiting for error group managing segments...
Jul 23 06:01:08.609523184 (103h10m50.458130271s) [noise=true , address=77.90.31.14:4661 , piece_id=MA3CRNAREYTT6ZDO7Q7QRSAOVQQ3EM3CHZTWYQYE6QOWNMYQBBQQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.612539671 (103h10m50.46114674s) [noise=true , address=45.91.116.5:4299 , piece_id=K3URA4ZO4R2CAGZLTO6YJTZQOZ6YIOB7XOJ3AQAVGDFIMVU63BIA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.617758941 (103h10m50.466366025s) [noise=true , address=45.91.116.13:4155 , piece_id=KMDEG3QMJNHRF7TETZAFVN3TOZWTQNW7IRTCVI3WLNE7NJAAAPCA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.618513849 (103h10m50.467120922s) [noise=true , address=45.91.116.8:4048 , piece_id=AROG4HVUX7R6MHG3DT3N2FIQJBBEVVEVKQHSAGZCWMISWTFSQMAQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.624184177 (103h10m50.47279126s) [noise=true , address=45.91.116.7:4010 , piece_id=WVMOM2PPTVXY7NYI5UH25PFW6RODIUJWN73YB6FW6UOAMNY4C4SA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.625290831 (103h10m50.473897916s) [noise=true , address=45.91.116.7:4011 , piece_id=GKVDRTZ2TY2ESEKU2FKIJND4LXDMITNWQORE4Z47OQ454UZMM7BA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.628897649 (103h10m50.477504746s) [noise=true , address=45.91.116.5:4298 , piece_id=AIURH3V65NZD3QI3A5YGBJ222ANC4LTHEYFWTOFIC6SPVH3O4CKA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.629542622 (103h10m50.478149697s) [noise=true , address=45.91.116.12:4225 , piece_id=MFM7FUANQ2WU4GK46NPKKKFDUXSVVQ3MGK7GM3QBR7GMOD563PTQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.630003986 (103h10m50.478611073s) [noise=true , address=77.90.31.17:4772 , piece_id=63V72ATK3KI3ADRC4N5HTRIXSEB3DRM7N73MVZNDPZJ7EVO3RZJA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.630441959 (103h10m50.479049047s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQJDPJRZQEJHTED4UJBNSJ2CGAL6FPTLLIIRFTOVBUK65VSPLGA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.630941221 (103h10m50.479548305s) [noise=true , address=77.90.31.12:4592 , piece_id=NAWAKPN4LUPCFZIXDIEPN2GKO3CSGX35LZNHI4QDCGYBYAMWBGOA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.631378970 (103h10m50.47998605s) [noise=true , address=77.90.31.13:4626 , piece_id=D63E7JGO5RRTEJGXVUMNEC3UYXHS45HBOB7S67VPCWLHJXNH7OSQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.631840658 (103h10m50.480447733s) [noise=true , address=77.90.31.17:4771 , piece_id=VCHLVD4PCT6SGJNCFA4LSEDIXSRMX27JH7VLUXDBM2PMIGZ54CNA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.631850523 (103h10m50.480457598s) [seg_pos={0 0}, upload=82463]: Segment reached optimal threshold of 13 pieces.
Jul 23 06:01:08.631982956 (103h10m50.480590031s) [noise=true , address=104.36.82.13:4186 , piece_id=O5BVVJBIWINWK2GXWWJDETED6T3Z462AAUQWCG6LGC7PVVFSHZBA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF
Jul 23 06:01:08.632087803 (103h10m50.480694892s) [noise=true , address=45.91.116.4:4937 , piece_id=DZILXASXCQNDQGTAIYQIY4LCXHENKF6LLWHME5ILDWL3IZOSWMCQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF
Jul 23 06:01:08.633101069 (103h10m50.481708157s) [noise=true , address=45.91.116.11:4119 , piece_id=LOMOQU77BVHAPGZ23ZY6CG63FZFQX2IG5QRY2P46WKSGB5AF5O3Q, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.633132335 (103h10m50.481739428s) [seg_pos={0 0}, upload=82463]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 23 06:01:08.633221956 (103h10m50.481829049s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 23 06:01:08.633272252 (103h10m50.481879341s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 23 06:01:08.710605841 (103h10m50.559212933s) [upload=82463]: Done uploading segments...
Jul 23 06:01:08.710632085 (103h10m50.559239157s) [upload=82463]: Upload finished. err: <nil>
Duration: 550.822021ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [........................................X...................] duration:
562.759168ms
FAIL (2025-07-23 06:16:24 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
s) [upload=82448]: Got next segment. Inline: false
Jul 23 05:59:58.732247463 (103h9m40.580854536s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 23 05:59:58.732253833 (103h9m40.580860918s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 23 05:59:58.787057645 (103h9m40.635664731s) [seg_pos={0 0}, upload=82448]: Begin upload segment...
Jul 23 05:59:58.787102519 (103h9m40.635709584s) [seg_pos={0 0}, upload=82448]: Done begin upload segment.
Jul 23 05:59:58.787136403 (103h9m40.635743482s) [noise=true , address=77.90.31.12:4622 , piece_id=EZJAG75JRZTYILS7545KYAI37YOUTHHIOOSH26KIWMUQ4BCG556A, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787169354 (103h9m40.635776438s) [noise=true , address=79.127.205.235:4680 , piece_id=T3V7PMZXJOHWT77MQ52PUMFNAIAX4DLMTIIVL6XNVTNA565O5WCQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787544550 (103h9m40.636151681s) [noise=true , address=104.36.82.10:4105 , piece_id=GY72JPHTZOGEVFDG3FM5W5ZSYOPSY3BZXEME2Y37VVR2LR37SIUA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787598825 (103h9m40.636205915s) [noise=true , address=79.127.205.246:4280 , piece_id=NPBN75MFQYLCG4XKRJPLHJU5JCYBNQYIFQ7MKKB3JKXLT4TKF7AA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787946084 (103h9m40.636553167s) [noise=true , address=77.90.31.17:4799 , piece_id=RUTSDNZEXKJ5Y7WQBIVTMFUIJRYLL6DDNNJEXZAL5S7CIATWYHBQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788041731 (103h9m40.636648823s) [noise=true , address=77.90.31.11:4569 , piece_id=MTNLU22JHOYH6VSRMWLKQGSCDI5GIADPIXBVVSC2RIJLTHPMXTOA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788383218 (103h9m40.636990305s) [noise=true , address=104.36.82.42:4818 , piece_id=A6U7TDLTAZHE5IILHHCZKIHCQINHCJSWTBYNXJV7GLCWTIBNB2VA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788633554 (103h9m40.637240644s) [noise=true , address=79.127.219.51:4202 , piece_id=LJZ5A5HAHG4CGXT5J2WNSMXD7HCYSCBUEYGAGOES3NWIVV34BXEQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788735479 (103h9m40.637342568s) [noise=true , address=79.127.219.47:4040 , piece_id=BKWXBAVFXQ2RKG27WGPEAWG5UTHBVODW2BB4EGKHV6SN4EFI7KIQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789072444 (103h9m40.637679533s) [noise=true , address=77.90.31.15:4706 , piece_id=C3JOUPYB77PA2EHPSNPMDETERE724PBALBCSZ6MJCTZ6YRPUC2UA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789122202 (103h9m40.637729291s) [noise=true , address=104.36.82.9:4035 , piece_id=74BSWBLVWYQXPUI2PGV5NIJDSE6MSFKWK65TCVV6CNZKVQBQVNEA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789393961 (103h9m40.638001047s) [noise=true , address=45.91.116.10:4103 , piece_id=UGRDLKKFJRNCCDNDQTAY2LLOIS7CGI6KLNDZ7KJDOUNYWFNUAYQQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789591397 (103h9m40.638198479s) [noise=true , address=104.36.82.5:4870 , piece_id=AYTLB2OL7OAI5EO2ZE6LXIQLCGSGDXKDKT6V5HXIPAREK656MDPQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789716087 (103h9m40.638323176s) [noise=true , address=79.127.205.233:4645 , piece_id=6YAS5E7QYEZAJXB6HUY3R4PLHC4Z7EYTMSFWNFWD5AP7HKXOKTUQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789937996 (103h9m40.638545078s) [noise=true , address=104.36.82.53:4121 , piece_id=GJLXLW2A5RZUF4MBUP5AZAWFKTWGBDADIIWVIUBXPQPPGPKYZ4YA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.790288745 (103h9m40.638895835s) [noise=true , address=77.90.31.15:4708 , piece_id=67LX4NZJUKLYHRTSW3RKHHXZRKMGWJ2KV32FDA2KE3DY364CJ7OA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.790730467 (103h9m40.639337554s) [noise=true , address=45.91.116.14:4206 , piece_id=OVX63SL5OGJTT7HWDRIUV4BDVEKN2MD4ORUUK5M4JIEWX4J6VHFA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.791174576 (103h9m40.639781656s) [noise=true , address=45.91.116.13:4176 , piece_id=GOWXIBOORGZWFIBLMG46HQ4J632B3E6I3MVXNLTWPP2TY3IF2JUQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.898860128 (103h9m40.747467214s) [upload=82448]: Next returned nil segment
Jul 23 05:59:58.898881813 (103h9m40.747488906s) [upload=82448]: Waiting for error group managing segments...
Jul 23 05:59:58.905078785 (103h9m40.753685854s) [noise=true , address=45.91.116.14:4206 , piece_id=OVX63SL5OGJTT7HWDRIUV4BDVEKN2MD4ORUUK5M4JIEWX4J6VHFA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.906441982 (103h9m40.755049063s) [noise=true , address=77.90.31.15:4708 , piece_id=67LX4NZJUKLYHRTSW3RKHHXZRKMGWJ2KV32FDA2KE3DY364CJ7OA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.910115252 (103h9m40.758722337s) [noise=true , address=77.90.31.15:4706 , piece_id=C3JOUPYB77PA2EHPSNPMDETERE724PBALBCSZ6MJCTZ6YRPUC2UA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.910652695 (103h9m40.759259762s) [noise=true , address=104.36.82.9:4035 , piece_id=74BSWBLVWYQXPUI2PGV5NIJDSE6MSFKWK65TCVV6CNZKVQBQVNEA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.910672036 (103h9m40.759279117s) [noise=true , address=77.90.31.11:4569 , piece_id=MTNLU22JHOYH6VSRMWLKQGSCDI5GIADPIXBVVSC2RIJLTHPMXTOA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.911483080 (103h9m40.760090146s) [noise=true , address=45.91.116.13:4176 , piece_id=GOWXIBOORGZWFIBLMG46HQ4J632B3E6I3MVXNLTWPP2TY3IF2JUQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.911951970 (103h9m40.760559052s) [noise=true , address=45.91.116.10:4103 , piece_id=UGRDLKKFJRNCCDNDQTAY2LLOIS7CGI6KLNDZ7KJDOUNYWFNUAYQQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.912044937 (103h9m40.760652006s) [noise=true , address=77.90.31.12:4622 , piece_id=EZJAG75JRZTYILS7545KYAI37YOUTHHIOOSH26KIWMUQ4BCG556A, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.912649865 (103h9m40.761256931s) [noise=true , address=77.90.31.17:4799 , piece_id=RUTSDNZEXKJ5Y7WQBIVTMFUIJRYLL6DDNNJEXZAL5S7CIATWYHBQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.913487977 (103h9m40.76209505s) [noise=true , address=79.127.219.47:4040 , piece_id=BKWXBAVFXQ2RKG27WGPEAWG5UTHBVODW2BB4EGKHV6SN4EFI7KIQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.914486860 (103h9m40.763093927s) [noise=true , address=104.36.82.53:4121 , piece_id=GJLXLW2A5RZUF4MBUP5AZAWFKTWGBDADIIWVIUBXPQPPGPKYZ4YA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.918080770 (103h9m40.766687852s) [noise=true , address=104.36.82.5:4870 , piece_id=AYTLB2OL7OAI5EO2ZE6LXIQLCGSGDXKDKT6V5HXIPAREK656MDPQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.924012392 (103h9m40.772619461s) [noise=true , address=79.127.219.51:4202 , piece_id=LJZ5A5HAHG4CGXT5J2WNSMXD7HCYSCBUEYGAGOES3NWIVV34BXEQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.932892483 (103h9m40.78149955s) [noise=true , address=104.36.82.42:4818 , piece_id=A6U7TDLTAZHE5IILHHCZKIHCQINHCJSWTBYNXJV7GLCWTIBNB2VA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.932913099 (103h9m40.781520167s) [seg_pos={0 0}, upload=82448]: Segment reached optimal threshold of 14 pieces.
Jul 23 05:59:58.933104957 (103h9m40.781712037s) [noise=true , address=104.36.82.10:4105 , piece_id=GY72JPHTZOGEVFDG3FM5W5ZSYOPSY3BZXEME2Y37VVR2LR37SIUA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GhDrKMcDEg2RxamqKtiHphpgNdvtZYdHXrjzGsNJFjBFYeEen): protocol: expected piece hash; context canceled; EOF
Jul 23 05:59:58.933505265 (103h9m40.782112345s) [noise=true , address=79.127.205.233:4645 , piece_id=6YAS5E7QYEZAJXB6HUY3R4PLHC4Z7EYTMSFWNFWD5AP7HKXOKTUQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12NR52pweTqZNNMTyc1EHSbifDKB4awrnnb4megtZHwANSrR18T): protocol: read tcp 159.223.143.39:46566->79.127.205.233:4645: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 23 05:59:58.933568113 (103h9m40.782175194s) [noise=true , address=79.127.205.235:4680 , piece_id=T3V7PMZXJOHWT77MQ52PUMFNAIAX4DLMTIIVL6XNVTNA565O5WCQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GGvQQLoBcTniZST8Hz9h8eCLWbU4DrXn66DGy33iGLXaFgJLj): protocol: read tcp 159.223.143.39:45394->79.127.205.235:4680: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 23 05:59:58.933618915 (103h9m40.782225986s) [noise=true , address=79.127.205.246:4280 , piece_id=NPBN75MFQYLCG4XKRJPLHJU5JCYBNQYIFQ7MKKB3JKXLT4TKF7AA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rZDBvhrfNrYQB6h9MGhx1tNtQ7AH9ehUF4SfNMCJRpv4SKrDX): protocol: read tcp 159.223.143.39:45458->79.127.205.246:4280: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 23 05:59:58.933631191 (103h9m40.78223826s) [seg_pos={0 0}, upload=82448]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jul 23 05:59:58.933756265 (103h9m40.782363345s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 23 05:59:58.933762566 (103h9m40.782369645s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 23 05:59:59.025408726 (103h9m40.874015808s) [upload=82448]: Done uploading segments...
Jul 23 05:59:59.025427547 (103h9m40.874034627s) [upload=82448]: Upload finished. err: <nil>
Duration: 562.790151ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......................................X....................] duration:
606.77452ms
FAIL (2025-07-23 06:16:01 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
23 06:00:19.742341858 (103h10m1.590948949s) [noise=true , address=104.36.82.4:4804 , piece_id=SMCQKI5GIWKEHYLNSVUGHWQNBHP3S4C5RBB4E3K5CFMXE54Q4WAQ, seg_pos={0 0}, upload=82455]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g): protocol: expected piece hash; context canceled; EOF
Jul 23 06:00:19.742959346 (103h10m1.59156642s) [noise=true , address=77.90.31.16:4734 , piece_id=NFCPPB6P25YMO5UZRM4CQKMNRV5Q2FXMSR6M4VSTEERBMVNLYJJA, seg_pos={0 0}, upload=82455]: Done uploading piece. err: <nil>
Jul 23 06:00:19.743002232 (103h10m1.591609306s) [seg_pos={0 0}, upload=82455]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 23 06:00:19.743286680 (103h10m1.591893764s) [upload=82455]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 23 06:00:19.743293769 (103h10m1.591900841s) [upload=82455]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 23 06:00:19.815641707 (103h10m1.664248798s) [upload=82455]: Done uploading segments...
Jul 23 06:00:19.815666887 (103h10m1.664273954s) [upload=82455]: Upload finished. err: <nil>
Jul 23 06:01:08.401614022 (103h10m50.250221093s) [upload=82463]: Starting upload
Jul 23 06:01:08.401709823 (103h10m50.250316908s) [upload=82463]: Done starting upload
Jul 23 06:01:08.402138436 (103h10m50.250745505s) [upload=82463]: Uploading segments...
Jul 23 06:01:08.402189044 (103h10m50.250796137s) [upload=82463]: Got next segment. Inline: false
Jul 23 06:01:08.402231416 (103h10m50.250838487s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 23 06:01:08.402237731 (103h10m50.250844804s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 23 06:01:08.450301042 (103h10m50.298908146s) [seg_pos={0 0}, upload=82463]: Begin upload segment...
Jul 23 06:01:08.450364720 (103h10m50.298971798s) [seg_pos={0 0}, upload=82463]: Done begin upload segment.
Jul 23 06:01:08.450425015 (103h10m50.2990321s) [noise=true , address=45.91.116.5:4299 , piece_id=K3URA4ZO4R2CAGZLTO6YJTZQOZ6YIOB7XOJ3AQAVGDFIMVU63BIA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.450431954 (103h10m50.299039031s) [noise=true , address=45.91.116.7:4010 , piece_id=WVMOM2PPTVXY7NYI5UH25PFW6RODIUJWN73YB6FW6UOAMNY4C4SA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.450900413 (103h10m50.299507502s) [noise=true , address=77.90.31.17:4772 , piece_id=63V72ATK3KI3ADRC4N5HTRIXSEB3DRM7N73MVZNDPZJ7EVO3RZJA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.450963993 (103h10m50.299571083s) [noise=true , address=45.91.116.8:4048 , piece_id=AROG4HVUX7R6MHG3DT3N2FIQJBBEVVEVKQHSAGZCWMISWTFSQMAQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.451375091 (103h10m50.299982178s) [noise=true , address=45.91.116.13:4155 , piece_id=KMDEG3QMJNHRF7TETZAFVN3TOZWTQNW7IRTCVI3WLNE7NJAAAPCA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.451400410 (103h10m50.300007509s) [noise=true , address=77.90.31.12:4592 , piece_id=NAWAKPN4LUPCFZIXDIEPN2GKO3CSGX35LZNHI4QDCGYBYAMWBGOA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.451837258 (103h10m50.300444352s) [noise=true , address=77.90.31.17:4771 , piece_id=VCHLVD4PCT6SGJNCFA4LSEDIXSRMX27JH7VLUXDBM2PMIGZ54CNA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.452344092 (103h10m50.300951186s) [noise=true , address=45.91.116.4:4937 , piece_id=DZILXASXCQNDQGTAIYQIY4LCXHENKF6LLWHME5ILDWL3IZOSWMCQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.452773889 (103h10m50.301380982s) [noise=true , address=45.91.116.11:4119 , piece_id=LOMOQU77BVHAPGZ23ZY6CG63FZFQX2IG5QRY2P46WKSGB5AF5O3Q, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.453168858 (103h10m50.301775945s) [noise=true , address=45.91.116.5:4298 , piece_id=AIURH3V65NZD3QI3A5YGBJ222ANC4LTHEYFWTOFIC6SPVH3O4CKA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.453559736 (103h10m50.302166813s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQJDPJRZQEJHTED4UJBNSJ2CGAL6FPTLLIIRFTOVBUK65VSPLGA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.460854171 (103h10m50.309461265s) [noise=true , address=77.90.31.14:4661 , piece_id=MA3CRNAREYTT6ZDO7Q7QRSAOVQQ3EM3CHZTWYQYE6QOWNMYQBBQQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.462111720 (103h10m50.310718803s) [noise=true , address=45.91.116.12:4225 , piece_id=MFM7FUANQ2WU4GK46NPKKKFDUXSVVQ3MGK7GM3QBR7GMOD563PTQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.462819102 (103h10m50.311426189s) [noise=true , address=104.36.82.13:4186 , piece_id=O5BVVJBIWINWK2GXWWJDETED6T3Z462AAUQWCG6LGC7PVVFSHZBA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.463600977 (103h10m50.312208063s) [noise=true , address=45.91.116.7:4011 , piece_id=GKVDRTZ2TY2ESEKU2FKIJND4LXDMITNWQORE4Z47OQ454UZMM7BA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.464356397 (103h10m50.312963485s) [noise=true , address=77.90.31.13:4626 , piece_id=D63E7JGO5RRTEJGXVUMNEC3UYXHS45HBOB7S67VPCWLHJXNH7OSQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.587088382 (103h10m50.435695454s) [upload=82463]: Next returned nil segment
Jul 23 06:01:08.587107670 (103h10m50.435714757s) [upload=82463]: Waiting for error group managing segments...
Jul 23 06:01:08.609523184 (103h10m50.458130271s) [noise=true , address=77.90.31.14:4661 , piece_id=MA3CRNAREYTT6ZDO7Q7QRSAOVQQ3EM3CHZTWYQYE6QOWNMYQBBQQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.612539671 (103h10m50.46114674s) [noise=true , address=45.91.116.5:4299 , piece_id=K3URA4ZO4R2CAGZLTO6YJTZQOZ6YIOB7XOJ3AQAVGDFIMVU63BIA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.617758941 (103h10m50.466366025s) [noise=true , address=45.91.116.13:4155 , piece_id=KMDEG3QMJNHRF7TETZAFVN3TOZWTQNW7IRTCVI3WLNE7NJAAAPCA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.618513849 (103h10m50.467120922s) [noise=true , address=45.91.116.8:4048 , piece_id=AROG4HVUX7R6MHG3DT3N2FIQJBBEVVEVKQHSAGZCWMISWTFSQMAQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.624184177 (103h10m50.47279126s) [noise=true , address=45.91.116.7:4010 , piece_id=WVMOM2PPTVXY7NYI5UH25PFW6RODIUJWN73YB6FW6UOAMNY4C4SA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.625290831 (103h10m50.473897916s) [noise=true , address=45.91.116.7:4011 , piece_id=GKVDRTZ2TY2ESEKU2FKIJND4LXDMITNWQORE4Z47OQ454UZMM7BA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.628897649 (103h10m50.477504746s) [noise=true , address=45.91.116.5:4298 , piece_id=AIURH3V65NZD3QI3A5YGBJ222ANC4LTHEYFWTOFIC6SPVH3O4CKA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.629542622 (103h10m50.478149697s) [noise=true , address=45.91.116.12:4225 , piece_id=MFM7FUANQ2WU4GK46NPKKKFDUXSVVQ3MGK7GM3QBR7GMOD563PTQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.630003986 (103h10m50.478611073s) [noise=true , address=77.90.31.17:4772 , piece_id=63V72ATK3KI3ADRC4N5HTRIXSEB3DRM7N73MVZNDPZJ7EVO3RZJA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.630441959 (103h10m50.479049047s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQJDPJRZQEJHTED4UJBNSJ2CGAL6FPTLLIIRFTOVBUK65VSPLGA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.630941221 (103h10m50.479548305s) [noise=true , address=77.90.31.12:4592 , piece_id=NAWAKPN4LUPCFZIXDIEPN2GKO3CSGX35LZNHI4QDCGYBYAMWBGOA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.631378970 (103h10m50.47998605s) [noise=true , address=77.90.31.13:4626 , piece_id=D63E7JGO5RRTEJGXVUMNEC3UYXHS45HBOB7S67VPCWLHJXNH7OSQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.631840658 (103h10m50.480447733s) [noise=true , address=77.90.31.17:4771 , piece_id=VCHLVD4PCT6SGJNCFA4LSEDIXSRMX27JH7VLUXDBM2PMIGZ54CNA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.631850523 (103h10m50.480457598s) [seg_pos={0 0}, upload=82463]: Segment reached optimal threshold of 13 pieces.
Jul 23 06:01:08.631982956 (103h10m50.480590031s) [noise=true , address=104.36.82.13:4186 , piece_id=O5BVVJBIWINWK2GXWWJDETED6T3Z462AAUQWCG6LGC7PVVFSHZBA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF
Jul 23 06:01:08.632087803 (103h10m50.480694892s) [noise=true , address=45.91.116.4:4937 , piece_id=DZILXASXCQNDQGTAIYQIY4LCXHENKF6LLWHME5ILDWL3IZOSWMCQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF
Jul 23 06:01:08.633101069 (103h10m50.481708157s) [noise=true , address=45.91.116.11:4119 , piece_id=LOMOQU77BVHAPGZ23ZY6CG63FZFQX2IG5QRY2P46WKSGB5AF5O3Q, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.633132335 (103h10m50.481739428s) [seg_pos={0 0}, upload=82463]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 23 06:01:08.633221956 (103h10m50.481829049s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 23 06:01:08.633272252 (103h10m50.481879341s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 23 06:01:08.710605841 (103h10m50.559212933s) [upload=82463]: Done uploading segments...
Jul 23 06:01:08.710632085 (103h10m50.559239157s) [upload=82463]: Upload finished. err: <nil>
Duration: 606.81231ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................................X...................] duration:
542.068593ms
FAIL (2025-07-23 06:15:38 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
23 06:00:19.742341858 (103h10m1.590948949s) [noise=true , address=104.36.82.4:4804 , piece_id=SMCQKI5GIWKEHYLNSVUGHWQNBHP3S4C5RBB4E3K5CFMXE54Q4WAQ, seg_pos={0 0}, upload=82455]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g): protocol: expected piece hash; context canceled; EOF
Jul 23 06:00:19.742959346 (103h10m1.59156642s) [noise=true , address=77.90.31.16:4734 , piece_id=NFCPPB6P25YMO5UZRM4CQKMNRV5Q2FXMSR6M4VSTEERBMVNLYJJA, seg_pos={0 0}, upload=82455]: Done uploading piece. err: <nil>
Jul 23 06:00:19.743002232 (103h10m1.591609306s) [seg_pos={0 0}, upload=82455]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 23 06:00:19.743286680 (103h10m1.591893764s) [upload=82455]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 23 06:00:19.743293769 (103h10m1.591900841s) [upload=82455]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 23 06:00:19.815641707 (103h10m1.664248798s) [upload=82455]: Done uploading segments...
Jul 23 06:00:19.815666887 (103h10m1.664273954s) [upload=82455]: Upload finished. err: <nil>
Jul 23 06:01:08.401614022 (103h10m50.250221093s) [upload=82463]: Starting upload
Jul 23 06:01:08.401709823 (103h10m50.250316908s) [upload=82463]: Done starting upload
Jul 23 06:01:08.402138436 (103h10m50.250745505s) [upload=82463]: Uploading segments...
Jul 23 06:01:08.402189044 (103h10m50.250796137s) [upload=82463]: Got next segment. Inline: false
Jul 23 06:01:08.402231416 (103h10m50.250838487s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 23 06:01:08.402237731 (103h10m50.250844804s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 23 06:01:08.450301042 (103h10m50.298908146s) [seg_pos={0 0}, upload=82463]: Begin upload segment...
Jul 23 06:01:08.450364720 (103h10m50.298971798s) [seg_pos={0 0}, upload=82463]: Done begin upload segment.
Jul 23 06:01:08.450425015 (103h10m50.2990321s) [noise=true , address=45.91.116.5:4299 , piece_id=K3URA4ZO4R2CAGZLTO6YJTZQOZ6YIOB7XOJ3AQAVGDFIMVU63BIA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.450431954 (103h10m50.299039031s) [noise=true , address=45.91.116.7:4010 , piece_id=WVMOM2PPTVXY7NYI5UH25PFW6RODIUJWN73YB6FW6UOAMNY4C4SA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.450900413 (103h10m50.299507502s) [noise=true , address=77.90.31.17:4772 , piece_id=63V72ATK3KI3ADRC4N5HTRIXSEB3DRM7N73MVZNDPZJ7EVO3RZJA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.450963993 (103h10m50.299571083s) [noise=true , address=45.91.116.8:4048 , piece_id=AROG4HVUX7R6MHG3DT3N2FIQJBBEVVEVKQHSAGZCWMISWTFSQMAQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.451375091 (103h10m50.299982178s) [noise=true , address=45.91.116.13:4155 , piece_id=KMDEG3QMJNHRF7TETZAFVN3TOZWTQNW7IRTCVI3WLNE7NJAAAPCA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.451400410 (103h10m50.300007509s) [noise=true , address=77.90.31.12:4592 , piece_id=NAWAKPN4LUPCFZIXDIEPN2GKO3CSGX35LZNHI4QDCGYBYAMWBGOA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.451837258 (103h10m50.300444352s) [noise=true , address=77.90.31.17:4771 , piece_id=VCHLVD4PCT6SGJNCFA4LSEDIXSRMX27JH7VLUXDBM2PMIGZ54CNA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.452344092 (103h10m50.300951186s) [noise=true , address=45.91.116.4:4937 , piece_id=DZILXASXCQNDQGTAIYQIY4LCXHENKF6LLWHME5ILDWL3IZOSWMCQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.452773889 (103h10m50.301380982s) [noise=true , address=45.91.116.11:4119 , piece_id=LOMOQU77BVHAPGZ23ZY6CG63FZFQX2IG5QRY2P46WKSGB5AF5O3Q, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.453168858 (103h10m50.301775945s) [noise=true , address=45.91.116.5:4298 , piece_id=AIURH3V65NZD3QI3A5YGBJ222ANC4LTHEYFWTOFIC6SPVH3O4CKA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.453559736 (103h10m50.302166813s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQJDPJRZQEJHTED4UJBNSJ2CGAL6FPTLLIIRFTOVBUK65VSPLGA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.460854171 (103h10m50.309461265s) [noise=true , address=77.90.31.14:4661 , piece_id=MA3CRNAREYTT6ZDO7Q7QRSAOVQQ3EM3CHZTWYQYE6QOWNMYQBBQQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.462111720 (103h10m50.310718803s) [noise=true , address=45.91.116.12:4225 , piece_id=MFM7FUANQ2WU4GK46NPKKKFDUXSVVQ3MGK7GM3QBR7GMOD563PTQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.462819102 (103h10m50.311426189s) [noise=true , address=104.36.82.13:4186 , piece_id=O5BVVJBIWINWK2GXWWJDETED6T3Z462AAUQWCG6LGC7PVVFSHZBA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.463600977 (103h10m50.312208063s) [noise=true , address=45.91.116.7:4011 , piece_id=GKVDRTZ2TY2ESEKU2FKIJND4LXDMITNWQORE4Z47OQ454UZMM7BA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.464356397 (103h10m50.312963485s) [noise=true , address=77.90.31.13:4626 , piece_id=D63E7JGO5RRTEJGXVUMNEC3UYXHS45HBOB7S67VPCWLHJXNH7OSQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.587088382 (103h10m50.435695454s) [upload=82463]: Next returned nil segment
Jul 23 06:01:08.587107670 (103h10m50.435714757s) [upload=82463]: Waiting for error group managing segments...
Jul 23 06:01:08.609523184 (103h10m50.458130271s) [noise=true , address=77.90.31.14:4661 , piece_id=MA3CRNAREYTT6ZDO7Q7QRSAOVQQ3EM3CHZTWYQYE6QOWNMYQBBQQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.612539671 (103h10m50.46114674s) [noise=true , address=45.91.116.5:4299 , piece_id=K3URA4ZO4R2CAGZLTO6YJTZQOZ6YIOB7XOJ3AQAVGDFIMVU63BIA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.617758941 (103h10m50.466366025s) [noise=true , address=45.91.116.13:4155 , piece_id=KMDEG3QMJNHRF7TETZAFVN3TOZWTQNW7IRTCVI3WLNE7NJAAAPCA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.618513849 (103h10m50.467120922s) [noise=true , address=45.91.116.8:4048 , piece_id=AROG4HVUX7R6MHG3DT3N2FIQJBBEVVEVKQHSAGZCWMISWTFSQMAQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.624184177 (103h10m50.47279126s) [noise=true , address=45.91.116.7:4010 , piece_id=WVMOM2PPTVXY7NYI5UH25PFW6RODIUJWN73YB6FW6UOAMNY4C4SA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.625290831 (103h10m50.473897916s) [noise=true , address=45.91.116.7:4011 , piece_id=GKVDRTZ2TY2ESEKU2FKIJND4LXDMITNWQORE4Z47OQ454UZMM7BA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.628897649 (103h10m50.477504746s) [noise=true , address=45.91.116.5:4298 , piece_id=AIURH3V65NZD3QI3A5YGBJ222ANC4LTHEYFWTOFIC6SPVH3O4CKA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.629542622 (103h10m50.478149697s) [noise=true , address=45.91.116.12:4225 , piece_id=MFM7FUANQ2WU4GK46NPKKKFDUXSVVQ3MGK7GM3QBR7GMOD563PTQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.630003986 (103h10m50.478611073s) [noise=true , address=77.90.31.17:4772 , piece_id=63V72ATK3KI3ADRC4N5HTRIXSEB3DRM7N73MVZNDPZJ7EVO3RZJA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.630441959 (103h10m50.479049047s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQJDPJRZQEJHTED4UJBNSJ2CGAL6FPTLLIIRFTOVBUK65VSPLGA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.630941221 (103h10m50.479548305s) [noise=true , address=77.90.31.12:4592 , piece_id=NAWAKPN4LUPCFZIXDIEPN2GKO3CSGX35LZNHI4QDCGYBYAMWBGOA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.631378970 (103h10m50.47998605s) [noise=true , address=77.90.31.13:4626 , piece_id=D63E7JGO5RRTEJGXVUMNEC3UYXHS45HBOB7S67VPCWLHJXNH7OSQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.631840658 (103h10m50.480447733s) [noise=true , address=77.90.31.17:4771 , piece_id=VCHLVD4PCT6SGJNCFA4LSEDIXSRMX27JH7VLUXDBM2PMIGZ54CNA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.631850523 (103h10m50.480457598s) [seg_pos={0 0}, upload=82463]: Segment reached optimal threshold of 13 pieces.
Jul 23 06:01:08.631982956 (103h10m50.480590031s) [noise=true , address=104.36.82.13:4186 , piece_id=O5BVVJBIWINWK2GXWWJDETED6T3Z462AAUQWCG6LGC7PVVFSHZBA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF
Jul 23 06:01:08.632087803 (103h10m50.480694892s) [noise=true , address=45.91.116.4:4937 , piece_id=DZILXASXCQNDQGTAIYQIY4LCXHENKF6LLWHME5ILDWL3IZOSWMCQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF
Jul 23 06:01:08.633101069 (103h10m50.481708157s) [noise=true , address=45.91.116.11:4119 , piece_id=LOMOQU77BVHAPGZ23ZY6CG63FZFQX2IG5QRY2P46WKSGB5AF5O3Q, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.633132335 (103h10m50.481739428s) [seg_pos={0 0}, upload=82463]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 23 06:01:08.633221956 (103h10m50.481829049s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 23 06:01:08.633272252 (103h10m50.481879341s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 23 06:01:08.710605841 (103h10m50.559212933s) [upload=82463]: Done uploading segments...
Jul 23 06:01:08.710632085 (103h10m50.559239157s) [upload=82463]: Upload finished. err: <nil>
Duration: 542.101205ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.........................................X..................] duration:
538.944082ms
FAIL (2025-07-23 06:15:28 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
s) [upload=82448]: Got next segment. Inline: false
Jul 23 05:59:58.732247463 (103h9m40.580854536s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 23 05:59:58.732253833 (103h9m40.580860918s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 23 05:59:58.787057645 (103h9m40.635664731s) [seg_pos={0 0}, upload=82448]: Begin upload segment...
Jul 23 05:59:58.787102519 (103h9m40.635709584s) [seg_pos={0 0}, upload=82448]: Done begin upload segment.
Jul 23 05:59:58.787136403 (103h9m40.635743482s) [noise=true , address=77.90.31.12:4622 , piece_id=EZJAG75JRZTYILS7545KYAI37YOUTHHIOOSH26KIWMUQ4BCG556A, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787169354 (103h9m40.635776438s) [noise=true , address=79.127.205.235:4680 , piece_id=T3V7PMZXJOHWT77MQ52PUMFNAIAX4DLMTIIVL6XNVTNA565O5WCQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787544550 (103h9m40.636151681s) [noise=true , address=104.36.82.10:4105 , piece_id=GY72JPHTZOGEVFDG3FM5W5ZSYOPSY3BZXEME2Y37VVR2LR37SIUA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787598825 (103h9m40.636205915s) [noise=true , address=79.127.205.246:4280 , piece_id=NPBN75MFQYLCG4XKRJPLHJU5JCYBNQYIFQ7MKKB3JKXLT4TKF7AA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787946084 (103h9m40.636553167s) [noise=true , address=77.90.31.17:4799 , piece_id=RUTSDNZEXKJ5Y7WQBIVTMFUIJRYLL6DDNNJEXZAL5S7CIATWYHBQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788041731 (103h9m40.636648823s) [noise=true , address=77.90.31.11:4569 , piece_id=MTNLU22JHOYH6VSRMWLKQGSCDI5GIADPIXBVVSC2RIJLTHPMXTOA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788383218 (103h9m40.636990305s) [noise=true , address=104.36.82.42:4818 , piece_id=A6U7TDLTAZHE5IILHHCZKIHCQINHCJSWTBYNXJV7GLCWTIBNB2VA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788633554 (103h9m40.637240644s) [noise=true , address=79.127.219.51:4202 , piece_id=LJZ5A5HAHG4CGXT5J2WNSMXD7HCYSCBUEYGAGOES3NWIVV34BXEQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788735479 (103h9m40.637342568s) [noise=true , address=79.127.219.47:4040 , piece_id=BKWXBAVFXQ2RKG27WGPEAWG5UTHBVODW2BB4EGKHV6SN4EFI7KIQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789072444 (103h9m40.637679533s) [noise=true , address=77.90.31.15:4706 , piece_id=C3JOUPYB77PA2EHPSNPMDETERE724PBALBCSZ6MJCTZ6YRPUC2UA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789122202 (103h9m40.637729291s) [noise=true , address=104.36.82.9:4035 , piece_id=74BSWBLVWYQXPUI2PGV5NIJDSE6MSFKWK65TCVV6CNZKVQBQVNEA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789393961 (103h9m40.638001047s) [noise=true , address=45.91.116.10:4103 , piece_id=UGRDLKKFJRNCCDNDQTAY2LLOIS7CGI6KLNDZ7KJDOUNYWFNUAYQQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789591397 (103h9m40.638198479s) [noise=true , address=104.36.82.5:4870 , piece_id=AYTLB2OL7OAI5EO2ZE6LXIQLCGSGDXKDKT6V5HXIPAREK656MDPQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789716087 (103h9m40.638323176s) [noise=true , address=79.127.205.233:4645 , piece_id=6YAS5E7QYEZAJXB6HUY3R4PLHC4Z7EYTMSFWNFWD5AP7HKXOKTUQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789937996 (103h9m40.638545078s) [noise=true , address=104.36.82.53:4121 , piece_id=GJLXLW2A5RZUF4MBUP5AZAWFKTWGBDADIIWVIUBXPQPPGPKYZ4YA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.790288745 (103h9m40.638895835s) [noise=true , address=77.90.31.15:4708 , piece_id=67LX4NZJUKLYHRTSW3RKHHXZRKMGWJ2KV32FDA2KE3DY364CJ7OA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.790730467 (103h9m40.639337554s) [noise=true , address=45.91.116.14:4206 , piece_id=OVX63SL5OGJTT7HWDRIUV4BDVEKN2MD4ORUUK5M4JIEWX4J6VHFA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.791174576 (103h9m40.639781656s) [noise=true , address=45.91.116.13:4176 , piece_id=GOWXIBOORGZWFIBLMG46HQ4J632B3E6I3MVXNLTWPP2TY3IF2JUQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.898860128 (103h9m40.747467214s) [upload=82448]: Next returned nil segment
Jul 23 05:59:58.898881813 (103h9m40.747488906s) [upload=82448]: Waiting for error group managing segments...
Jul 23 05:59:58.905078785 (103h9m40.753685854s) [noise=true , address=45.91.116.14:4206 , piece_id=OVX63SL5OGJTT7HWDRIUV4BDVEKN2MD4ORUUK5M4JIEWX4J6VHFA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.906441982 (103h9m40.755049063s) [noise=true , address=77.90.31.15:4708 , piece_id=67LX4NZJUKLYHRTSW3RKHHXZRKMGWJ2KV32FDA2KE3DY364CJ7OA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.910115252 (103h9m40.758722337s) [noise=true , address=77.90.31.15:4706 , piece_id=C3JOUPYB77PA2EHPSNPMDETERE724PBALBCSZ6MJCTZ6YRPUC2UA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.910652695 (103h9m40.759259762s) [noise=true , address=104.36.82.9:4035 , piece_id=74BSWBLVWYQXPUI2PGV5NIJDSE6MSFKWK65TCVV6CNZKVQBQVNEA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.910672036 (103h9m40.759279117s) [noise=true , address=77.90.31.11:4569 , piece_id=MTNLU22JHOYH6VSRMWLKQGSCDI5GIADPIXBVVSC2RIJLTHPMXTOA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.911483080 (103h9m40.760090146s) [noise=true , address=45.91.116.13:4176 , piece_id=GOWXIBOORGZWFIBLMG46HQ4J632B3E6I3MVXNLTWPP2TY3IF2JUQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.911951970 (103h9m40.760559052s) [noise=true , address=45.91.116.10:4103 , piece_id=UGRDLKKFJRNCCDNDQTAY2LLOIS7CGI6KLNDZ7KJDOUNYWFNUAYQQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.912044937 (103h9m40.760652006s) [noise=true , address=77.90.31.12:4622 , piece_id=EZJAG75JRZTYILS7545KYAI37YOUTHHIOOSH26KIWMUQ4BCG556A, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.912649865 (103h9m40.761256931s) [noise=true , address=77.90.31.17:4799 , piece_id=RUTSDNZEXKJ5Y7WQBIVTMFUIJRYLL6DDNNJEXZAL5S7CIATWYHBQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.913487977 (103h9m40.76209505s) [noise=true , address=79.127.219.47:4040 , piece_id=BKWXBAVFXQ2RKG27WGPEAWG5UTHBVODW2BB4EGKHV6SN4EFI7KIQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.914486860 (103h9m40.763093927s) [noise=true , address=104.36.82.53:4121 , piece_id=GJLXLW2A5RZUF4MBUP5AZAWFKTWGBDADIIWVIUBXPQPPGPKYZ4YA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.918080770 (103h9m40.766687852s) [noise=true , address=104.36.82.5:4870 , piece_id=AYTLB2OL7OAI5EO2ZE6LXIQLCGSGDXKDKT6V5HXIPAREK656MDPQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.924012392 (103h9m40.772619461s) [noise=true , address=79.127.219.51:4202 , piece_id=LJZ5A5HAHG4CGXT5J2WNSMXD7HCYSCBUEYGAGOES3NWIVV34BXEQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.932892483 (103h9m40.78149955s) [noise=true , address=104.36.82.42:4818 , piece_id=A6U7TDLTAZHE5IILHHCZKIHCQINHCJSWTBYNXJV7GLCWTIBNB2VA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.932913099 (103h9m40.781520167s) [seg_pos={0 0}, upload=82448]: Segment reached optimal threshold of 14 pieces.
Jul 23 05:59:58.933104957 (103h9m40.781712037s) [noise=true , address=104.36.82.10:4105 , piece_id=GY72JPHTZOGEVFDG3FM5W5ZSYOPSY3BZXEME2Y37VVR2LR37SIUA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GhDrKMcDEg2RxamqKtiHphpgNdvtZYdHXrjzGsNJFjBFYeEen): protocol: expected piece hash; context canceled; EOF
Jul 23 05:59:58.933505265 (103h9m40.782112345s) [noise=true , address=79.127.205.233:4645 , piece_id=6YAS5E7QYEZAJXB6HUY3R4PLHC4Z7EYTMSFWNFWD5AP7HKXOKTUQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12NR52pweTqZNNMTyc1EHSbifDKB4awrnnb4megtZHwANSrR18T): protocol: read tcp 159.223.143.39:46566->79.127.205.233:4645: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 23 05:59:58.933568113 (103h9m40.782175194s) [noise=true , address=79.127.205.235:4680 , piece_id=T3V7PMZXJOHWT77MQ52PUMFNAIAX4DLMTIIVL6XNVTNA565O5WCQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GGvQQLoBcTniZST8Hz9h8eCLWbU4DrXn66DGy33iGLXaFgJLj): protocol: read tcp 159.223.143.39:45394->79.127.205.235:4680: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 23 05:59:58.933618915 (103h9m40.782225986s) [noise=true , address=79.127.205.246:4280 , piece_id=NPBN75MFQYLCG4XKRJPLHJU5JCYBNQYIFQ7MKKB3JKXLT4TKF7AA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rZDBvhrfNrYQB6h9MGhx1tNtQ7AH9ehUF4SfNMCJRpv4SKrDX): protocol: read tcp 159.223.143.39:45458->79.127.205.246:4280: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 23 05:59:58.933631191 (103h9m40.78223826s) [seg_pos={0 0}, upload=82448]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jul 23 05:59:58.933756265 (103h9m40.782363345s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 23 05:59:58.933762566 (103h9m40.782369645s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 23 05:59:59.025408726 (103h9m40.874015808s) [upload=82448]: Done uploading segments...
Jul 23 05:59:59.025427547 (103h9m40.874034627s) [upload=82448]: Upload finished. err: <nil>
Duration: 538.974396ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................................X..................] duration:
509.26133ms
FAIL (2025-07-23 06:14:50 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
23 06:00:19.742341858 (103h10m1.590948949s) [noise=true , address=104.36.82.4:4804 , piece_id=SMCQKI5GIWKEHYLNSVUGHWQNBHP3S4C5RBB4E3K5CFMXE54Q4WAQ, seg_pos={0 0}, upload=82455]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g): protocol: expected piece hash; context canceled; EOF
Jul 23 06:00:19.742959346 (103h10m1.59156642s) [noise=true , address=77.90.31.16:4734 , piece_id=NFCPPB6P25YMO5UZRM4CQKMNRV5Q2FXMSR6M4VSTEERBMVNLYJJA, seg_pos={0 0}, upload=82455]: Done uploading piece. err: <nil>
Jul 23 06:00:19.743002232 (103h10m1.591609306s) [seg_pos={0 0}, upload=82455]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 23 06:00:19.743286680 (103h10m1.591893764s) [upload=82455]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 23 06:00:19.743293769 (103h10m1.591900841s) [upload=82455]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 23 06:00:19.815641707 (103h10m1.664248798s) [upload=82455]: Done uploading segments...
Jul 23 06:00:19.815666887 (103h10m1.664273954s) [upload=82455]: Upload finished. err: <nil>
Jul 23 06:01:08.401614022 (103h10m50.250221093s) [upload=82463]: Starting upload
Jul 23 06:01:08.401709823 (103h10m50.250316908s) [upload=82463]: Done starting upload
Jul 23 06:01:08.402138436 (103h10m50.250745505s) [upload=82463]: Uploading segments...
Jul 23 06:01:08.402189044 (103h10m50.250796137s) [upload=82463]: Got next segment. Inline: false
Jul 23 06:01:08.402231416 (103h10m50.250838487s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 23 06:01:08.402237731 (103h10m50.250844804s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 23 06:01:08.450301042 (103h10m50.298908146s) [seg_pos={0 0}, upload=82463]: Begin upload segment...
Jul 23 06:01:08.450364720 (103h10m50.298971798s) [seg_pos={0 0}, upload=82463]: Done begin upload segment.
Jul 23 06:01:08.450425015 (103h10m50.2990321s) [noise=true , address=45.91.116.5:4299 , piece_id=K3URA4ZO4R2CAGZLTO6YJTZQOZ6YIOB7XOJ3AQAVGDFIMVU63BIA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.450431954 (103h10m50.299039031s) [noise=true , address=45.91.116.7:4010 , piece_id=WVMOM2PPTVXY7NYI5UH25PFW6RODIUJWN73YB6FW6UOAMNY4C4SA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.450900413 (103h10m50.299507502s) [noise=true , address=77.90.31.17:4772 , piece_id=63V72ATK3KI3ADRC4N5HTRIXSEB3DRM7N73MVZNDPZJ7EVO3RZJA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.450963993 (103h10m50.299571083s) [noise=true , address=45.91.116.8:4048 , piece_id=AROG4HVUX7R6MHG3DT3N2FIQJBBEVVEVKQHSAGZCWMISWTFSQMAQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.451375091 (103h10m50.299982178s) [noise=true , address=45.91.116.13:4155 , piece_id=KMDEG3QMJNHRF7TETZAFVN3TOZWTQNW7IRTCVI3WLNE7NJAAAPCA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.451400410 (103h10m50.300007509s) [noise=true , address=77.90.31.12:4592 , piece_id=NAWAKPN4LUPCFZIXDIEPN2GKO3CSGX35LZNHI4QDCGYBYAMWBGOA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.451837258 (103h10m50.300444352s) [noise=true , address=77.90.31.17:4771 , piece_id=VCHLVD4PCT6SGJNCFA4LSEDIXSRMX27JH7VLUXDBM2PMIGZ54CNA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.452344092 (103h10m50.300951186s) [noise=true , address=45.91.116.4:4937 , piece_id=DZILXASXCQNDQGTAIYQIY4LCXHENKF6LLWHME5ILDWL3IZOSWMCQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.452773889 (103h10m50.301380982s) [noise=true , address=45.91.116.11:4119 , piece_id=LOMOQU77BVHAPGZ23ZY6CG63FZFQX2IG5QRY2P46WKSGB5AF5O3Q, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.453168858 (103h10m50.301775945s) [noise=true , address=45.91.116.5:4298 , piece_id=AIURH3V65NZD3QI3A5YGBJ222ANC4LTHEYFWTOFIC6SPVH3O4CKA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.453559736 (103h10m50.302166813s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQJDPJRZQEJHTED4UJBNSJ2CGAL6FPTLLIIRFTOVBUK65VSPLGA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.460854171 (103h10m50.309461265s) [noise=true , address=77.90.31.14:4661 , piece_id=MA3CRNAREYTT6ZDO7Q7QRSAOVQQ3EM3CHZTWYQYE6QOWNMYQBBQQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.462111720 (103h10m50.310718803s) [noise=true , address=45.91.116.12:4225 , piece_id=MFM7FUANQ2WU4GK46NPKKKFDUXSVVQ3MGK7GM3QBR7GMOD563PTQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.462819102 (103h10m50.311426189s) [noise=true , address=104.36.82.13:4186 , piece_id=O5BVVJBIWINWK2GXWWJDETED6T3Z462AAUQWCG6LGC7PVVFSHZBA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.463600977 (103h10m50.312208063s) [noise=true , address=45.91.116.7:4011 , piece_id=GKVDRTZ2TY2ESEKU2FKIJND4LXDMITNWQORE4Z47OQ454UZMM7BA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.464356397 (103h10m50.312963485s) [noise=true , address=77.90.31.13:4626 , piece_id=D63E7JGO5RRTEJGXVUMNEC3UYXHS45HBOB7S67VPCWLHJXNH7OSQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.587088382 (103h10m50.435695454s) [upload=82463]: Next returned nil segment
Jul 23 06:01:08.587107670 (103h10m50.435714757s) [upload=82463]: Waiting for error group managing segments...
Jul 23 06:01:08.609523184 (103h10m50.458130271s) [noise=true , address=77.90.31.14:4661 , piece_id=MA3CRNAREYTT6ZDO7Q7QRSAOVQQ3EM3CHZTWYQYE6QOWNMYQBBQQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.612539671 (103h10m50.46114674s) [noise=true , address=45.91.116.5:4299 , piece_id=K3URA4ZO4R2CAGZLTO6YJTZQOZ6YIOB7XOJ3AQAVGDFIMVU63BIA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.617758941 (103h10m50.466366025s) [noise=true , address=45.91.116.13:4155 , piece_id=KMDEG3QMJNHRF7TETZAFVN3TOZWTQNW7IRTCVI3WLNE7NJAAAPCA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.618513849 (103h10m50.467120922s) [noise=true , address=45.91.116.8:4048 , piece_id=AROG4HVUX7R6MHG3DT3N2FIQJBBEVVEVKQHSAGZCWMISWTFSQMAQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.624184177 (103h10m50.47279126s) [noise=true , address=45.91.116.7:4010 , piece_id=WVMOM2PPTVXY7NYI5UH25PFW6RODIUJWN73YB6FW6UOAMNY4C4SA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.625290831 (103h10m50.473897916s) [noise=true , address=45.91.116.7:4011 , piece_id=GKVDRTZ2TY2ESEKU2FKIJND4LXDMITNWQORE4Z47OQ454UZMM7BA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.628897649 (103h10m50.477504746s) [noise=true , address=45.91.116.5:4298 , piece_id=AIURH3V65NZD3QI3A5YGBJ222ANC4LTHEYFWTOFIC6SPVH3O4CKA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.629542622 (103h10m50.478149697s) [noise=true , address=45.91.116.12:4225 , piece_id=MFM7FUANQ2WU4GK46NPKKKFDUXSVVQ3MGK7GM3QBR7GMOD563PTQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.630003986 (103h10m50.478611073s) [noise=true , address=77.90.31.17:4772 , piece_id=63V72ATK3KI3ADRC4N5HTRIXSEB3DRM7N73MVZNDPZJ7EVO3RZJA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.630441959 (103h10m50.479049047s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQJDPJRZQEJHTED4UJBNSJ2CGAL6FPTLLIIRFTOVBUK65VSPLGA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.630941221 (103h10m50.479548305s) [noise=true , address=77.90.31.12:4592 , piece_id=NAWAKPN4LUPCFZIXDIEPN2GKO3CSGX35LZNHI4QDCGYBYAMWBGOA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.631378970 (103h10m50.47998605s) [noise=true , address=77.90.31.13:4626 , piece_id=D63E7JGO5RRTEJGXVUMNEC3UYXHS45HBOB7S67VPCWLHJXNH7OSQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.631840658 (103h10m50.480447733s) [noise=true , address=77.90.31.17:4771 , piece_id=VCHLVD4PCT6SGJNCFA4LSEDIXSRMX27JH7VLUXDBM2PMIGZ54CNA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.631850523 (103h10m50.480457598s) [seg_pos={0 0}, upload=82463]: Segment reached optimal threshold of 13 pieces.
Jul 23 06:01:08.631982956 (103h10m50.480590031s) [noise=true , address=104.36.82.13:4186 , piece_id=O5BVVJBIWINWK2GXWWJDETED6T3Z462AAUQWCG6LGC7PVVFSHZBA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF
Jul 23 06:01:08.632087803 (103h10m50.480694892s) [noise=true , address=45.91.116.4:4937 , piece_id=DZILXASXCQNDQGTAIYQIY4LCXHENKF6LLWHME5ILDWL3IZOSWMCQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF
Jul 23 06:01:08.633101069 (103h10m50.481708157s) [noise=true , address=45.91.116.11:4119 , piece_id=LOMOQU77BVHAPGZ23ZY6CG63FZFQX2IG5QRY2P46WKSGB5AF5O3Q, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.633132335 (103h10m50.481739428s) [seg_pos={0 0}, upload=82463]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 23 06:01:08.633221956 (103h10m50.481829049s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 23 06:01:08.633272252 (103h10m50.481879341s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 23 06:01:08.710605841 (103h10m50.559212933s) [upload=82463]: Done uploading segments...
Jul 23 06:01:08.710632085 (103h10m50.559239157s) [upload=82463]: Upload finished. err: <nil>
Duration: 509.290573ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [..........................................X.................] duration:
492.517668ms
FAIL (2025-07-23 06:14:50 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
s) [upload=82448]: Got next segment. Inline: false
Jul 23 05:59:58.732247463 (103h9m40.580854536s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 23 05:59:58.732253833 (103h9m40.580860918s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 23 05:59:58.787057645 (103h9m40.635664731s) [seg_pos={0 0}, upload=82448]: Begin upload segment...
Jul 23 05:59:58.787102519 (103h9m40.635709584s) [seg_pos={0 0}, upload=82448]: Done begin upload segment.
Jul 23 05:59:58.787136403 (103h9m40.635743482s) [noise=true , address=77.90.31.12:4622 , piece_id=EZJAG75JRZTYILS7545KYAI37YOUTHHIOOSH26KIWMUQ4BCG556A, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787169354 (103h9m40.635776438s) [noise=true , address=79.127.205.235:4680 , piece_id=T3V7PMZXJOHWT77MQ52PUMFNAIAX4DLMTIIVL6XNVTNA565O5WCQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787544550 (103h9m40.636151681s) [noise=true , address=104.36.82.10:4105 , piece_id=GY72JPHTZOGEVFDG3FM5W5ZSYOPSY3BZXEME2Y37VVR2LR37SIUA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787598825 (103h9m40.636205915s) [noise=true , address=79.127.205.246:4280 , piece_id=NPBN75MFQYLCG4XKRJPLHJU5JCYBNQYIFQ7MKKB3JKXLT4TKF7AA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787946084 (103h9m40.636553167s) [noise=true , address=77.90.31.17:4799 , piece_id=RUTSDNZEXKJ5Y7WQBIVTMFUIJRYLL6DDNNJEXZAL5S7CIATWYHBQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788041731 (103h9m40.636648823s) [noise=true , address=77.90.31.11:4569 , piece_id=MTNLU22JHOYH6VSRMWLKQGSCDI5GIADPIXBVVSC2RIJLTHPMXTOA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788383218 (103h9m40.636990305s) [noise=true , address=104.36.82.42:4818 , piece_id=A6U7TDLTAZHE5IILHHCZKIHCQINHCJSWTBYNXJV7GLCWTIBNB2VA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788633554 (103h9m40.637240644s) [noise=true , address=79.127.219.51:4202 , piece_id=LJZ5A5HAHG4CGXT5J2WNSMXD7HCYSCBUEYGAGOES3NWIVV34BXEQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788735479 (103h9m40.637342568s) [noise=true , address=79.127.219.47:4040 , piece_id=BKWXBAVFXQ2RKG27WGPEAWG5UTHBVODW2BB4EGKHV6SN4EFI7KIQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789072444 (103h9m40.637679533s) [noise=true , address=77.90.31.15:4706 , piece_id=C3JOUPYB77PA2EHPSNPMDETERE724PBALBCSZ6MJCTZ6YRPUC2UA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789122202 (103h9m40.637729291s) [noise=true , address=104.36.82.9:4035 , piece_id=74BSWBLVWYQXPUI2PGV5NIJDSE6MSFKWK65TCVV6CNZKVQBQVNEA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789393961 (103h9m40.638001047s) [noise=true , address=45.91.116.10:4103 , piece_id=UGRDLKKFJRNCCDNDQTAY2LLOIS7CGI6KLNDZ7KJDOUNYWFNUAYQQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789591397 (103h9m40.638198479s) [noise=true , address=104.36.82.5:4870 , piece_id=AYTLB2OL7OAI5EO2ZE6LXIQLCGSGDXKDKT6V5HXIPAREK656MDPQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789716087 (103h9m40.638323176s) [noise=true , address=79.127.205.233:4645 , piece_id=6YAS5E7QYEZAJXB6HUY3R4PLHC4Z7EYTMSFWNFWD5AP7HKXOKTUQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789937996 (103h9m40.638545078s) [noise=true , address=104.36.82.53:4121 , piece_id=GJLXLW2A5RZUF4MBUP5AZAWFKTWGBDADIIWVIUBXPQPPGPKYZ4YA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.790288745 (103h9m40.638895835s) [noise=true , address=77.90.31.15:4708 , piece_id=67LX4NZJUKLYHRTSW3RKHHXZRKMGWJ2KV32FDA2KE3DY364CJ7OA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.790730467 (103h9m40.639337554s) [noise=true , address=45.91.116.14:4206 , piece_id=OVX63SL5OGJTT7HWDRIUV4BDVEKN2MD4ORUUK5M4JIEWX4J6VHFA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.791174576 (103h9m40.639781656s) [noise=true , address=45.91.116.13:4176 , piece_id=GOWXIBOORGZWFIBLMG46HQ4J632B3E6I3MVXNLTWPP2TY3IF2JUQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.898860128 (103h9m40.747467214s) [upload=82448]: Next returned nil segment
Jul 23 05:59:58.898881813 (103h9m40.747488906s) [upload=82448]: Waiting for error group managing segments...
Jul 23 05:59:58.905078785 (103h9m40.753685854s) [noise=true , address=45.91.116.14:4206 , piece_id=OVX63SL5OGJTT7HWDRIUV4BDVEKN2MD4ORUUK5M4JIEWX4J6VHFA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.906441982 (103h9m40.755049063s) [noise=true , address=77.90.31.15:4708 , piece_id=67LX4NZJUKLYHRTSW3RKHHXZRKMGWJ2KV32FDA2KE3DY364CJ7OA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.910115252 (103h9m40.758722337s) [noise=true , address=77.90.31.15:4706 , piece_id=C3JOUPYB77PA2EHPSNPMDETERE724PBALBCSZ6MJCTZ6YRPUC2UA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.910652695 (103h9m40.759259762s) [noise=true , address=104.36.82.9:4035 , piece_id=74BSWBLVWYQXPUI2PGV5NIJDSE6MSFKWK65TCVV6CNZKVQBQVNEA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.910672036 (103h9m40.759279117s) [noise=true , address=77.90.31.11:4569 , piece_id=MTNLU22JHOYH6VSRMWLKQGSCDI5GIADPIXBVVSC2RIJLTHPMXTOA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.911483080 (103h9m40.760090146s) [noise=true , address=45.91.116.13:4176 , piece_id=GOWXIBOORGZWFIBLMG46HQ4J632B3E6I3MVXNLTWPP2TY3IF2JUQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.911951970 (103h9m40.760559052s) [noise=true , address=45.91.116.10:4103 , piece_id=UGRDLKKFJRNCCDNDQTAY2LLOIS7CGI6KLNDZ7KJDOUNYWFNUAYQQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.912044937 (103h9m40.760652006s) [noise=true , address=77.90.31.12:4622 , piece_id=EZJAG75JRZTYILS7545KYAI37YOUTHHIOOSH26KIWMUQ4BCG556A, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.912649865 (103h9m40.761256931s) [noise=true , address=77.90.31.17:4799 , piece_id=RUTSDNZEXKJ5Y7WQBIVTMFUIJRYLL6DDNNJEXZAL5S7CIATWYHBQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.913487977 (103h9m40.76209505s) [noise=true , address=79.127.219.47:4040 , piece_id=BKWXBAVFXQ2RKG27WGPEAWG5UTHBVODW2BB4EGKHV6SN4EFI7KIQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.914486860 (103h9m40.763093927s) [noise=true , address=104.36.82.53:4121 , piece_id=GJLXLW2A5RZUF4MBUP5AZAWFKTWGBDADIIWVIUBXPQPPGPKYZ4YA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.918080770 (103h9m40.766687852s) [noise=true , address=104.36.82.5:4870 , piece_id=AYTLB2OL7OAI5EO2ZE6LXIQLCGSGDXKDKT6V5HXIPAREK656MDPQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.924012392 (103h9m40.772619461s) [noise=true , address=79.127.219.51:4202 , piece_id=LJZ5A5HAHG4CGXT5J2WNSMXD7HCYSCBUEYGAGOES3NWIVV34BXEQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.932892483 (103h9m40.78149955s) [noise=true , address=104.36.82.42:4818 , piece_id=A6U7TDLTAZHE5IILHHCZKIHCQINHCJSWTBYNXJV7GLCWTIBNB2VA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.932913099 (103h9m40.781520167s) [seg_pos={0 0}, upload=82448]: Segment reached optimal threshold of 14 pieces.
Jul 23 05:59:58.933104957 (103h9m40.781712037s) [noise=true , address=104.36.82.10:4105 , piece_id=GY72JPHTZOGEVFDG3FM5W5ZSYOPSY3BZXEME2Y37VVR2LR37SIUA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GhDrKMcDEg2RxamqKtiHphpgNdvtZYdHXrjzGsNJFjBFYeEen): protocol: expected piece hash; context canceled; EOF
Jul 23 05:59:58.933505265 (103h9m40.782112345s) [noise=true , address=79.127.205.233:4645 , piece_id=6YAS5E7QYEZAJXB6HUY3R4PLHC4Z7EYTMSFWNFWD5AP7HKXOKTUQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12NR52pweTqZNNMTyc1EHSbifDKB4awrnnb4megtZHwANSrR18T): protocol: read tcp 159.223.143.39:46566->79.127.205.233:4645: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 23 05:59:58.933568113 (103h9m40.782175194s) [noise=true , address=79.127.205.235:4680 , piece_id=T3V7PMZXJOHWT77MQ52PUMFNAIAX4DLMTIIVL6XNVTNA565O5WCQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GGvQQLoBcTniZST8Hz9h8eCLWbU4DrXn66DGy33iGLXaFgJLj): protocol: read tcp 159.223.143.39:45394->79.127.205.235:4680: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 23 05:59:58.933618915 (103h9m40.782225986s) [noise=true , address=79.127.205.246:4280 , piece_id=NPBN75MFQYLCG4XKRJPLHJU5JCYBNQYIFQ7MKKB3JKXLT4TKF7AA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rZDBvhrfNrYQB6h9MGhx1tNtQ7AH9ehUF4SfNMCJRpv4SKrDX): protocol: read tcp 159.223.143.39:45458->79.127.205.246:4280: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 23 05:59:58.933631191 (103h9m40.78223826s) [seg_pos={0 0}, upload=82448]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jul 23 05:59:58.933756265 (103h9m40.782363345s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 23 05:59:58.933762566 (103h9m40.782369645s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 23 05:59:59.025408726 (103h9m40.874015808s) [upload=82448]: Done uploading segments...
Jul 23 05:59:59.025427547 (103h9m40.874034627s) [upload=82448]: Upload finished. err: <nil>
Duration: 492.552006ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................................X.................] duration:
465.879813ms
FAIL (2025-07-23 06:14:18 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
23 06:00:19.742341858 (103h10m1.590948949s) [noise=true , address=104.36.82.4:4804 , piece_id=SMCQKI5GIWKEHYLNSVUGHWQNBHP3S4C5RBB4E3K5CFMXE54Q4WAQ, seg_pos={0 0}, upload=82455]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g): protocol: expected piece hash; context canceled; EOF
Jul 23 06:00:19.742959346 (103h10m1.59156642s) [noise=true , address=77.90.31.16:4734 , piece_id=NFCPPB6P25YMO5UZRM4CQKMNRV5Q2FXMSR6M4VSTEERBMVNLYJJA, seg_pos={0 0}, upload=82455]: Done uploading piece. err: <nil>
Jul 23 06:00:19.743002232 (103h10m1.591609306s) [seg_pos={0 0}, upload=82455]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 23 06:00:19.743286680 (103h10m1.591893764s) [upload=82455]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 23 06:00:19.743293769 (103h10m1.591900841s) [upload=82455]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 23 06:00:19.815641707 (103h10m1.664248798s) [upload=82455]: Done uploading segments...
Jul 23 06:00:19.815666887 (103h10m1.664273954s) [upload=82455]: Upload finished. err: <nil>
Jul 23 06:01:08.401614022 (103h10m50.250221093s) [upload=82463]: Starting upload
Jul 23 06:01:08.401709823 (103h10m50.250316908s) [upload=82463]: Done starting upload
Jul 23 06:01:08.402138436 (103h10m50.250745505s) [upload=82463]: Uploading segments...
Jul 23 06:01:08.402189044 (103h10m50.250796137s) [upload=82463]: Got next segment. Inline: false
Jul 23 06:01:08.402231416 (103h10m50.250838487s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 23 06:01:08.402237731 (103h10m50.250844804s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 23 06:01:08.450301042 (103h10m50.298908146s) [seg_pos={0 0}, upload=82463]: Begin upload segment...
Jul 23 06:01:08.450364720 (103h10m50.298971798s) [seg_pos={0 0}, upload=82463]: Done begin upload segment.
Jul 23 06:01:08.450425015 (103h10m50.2990321s) [noise=true , address=45.91.116.5:4299 , piece_id=K3URA4ZO4R2CAGZLTO6YJTZQOZ6YIOB7XOJ3AQAVGDFIMVU63BIA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.450431954 (103h10m50.299039031s) [noise=true , address=45.91.116.7:4010 , piece_id=WVMOM2PPTVXY7NYI5UH25PFW6RODIUJWN73YB6FW6UOAMNY4C4SA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.450900413 (103h10m50.299507502s) [noise=true , address=77.90.31.17:4772 , piece_id=63V72ATK3KI3ADRC4N5HTRIXSEB3DRM7N73MVZNDPZJ7EVO3RZJA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.450963993 (103h10m50.299571083s) [noise=true , address=45.91.116.8:4048 , piece_id=AROG4HVUX7R6MHG3DT3N2FIQJBBEVVEVKQHSAGZCWMISWTFSQMAQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.451375091 (103h10m50.299982178s) [noise=true , address=45.91.116.13:4155 , piece_id=KMDEG3QMJNHRF7TETZAFVN3TOZWTQNW7IRTCVI3WLNE7NJAAAPCA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.451400410 (103h10m50.300007509s) [noise=true , address=77.90.31.12:4592 , piece_id=NAWAKPN4LUPCFZIXDIEPN2GKO3CSGX35LZNHI4QDCGYBYAMWBGOA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.451837258 (103h10m50.300444352s) [noise=true , address=77.90.31.17:4771 , piece_id=VCHLVD4PCT6SGJNCFA4LSEDIXSRMX27JH7VLUXDBM2PMIGZ54CNA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.452344092 (103h10m50.300951186s) [noise=true , address=45.91.116.4:4937 , piece_id=DZILXASXCQNDQGTAIYQIY4LCXHENKF6LLWHME5ILDWL3IZOSWMCQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.452773889 (103h10m50.301380982s) [noise=true , address=45.91.116.11:4119 , piece_id=LOMOQU77BVHAPGZ23ZY6CG63FZFQX2IG5QRY2P46WKSGB5AF5O3Q, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.453168858 (103h10m50.301775945s) [noise=true , address=45.91.116.5:4298 , piece_id=AIURH3V65NZD3QI3A5YGBJ222ANC4LTHEYFWTOFIC6SPVH3O4CKA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.453559736 (103h10m50.302166813s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQJDPJRZQEJHTED4UJBNSJ2CGAL6FPTLLIIRFTOVBUK65VSPLGA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.460854171 (103h10m50.309461265s) [noise=true , address=77.90.31.14:4661 , piece_id=MA3CRNAREYTT6ZDO7Q7QRSAOVQQ3EM3CHZTWYQYE6QOWNMYQBBQQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.462111720 (103h10m50.310718803s) [noise=true , address=45.91.116.12:4225 , piece_id=MFM7FUANQ2WU4GK46NPKKKFDUXSVVQ3MGK7GM3QBR7GMOD563PTQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.462819102 (103h10m50.311426189s) [noise=true , address=104.36.82.13:4186 , piece_id=O5BVVJBIWINWK2GXWWJDETED6T3Z462AAUQWCG6LGC7PVVFSHZBA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.463600977 (103h10m50.312208063s) [noise=true , address=45.91.116.7:4011 , piece_id=GKVDRTZ2TY2ESEKU2FKIJND4LXDMITNWQORE4Z47OQ454UZMM7BA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.464356397 (103h10m50.312963485s) [noise=true , address=77.90.31.13:4626 , piece_id=D63E7JGO5RRTEJGXVUMNEC3UYXHS45HBOB7S67VPCWLHJXNH7OSQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.587088382 (103h10m50.435695454s) [upload=82463]: Next returned nil segment
Jul 23 06:01:08.587107670 (103h10m50.435714757s) [upload=82463]: Waiting for error group managing segments...
Jul 23 06:01:08.609523184 (103h10m50.458130271s) [noise=true , address=77.90.31.14:4661 , piece_id=MA3CRNAREYTT6ZDO7Q7QRSAOVQQ3EM3CHZTWYQYE6QOWNMYQBBQQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.612539671 (103h10m50.46114674s) [noise=true , address=45.91.116.5:4299 , piece_id=K3URA4ZO4R2CAGZLTO6YJTZQOZ6YIOB7XOJ3AQAVGDFIMVU63BIA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.617758941 (103h10m50.466366025s) [noise=true , address=45.91.116.13:4155 , piece_id=KMDEG3QMJNHRF7TETZAFVN3TOZWTQNW7IRTCVI3WLNE7NJAAAPCA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.618513849 (103h10m50.467120922s) [noise=true , address=45.91.116.8:4048 , piece_id=AROG4HVUX7R6MHG3DT3N2FIQJBBEVVEVKQHSAGZCWMISWTFSQMAQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.624184177 (103h10m50.47279126s) [noise=true , address=45.91.116.7:4010 , piece_id=WVMOM2PPTVXY7NYI5UH25PFW6RODIUJWN73YB6FW6UOAMNY4C4SA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.625290831 (103h10m50.473897916s) [noise=true , address=45.91.116.7:4011 , piece_id=GKVDRTZ2TY2ESEKU2FKIJND4LXDMITNWQORE4Z47OQ454UZMM7BA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.628897649 (103h10m50.477504746s) [noise=true , address=45.91.116.5:4298 , piece_id=AIURH3V65NZD3QI3A5YGBJ222ANC4LTHEYFWTOFIC6SPVH3O4CKA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.629542622 (103h10m50.478149697s) [noise=true , address=45.91.116.12:4225 , piece_id=MFM7FUANQ2WU4GK46NPKKKFDUXSVVQ3MGK7GM3QBR7GMOD563PTQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.630003986 (103h10m50.478611073s) [noise=true , address=77.90.31.17:4772 , piece_id=63V72ATK3KI3ADRC4N5HTRIXSEB3DRM7N73MVZNDPZJ7EVO3RZJA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.630441959 (103h10m50.479049047s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQJDPJRZQEJHTED4UJBNSJ2CGAL6FPTLLIIRFTOVBUK65VSPLGA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.630941221 (103h10m50.479548305s) [noise=true , address=77.90.31.12:4592 , piece_id=NAWAKPN4LUPCFZIXDIEPN2GKO3CSGX35LZNHI4QDCGYBYAMWBGOA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.631378970 (103h10m50.47998605s) [noise=true , address=77.90.31.13:4626 , piece_id=D63E7JGO5RRTEJGXVUMNEC3UYXHS45HBOB7S67VPCWLHJXNH7OSQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.631840658 (103h10m50.480447733s) [noise=true , address=77.90.31.17:4771 , piece_id=VCHLVD4PCT6SGJNCFA4LSEDIXSRMX27JH7VLUXDBM2PMIGZ54CNA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.631850523 (103h10m50.480457598s) [seg_pos={0 0}, upload=82463]: Segment reached optimal threshold of 13 pieces.
Jul 23 06:01:08.631982956 (103h10m50.480590031s) [noise=true , address=104.36.82.13:4186 , piece_id=O5BVVJBIWINWK2GXWWJDETED6T3Z462AAUQWCG6LGC7PVVFSHZBA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF
Jul 23 06:01:08.632087803 (103h10m50.480694892s) [noise=true , address=45.91.116.4:4937 , piece_id=DZILXASXCQNDQGTAIYQIY4LCXHENKF6LLWHME5ILDWL3IZOSWMCQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF
Jul 23 06:01:08.633101069 (103h10m50.481708157s) [noise=true , address=45.91.116.11:4119 , piece_id=LOMOQU77BVHAPGZ23ZY6CG63FZFQX2IG5QRY2P46WKSGB5AF5O3Q, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.633132335 (103h10m50.481739428s) [seg_pos={0 0}, upload=82463]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 23 06:01:08.633221956 (103h10m50.481829049s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 23 06:01:08.633272252 (103h10m50.481879341s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 23 06:01:08.710605841 (103h10m50.559212933s) [upload=82463]: Done uploading segments...
Jul 23 06:01:08.710632085 (103h10m50.559239157s) [upload=82463]: Upload finished. err: <nil>
Duration: 465.91102ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................] duration:
688.84286ms
FAIL (2025-07-23 06:14:15 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
s) [upload=82448]: Got next segment. Inline: false
Jul 23 05:59:58.732247463 (103h9m40.580854536s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 23 05:59:58.732253833 (103h9m40.580860918s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 23 05:59:58.787057645 (103h9m40.635664731s) [seg_pos={0 0}, upload=82448]: Begin upload segment...
Jul 23 05:59:58.787102519 (103h9m40.635709584s) [seg_pos={0 0}, upload=82448]: Done begin upload segment.
Jul 23 05:59:58.787136403 (103h9m40.635743482s) [noise=true , address=77.90.31.12:4622 , piece_id=EZJAG75JRZTYILS7545KYAI37YOUTHHIOOSH26KIWMUQ4BCG556A, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787169354 (103h9m40.635776438s) [noise=true , address=79.127.205.235:4680 , piece_id=T3V7PMZXJOHWT77MQ52PUMFNAIAX4DLMTIIVL6XNVTNA565O5WCQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787544550 (103h9m40.636151681s) [noise=true , address=104.36.82.10:4105 , piece_id=GY72JPHTZOGEVFDG3FM5W5ZSYOPSY3BZXEME2Y37VVR2LR37SIUA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787598825 (103h9m40.636205915s) [noise=true , address=79.127.205.246:4280 , piece_id=NPBN75MFQYLCG4XKRJPLHJU5JCYBNQYIFQ7MKKB3JKXLT4TKF7AA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787946084 (103h9m40.636553167s) [noise=true , address=77.90.31.17:4799 , piece_id=RUTSDNZEXKJ5Y7WQBIVTMFUIJRYLL6DDNNJEXZAL5S7CIATWYHBQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788041731 (103h9m40.636648823s) [noise=true , address=77.90.31.11:4569 , piece_id=MTNLU22JHOYH6VSRMWLKQGSCDI5GIADPIXBVVSC2RIJLTHPMXTOA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788383218 (103h9m40.636990305s) [noise=true , address=104.36.82.42:4818 , piece_id=A6U7TDLTAZHE5IILHHCZKIHCQINHCJSWTBYNXJV7GLCWTIBNB2VA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788633554 (103h9m40.637240644s) [noise=true , address=79.127.219.51:4202 , piece_id=LJZ5A5HAHG4CGXT5J2WNSMXD7HCYSCBUEYGAGOES3NWIVV34BXEQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788735479 (103h9m40.637342568s) [noise=true , address=79.127.219.47:4040 , piece_id=BKWXBAVFXQ2RKG27WGPEAWG5UTHBVODW2BB4EGKHV6SN4EFI7KIQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789072444 (103h9m40.637679533s) [noise=true , address=77.90.31.15:4706 , piece_id=C3JOUPYB77PA2EHPSNPMDETERE724PBALBCSZ6MJCTZ6YRPUC2UA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789122202 (103h9m40.637729291s) [noise=true , address=104.36.82.9:4035 , piece_id=74BSWBLVWYQXPUI2PGV5NIJDSE6MSFKWK65TCVV6CNZKVQBQVNEA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789393961 (103h9m40.638001047s) [noise=true , address=45.91.116.10:4103 , piece_id=UGRDLKKFJRNCCDNDQTAY2LLOIS7CGI6KLNDZ7KJDOUNYWFNUAYQQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789591397 (103h9m40.638198479s) [noise=true , address=104.36.82.5:4870 , piece_id=AYTLB2OL7OAI5EO2ZE6LXIQLCGSGDXKDKT6V5HXIPAREK656MDPQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789716087 (103h9m40.638323176s) [noise=true , address=79.127.205.233:4645 , piece_id=6YAS5E7QYEZAJXB6HUY3R4PLHC4Z7EYTMSFWNFWD5AP7HKXOKTUQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789937996 (103h9m40.638545078s) [noise=true , address=104.36.82.53:4121 , piece_id=GJLXLW2A5RZUF4MBUP5AZAWFKTWGBDADIIWVIUBXPQPPGPKYZ4YA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.790288745 (103h9m40.638895835s) [noise=true , address=77.90.31.15:4708 , piece_id=67LX4NZJUKLYHRTSW3RKHHXZRKMGWJ2KV32FDA2KE3DY364CJ7OA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.790730467 (103h9m40.639337554s) [noise=true , address=45.91.116.14:4206 , piece_id=OVX63SL5OGJTT7HWDRIUV4BDVEKN2MD4ORUUK5M4JIEWX4J6VHFA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.791174576 (103h9m40.639781656s) [noise=true , address=45.91.116.13:4176 , piece_id=GOWXIBOORGZWFIBLMG46HQ4J632B3E6I3MVXNLTWPP2TY3IF2JUQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.898860128 (103h9m40.747467214s) [upload=82448]: Next returned nil segment
Jul 23 05:59:58.898881813 (103h9m40.747488906s) [upload=82448]: Waiting for error group managing segments...
Jul 23 05:59:58.905078785 (103h9m40.753685854s) [noise=true , address=45.91.116.14:4206 , piece_id=OVX63SL5OGJTT7HWDRIUV4BDVEKN2MD4ORUUK5M4JIEWX4J6VHFA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.906441982 (103h9m40.755049063s) [noise=true , address=77.90.31.15:4708 , piece_id=67LX4NZJUKLYHRTSW3RKHHXZRKMGWJ2KV32FDA2KE3DY364CJ7OA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.910115252 (103h9m40.758722337s) [noise=true , address=77.90.31.15:4706 , piece_id=C3JOUPYB77PA2EHPSNPMDETERE724PBALBCSZ6MJCTZ6YRPUC2UA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.910652695 (103h9m40.759259762s) [noise=true , address=104.36.82.9:4035 , piece_id=74BSWBLVWYQXPUI2PGV5NIJDSE6MSFKWK65TCVV6CNZKVQBQVNEA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.910672036 (103h9m40.759279117s) [noise=true , address=77.90.31.11:4569 , piece_id=MTNLU22JHOYH6VSRMWLKQGSCDI5GIADPIXBVVSC2RIJLTHPMXTOA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.911483080 (103h9m40.760090146s) [noise=true , address=45.91.116.13:4176 , piece_id=GOWXIBOORGZWFIBLMG46HQ4J632B3E6I3MVXNLTWPP2TY3IF2JUQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.911951970 (103h9m40.760559052s) [noise=true , address=45.91.116.10:4103 , piece_id=UGRDLKKFJRNCCDNDQTAY2LLOIS7CGI6KLNDZ7KJDOUNYWFNUAYQQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.912044937 (103h9m40.760652006s) [noise=true , address=77.90.31.12:4622 , piece_id=EZJAG75JRZTYILS7545KYAI37YOUTHHIOOSH26KIWMUQ4BCG556A, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.912649865 (103h9m40.761256931s) [noise=true , address=77.90.31.17:4799 , piece_id=RUTSDNZEXKJ5Y7WQBIVTMFUIJRYLL6DDNNJEXZAL5S7CIATWYHBQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.913487977 (103h9m40.76209505s) [noise=true , address=79.127.219.47:4040 , piece_id=BKWXBAVFXQ2RKG27WGPEAWG5UTHBVODW2BB4EGKHV6SN4EFI7KIQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.914486860 (103h9m40.763093927s) [noise=true , address=104.36.82.53:4121 , piece_id=GJLXLW2A5RZUF4MBUP5AZAWFKTWGBDADIIWVIUBXPQPPGPKYZ4YA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.918080770 (103h9m40.766687852s) [noise=true , address=104.36.82.5:4870 , piece_id=AYTLB2OL7OAI5EO2ZE6LXIQLCGSGDXKDKT6V5HXIPAREK656MDPQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.924012392 (103h9m40.772619461s) [noise=true , address=79.127.219.51:4202 , piece_id=LJZ5A5HAHG4CGXT5J2WNSMXD7HCYSCBUEYGAGOES3NWIVV34BXEQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.932892483 (103h9m40.78149955s) [noise=true , address=104.36.82.42:4818 , piece_id=A6U7TDLTAZHE5IILHHCZKIHCQINHCJSWTBYNXJV7GLCWTIBNB2VA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.932913099 (103h9m40.781520167s) [seg_pos={0 0}, upload=82448]: Segment reached optimal threshold of 14 pieces.
Jul 23 05:59:58.933104957 (103h9m40.781712037s) [noise=true , address=104.36.82.10:4105 , piece_id=GY72JPHTZOGEVFDG3FM5W5ZSYOPSY3BZXEME2Y37VVR2LR37SIUA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GhDrKMcDEg2RxamqKtiHphpgNdvtZYdHXrjzGsNJFjBFYeEen): protocol: expected piece hash; context canceled; EOF
Jul 23 05:59:58.933505265 (103h9m40.782112345s) [noise=true , address=79.127.205.233:4645 , piece_id=6YAS5E7QYEZAJXB6HUY3R4PLHC4Z7EYTMSFWNFWD5AP7HKXOKTUQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12NR52pweTqZNNMTyc1EHSbifDKB4awrnnb4megtZHwANSrR18T): protocol: read tcp 159.223.143.39:46566->79.127.205.233:4645: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 23 05:59:58.933568113 (103h9m40.782175194s) [noise=true , address=79.127.205.235:4680 , piece_id=T3V7PMZXJOHWT77MQ52PUMFNAIAX4DLMTIIVL6XNVTNA565O5WCQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GGvQQLoBcTniZST8Hz9h8eCLWbU4DrXn66DGy33iGLXaFgJLj): protocol: read tcp 159.223.143.39:45394->79.127.205.235:4680: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 23 05:59:58.933618915 (103h9m40.782225986s) [noise=true , address=79.127.205.246:4280 , piece_id=NPBN75MFQYLCG4XKRJPLHJU5JCYBNQYIFQ7MKKB3JKXLT4TKF7AA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rZDBvhrfNrYQB6h9MGhx1tNtQ7AH9ehUF4SfNMCJRpv4SKrDX): protocol: read tcp 159.223.143.39:45458->79.127.205.246:4280: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 23 05:59:58.933631191 (103h9m40.78223826s) [seg_pos={0 0}, upload=82448]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jul 23 05:59:58.933756265 (103h9m40.782363345s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 23 05:59:58.933762566 (103h9m40.782369645s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 23 05:59:59.025408726 (103h9m40.874015808s) [upload=82448]: Done uploading segments...
Jul 23 05:59:59.025427547 (103h9m40.874034627s) [upload=82448]: Upload finished. err: <nil>
Duration: 688.873352ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................] duration:
508.948872ms
FAIL (2025-07-23 06:13:44 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
23 06:00:19.742341858 (103h10m1.590948949s) [noise=true , address=104.36.82.4:4804 , piece_id=SMCQKI5GIWKEHYLNSVUGHWQNBHP3S4C5RBB4E3K5CFMXE54Q4WAQ, seg_pos={0 0}, upload=82455]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g): protocol: expected piece hash; context canceled; EOF
Jul 23 06:00:19.742959346 (103h10m1.59156642s) [noise=true , address=77.90.31.16:4734 , piece_id=NFCPPB6P25YMO5UZRM4CQKMNRV5Q2FXMSR6M4VSTEERBMVNLYJJA, seg_pos={0 0}, upload=82455]: Done uploading piece. err: <nil>
Jul 23 06:00:19.743002232 (103h10m1.591609306s) [seg_pos={0 0}, upload=82455]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 23 06:00:19.743286680 (103h10m1.591893764s) [upload=82455]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 23 06:00:19.743293769 (103h10m1.591900841s) [upload=82455]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 23 06:00:19.815641707 (103h10m1.664248798s) [upload=82455]: Done uploading segments...
Jul 23 06:00:19.815666887 (103h10m1.664273954s) [upload=82455]: Upload finished. err: <nil>
Jul 23 06:01:08.401614022 (103h10m50.250221093s) [upload=82463]: Starting upload
Jul 23 06:01:08.401709823 (103h10m50.250316908s) [upload=82463]: Done starting upload
Jul 23 06:01:08.402138436 (103h10m50.250745505s) [upload=82463]: Uploading segments...
Jul 23 06:01:08.402189044 (103h10m50.250796137s) [upload=82463]: Got next segment. Inline: false
Jul 23 06:01:08.402231416 (103h10m50.250838487s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 23 06:01:08.402237731 (103h10m50.250844804s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 23 06:01:08.450301042 (103h10m50.298908146s) [seg_pos={0 0}, upload=82463]: Begin upload segment...
Jul 23 06:01:08.450364720 (103h10m50.298971798s) [seg_pos={0 0}, upload=82463]: Done begin upload segment.
Jul 23 06:01:08.450425015 (103h10m50.2990321s) [noise=true , address=45.91.116.5:4299 , piece_id=K3URA4ZO4R2CAGZLTO6YJTZQOZ6YIOB7XOJ3AQAVGDFIMVU63BIA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.450431954 (103h10m50.299039031s) [noise=true , address=45.91.116.7:4010 , piece_id=WVMOM2PPTVXY7NYI5UH25PFW6RODIUJWN73YB6FW6UOAMNY4C4SA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.450900413 (103h10m50.299507502s) [noise=true , address=77.90.31.17:4772 , piece_id=63V72ATK3KI3ADRC4N5HTRIXSEB3DRM7N73MVZNDPZJ7EVO3RZJA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.450963993 (103h10m50.299571083s) [noise=true , address=45.91.116.8:4048 , piece_id=AROG4HVUX7R6MHG3DT3N2FIQJBBEVVEVKQHSAGZCWMISWTFSQMAQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.451375091 (103h10m50.299982178s) [noise=true , address=45.91.116.13:4155 , piece_id=KMDEG3QMJNHRF7TETZAFVN3TOZWTQNW7IRTCVI3WLNE7NJAAAPCA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.451400410 (103h10m50.300007509s) [noise=true , address=77.90.31.12:4592 , piece_id=NAWAKPN4LUPCFZIXDIEPN2GKO3CSGX35LZNHI4QDCGYBYAMWBGOA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.451837258 (103h10m50.300444352s) [noise=true , address=77.90.31.17:4771 , piece_id=VCHLVD4PCT6SGJNCFA4LSEDIXSRMX27JH7VLUXDBM2PMIGZ54CNA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.452344092 (103h10m50.300951186s) [noise=true , address=45.91.116.4:4937 , piece_id=DZILXASXCQNDQGTAIYQIY4LCXHENKF6LLWHME5ILDWL3IZOSWMCQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.452773889 (103h10m50.301380982s) [noise=true , address=45.91.116.11:4119 , piece_id=LOMOQU77BVHAPGZ23ZY6CG63FZFQX2IG5QRY2P46WKSGB5AF5O3Q, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.453168858 (103h10m50.301775945s) [noise=true , address=45.91.116.5:4298 , piece_id=AIURH3V65NZD3QI3A5YGBJ222ANC4LTHEYFWTOFIC6SPVH3O4CKA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.453559736 (103h10m50.302166813s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQJDPJRZQEJHTED4UJBNSJ2CGAL6FPTLLIIRFTOVBUK65VSPLGA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.460854171 (103h10m50.309461265s) [noise=true , address=77.90.31.14:4661 , piece_id=MA3CRNAREYTT6ZDO7Q7QRSAOVQQ3EM3CHZTWYQYE6QOWNMYQBBQQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.462111720 (103h10m50.310718803s) [noise=true , address=45.91.116.12:4225 , piece_id=MFM7FUANQ2WU4GK46NPKKKFDUXSVVQ3MGK7GM3QBR7GMOD563PTQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.462819102 (103h10m50.311426189s) [noise=true , address=104.36.82.13:4186 , piece_id=O5BVVJBIWINWK2GXWWJDETED6T3Z462AAUQWCG6LGC7PVVFSHZBA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.463600977 (103h10m50.312208063s) [noise=true , address=45.91.116.7:4011 , piece_id=GKVDRTZ2TY2ESEKU2FKIJND4LXDMITNWQORE4Z47OQ454UZMM7BA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.464356397 (103h10m50.312963485s) [noise=true , address=77.90.31.13:4626 , piece_id=D63E7JGO5RRTEJGXVUMNEC3UYXHS45HBOB7S67VPCWLHJXNH7OSQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.587088382 (103h10m50.435695454s) [upload=82463]: Next returned nil segment
Jul 23 06:01:08.587107670 (103h10m50.435714757s) [upload=82463]: Waiting for error group managing segments...
Jul 23 06:01:08.609523184 (103h10m50.458130271s) [noise=true , address=77.90.31.14:4661 , piece_id=MA3CRNAREYTT6ZDO7Q7QRSAOVQQ3EM3CHZTWYQYE6QOWNMYQBBQQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.612539671 (103h10m50.46114674s) [noise=true , address=45.91.116.5:4299 , piece_id=K3URA4ZO4R2CAGZLTO6YJTZQOZ6YIOB7XOJ3AQAVGDFIMVU63BIA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.617758941 (103h10m50.466366025s) [noise=true , address=45.91.116.13:4155 , piece_id=KMDEG3QMJNHRF7TETZAFVN3TOZWTQNW7IRTCVI3WLNE7NJAAAPCA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.618513849 (103h10m50.467120922s) [noise=true , address=45.91.116.8:4048 , piece_id=AROG4HVUX7R6MHG3DT3N2FIQJBBEVVEVKQHSAGZCWMISWTFSQMAQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.624184177 (103h10m50.47279126s) [noise=true , address=45.91.116.7:4010 , piece_id=WVMOM2PPTVXY7NYI5UH25PFW6RODIUJWN73YB6FW6UOAMNY4C4SA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.625290831 (103h10m50.473897916s) [noise=true , address=45.91.116.7:4011 , piece_id=GKVDRTZ2TY2ESEKU2FKIJND4LXDMITNWQORE4Z47OQ454UZMM7BA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.628897649 (103h10m50.477504746s) [noise=true , address=45.91.116.5:4298 , piece_id=AIURH3V65NZD3QI3A5YGBJ222ANC4LTHEYFWTOFIC6SPVH3O4CKA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.629542622 (103h10m50.478149697s) [noise=true , address=45.91.116.12:4225 , piece_id=MFM7FUANQ2WU4GK46NPKKKFDUXSVVQ3MGK7GM3QBR7GMOD563PTQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.630003986 (103h10m50.478611073s) [noise=true , address=77.90.31.17:4772 , piece_id=63V72ATK3KI3ADRC4N5HTRIXSEB3DRM7N73MVZNDPZJ7EVO3RZJA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.630441959 (103h10m50.479049047s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQJDPJRZQEJHTED4UJBNSJ2CGAL6FPTLLIIRFTOVBUK65VSPLGA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.630941221 (103h10m50.479548305s) [noise=true , address=77.90.31.12:4592 , piece_id=NAWAKPN4LUPCFZIXDIEPN2GKO3CSGX35LZNHI4QDCGYBYAMWBGOA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.631378970 (103h10m50.47998605s) [noise=true , address=77.90.31.13:4626 , piece_id=D63E7JGO5RRTEJGXVUMNEC3UYXHS45HBOB7S67VPCWLHJXNH7OSQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.631840658 (103h10m50.480447733s) [noise=true , address=77.90.31.17:4771 , piece_id=VCHLVD4PCT6SGJNCFA4LSEDIXSRMX27JH7VLUXDBM2PMIGZ54CNA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.631850523 (103h10m50.480457598s) [seg_pos={0 0}, upload=82463]: Segment reached optimal threshold of 13 pieces.
Jul 23 06:01:08.631982956 (103h10m50.480590031s) [noise=true , address=104.36.82.13:4186 , piece_id=O5BVVJBIWINWK2GXWWJDETED6T3Z462AAUQWCG6LGC7PVVFSHZBA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF
Jul 23 06:01:08.632087803 (103h10m50.480694892s) [noise=true , address=45.91.116.4:4937 , piece_id=DZILXASXCQNDQGTAIYQIY4LCXHENKF6LLWHME5ILDWL3IZOSWMCQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF
Jul 23 06:01:08.633101069 (103h10m50.481708157s) [noise=true , address=45.91.116.11:4119 , piece_id=LOMOQU77BVHAPGZ23ZY6CG63FZFQX2IG5QRY2P46WKSGB5AF5O3Q, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.633132335 (103h10m50.481739428s) [seg_pos={0 0}, upload=82463]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 23 06:01:08.633221956 (103h10m50.481829049s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 23 06:01:08.633272252 (103h10m50.481879341s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 23 06:01:08.710605841 (103h10m50.559212933s) [upload=82463]: Done uploading segments...
Jul 23 06:01:08.710632085 (103h10m50.559239157s) [upload=82463]: Upload finished. err: <nil>
Duration: 508.995135ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............] duration:
677.414666ms
FAIL (2025-07-23 06:13:24 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
s) [upload=82448]: Got next segment. Inline: false
Jul 23 05:59:58.732247463 (103h9m40.580854536s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 23 05:59:58.732253833 (103h9m40.580860918s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 23 05:59:58.787057645 (103h9m40.635664731s) [seg_pos={0 0}, upload=82448]: Begin upload segment...
Jul 23 05:59:58.787102519 (103h9m40.635709584s) [seg_pos={0 0}, upload=82448]: Done begin upload segment.
Jul 23 05:59:58.787136403 (103h9m40.635743482s) [noise=true , address=77.90.31.12:4622 , piece_id=EZJAG75JRZTYILS7545KYAI37YOUTHHIOOSH26KIWMUQ4BCG556A, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787169354 (103h9m40.635776438s) [noise=true , address=79.127.205.235:4680 , piece_id=T3V7PMZXJOHWT77MQ52PUMFNAIAX4DLMTIIVL6XNVTNA565O5WCQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787544550 (103h9m40.636151681s) [noise=true , address=104.36.82.10:4105 , piece_id=GY72JPHTZOGEVFDG3FM5W5ZSYOPSY3BZXEME2Y37VVR2LR37SIUA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787598825 (103h9m40.636205915s) [noise=true , address=79.127.205.246:4280 , piece_id=NPBN75MFQYLCG4XKRJPLHJU5JCYBNQYIFQ7MKKB3JKXLT4TKF7AA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787946084 (103h9m40.636553167s) [noise=true , address=77.90.31.17:4799 , piece_id=RUTSDNZEXKJ5Y7WQBIVTMFUIJRYLL6DDNNJEXZAL5S7CIATWYHBQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788041731 (103h9m40.636648823s) [noise=true , address=77.90.31.11:4569 , piece_id=MTNLU22JHOYH6VSRMWLKQGSCDI5GIADPIXBVVSC2RIJLTHPMXTOA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788383218 (103h9m40.636990305s) [noise=true , address=104.36.82.42:4818 , piece_id=A6U7TDLTAZHE5IILHHCZKIHCQINHCJSWTBYNXJV7GLCWTIBNB2VA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788633554 (103h9m40.637240644s) [noise=true , address=79.127.219.51:4202 , piece_id=LJZ5A5HAHG4CGXT5J2WNSMXD7HCYSCBUEYGAGOES3NWIVV34BXEQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788735479 (103h9m40.637342568s) [noise=true , address=79.127.219.47:4040 , piece_id=BKWXBAVFXQ2RKG27WGPEAWG5UTHBVODW2BB4EGKHV6SN4EFI7KIQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789072444 (103h9m40.637679533s) [noise=true , address=77.90.31.15:4706 , piece_id=C3JOUPYB77PA2EHPSNPMDETERE724PBALBCSZ6MJCTZ6YRPUC2UA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789122202 (103h9m40.637729291s) [noise=true , address=104.36.82.9:4035 , piece_id=74BSWBLVWYQXPUI2PGV5NIJDSE6MSFKWK65TCVV6CNZKVQBQVNEA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789393961 (103h9m40.638001047s) [noise=true , address=45.91.116.10:4103 , piece_id=UGRDLKKFJRNCCDNDQTAY2LLOIS7CGI6KLNDZ7KJDOUNYWFNUAYQQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789591397 (103h9m40.638198479s) [noise=true , address=104.36.82.5:4870 , piece_id=AYTLB2OL7OAI5EO2ZE6LXIQLCGSGDXKDKT6V5HXIPAREK656MDPQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789716087 (103h9m40.638323176s) [noise=true , address=79.127.205.233:4645 , piece_id=6YAS5E7QYEZAJXB6HUY3R4PLHC4Z7EYTMSFWNFWD5AP7HKXOKTUQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789937996 (103h9m40.638545078s) [noise=true , address=104.36.82.53:4121 , piece_id=GJLXLW2A5RZUF4MBUP5AZAWFKTWGBDADIIWVIUBXPQPPGPKYZ4YA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.790288745 (103h9m40.638895835s) [noise=true , address=77.90.31.15:4708 , piece_id=67LX4NZJUKLYHRTSW3RKHHXZRKMGWJ2KV32FDA2KE3DY364CJ7OA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.790730467 (103h9m40.639337554s) [noise=true , address=45.91.116.14:4206 , piece_id=OVX63SL5OGJTT7HWDRIUV4BDVEKN2MD4ORUUK5M4JIEWX4J6VHFA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.791174576 (103h9m40.639781656s) [noise=true , address=45.91.116.13:4176 , piece_id=GOWXIBOORGZWFIBLMG46HQ4J632B3E6I3MVXNLTWPP2TY3IF2JUQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.898860128 (103h9m40.747467214s) [upload=82448]: Next returned nil segment
Jul 23 05:59:58.898881813 (103h9m40.747488906s) [upload=82448]: Waiting for error group managing segments...
Jul 23 05:59:58.905078785 (103h9m40.753685854s) [noise=true , address=45.91.116.14:4206 , piece_id=OVX63SL5OGJTT7HWDRIUV4BDVEKN2MD4ORUUK5M4JIEWX4J6VHFA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.906441982 (103h9m40.755049063s) [noise=true , address=77.90.31.15:4708 , piece_id=67LX4NZJUKLYHRTSW3RKHHXZRKMGWJ2KV32FDA2KE3DY364CJ7OA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.910115252 (103h9m40.758722337s) [noise=true , address=77.90.31.15:4706 , piece_id=C3JOUPYB77PA2EHPSNPMDETERE724PBALBCSZ6MJCTZ6YRPUC2UA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.910652695 (103h9m40.759259762s) [noise=true , address=104.36.82.9:4035 , piece_id=74BSWBLVWYQXPUI2PGV5NIJDSE6MSFKWK65TCVV6CNZKVQBQVNEA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.910672036 (103h9m40.759279117s) [noise=true , address=77.90.31.11:4569 , piece_id=MTNLU22JHOYH6VSRMWLKQGSCDI5GIADPIXBVVSC2RIJLTHPMXTOA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.911483080 (103h9m40.760090146s) [noise=true , address=45.91.116.13:4176 , piece_id=GOWXIBOORGZWFIBLMG46HQ4J632B3E6I3MVXNLTWPP2TY3IF2JUQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.911951970 (103h9m40.760559052s) [noise=true , address=45.91.116.10:4103 , piece_id=UGRDLKKFJRNCCDNDQTAY2LLOIS7CGI6KLNDZ7KJDOUNYWFNUAYQQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.912044937 (103h9m40.760652006s) [noise=true , address=77.90.31.12:4622 , piece_id=EZJAG75JRZTYILS7545KYAI37YOUTHHIOOSH26KIWMUQ4BCG556A, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.912649865 (103h9m40.761256931s) [noise=true , address=77.90.31.17:4799 , piece_id=RUTSDNZEXKJ5Y7WQBIVTMFUIJRYLL6DDNNJEXZAL5S7CIATWYHBQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.913487977 (103h9m40.76209505s) [noise=true , address=79.127.219.47:4040 , piece_id=BKWXBAVFXQ2RKG27WGPEAWG5UTHBVODW2BB4EGKHV6SN4EFI7KIQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.914486860 (103h9m40.763093927s) [noise=true , address=104.36.82.53:4121 , piece_id=GJLXLW2A5RZUF4MBUP5AZAWFKTWGBDADIIWVIUBXPQPPGPKYZ4YA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.918080770 (103h9m40.766687852s) [noise=true , address=104.36.82.5:4870 , piece_id=AYTLB2OL7OAI5EO2ZE6LXIQLCGSGDXKDKT6V5HXIPAREK656MDPQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.924012392 (103h9m40.772619461s) [noise=true , address=79.127.219.51:4202 , piece_id=LJZ5A5HAHG4CGXT5J2WNSMXD7HCYSCBUEYGAGOES3NWIVV34BXEQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.932892483 (103h9m40.78149955s) [noise=true , address=104.36.82.42:4818 , piece_id=A6U7TDLTAZHE5IILHHCZKIHCQINHCJSWTBYNXJV7GLCWTIBNB2VA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.932913099 (103h9m40.781520167s) [seg_pos={0 0}, upload=82448]: Segment reached optimal threshold of 14 pieces.
Jul 23 05:59:58.933104957 (103h9m40.781712037s) [noise=true , address=104.36.82.10:4105 , piece_id=GY72JPHTZOGEVFDG3FM5W5ZSYOPSY3BZXEME2Y37VVR2LR37SIUA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GhDrKMcDEg2RxamqKtiHphpgNdvtZYdHXrjzGsNJFjBFYeEen): protocol: expected piece hash; context canceled; EOF
Jul 23 05:59:58.933505265 (103h9m40.782112345s) [noise=true , address=79.127.205.233:4645 , piece_id=6YAS5E7QYEZAJXB6HUY3R4PLHC4Z7EYTMSFWNFWD5AP7HKXOKTUQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12NR52pweTqZNNMTyc1EHSbifDKB4awrnnb4megtZHwANSrR18T): protocol: read tcp 159.223.143.39:46566->79.127.205.233:4645: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 23 05:59:58.933568113 (103h9m40.782175194s) [noise=true , address=79.127.205.235:4680 , piece_id=T3V7PMZXJOHWT77MQ52PUMFNAIAX4DLMTIIVL6XNVTNA565O5WCQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GGvQQLoBcTniZST8Hz9h8eCLWbU4DrXn66DGy33iGLXaFgJLj): protocol: read tcp 159.223.143.39:45394->79.127.205.235:4680: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 23 05:59:58.933618915 (103h9m40.782225986s) [noise=true , address=79.127.205.246:4280 , piece_id=NPBN75MFQYLCG4XKRJPLHJU5JCYBNQYIFQ7MKKB3JKXLT4TKF7AA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rZDBvhrfNrYQB6h9MGhx1tNtQ7AH9ehUF4SfNMCJRpv4SKrDX): protocol: read tcp 159.223.143.39:45458->79.127.205.246:4280: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 23 05:59:58.933631191 (103h9m40.78223826s) [seg_pos={0 0}, upload=82448]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jul 23 05:59:58.933756265 (103h9m40.782363345s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 23 05:59:58.933762566 (103h9m40.782369645s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 23 05:59:59.025408726 (103h9m40.874015808s) [upload=82448]: Done uploading segments...
Jul 23 05:59:59.025427547 (103h9m40.874034627s) [upload=82448]: Upload finished. err: <nil>
Duration: 677.440873ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............] duration:
563.778065ms
FAIL (2025-07-23 06:13:23 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
23 06:00:19.742341858 (103h10m1.590948949s) [noise=true , address=104.36.82.4:4804 , piece_id=SMCQKI5GIWKEHYLNSVUGHWQNBHP3S4C5RBB4E3K5CFMXE54Q4WAQ, seg_pos={0 0}, upload=82455]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g): protocol: expected piece hash; context canceled; EOF
Jul 23 06:00:19.742959346 (103h10m1.59156642s) [noise=true , address=77.90.31.16:4734 , piece_id=NFCPPB6P25YMO5UZRM4CQKMNRV5Q2FXMSR6M4VSTEERBMVNLYJJA, seg_pos={0 0}, upload=82455]: Done uploading piece. err: <nil>
Jul 23 06:00:19.743002232 (103h10m1.591609306s) [seg_pos={0 0}, upload=82455]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 23 06:00:19.743286680 (103h10m1.591893764s) [upload=82455]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 23 06:00:19.743293769 (103h10m1.591900841s) [upload=82455]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 23 06:00:19.815641707 (103h10m1.664248798s) [upload=82455]: Done uploading segments...
Jul 23 06:00:19.815666887 (103h10m1.664273954s) [upload=82455]: Upload finished. err: <nil>
Jul 23 06:01:08.401614022 (103h10m50.250221093s) [upload=82463]: Starting upload
Jul 23 06:01:08.401709823 (103h10m50.250316908s) [upload=82463]: Done starting upload
Jul 23 06:01:08.402138436 (103h10m50.250745505s) [upload=82463]: Uploading segments...
Jul 23 06:01:08.402189044 (103h10m50.250796137s) [upload=82463]: Got next segment. Inline: false
Jul 23 06:01:08.402231416 (103h10m50.250838487s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 23 06:01:08.402237731 (103h10m50.250844804s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 23 06:01:08.450301042 (103h10m50.298908146s) [seg_pos={0 0}, upload=82463]: Begin upload segment...
Jul 23 06:01:08.450364720 (103h10m50.298971798s) [seg_pos={0 0}, upload=82463]: Done begin upload segment.
Jul 23 06:01:08.450425015 (103h10m50.2990321s) [noise=true , address=45.91.116.5:4299 , piece_id=K3URA4ZO4R2CAGZLTO6YJTZQOZ6YIOB7XOJ3AQAVGDFIMVU63BIA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.450431954 (103h10m50.299039031s) [noise=true , address=45.91.116.7:4010 , piece_id=WVMOM2PPTVXY7NYI5UH25PFW6RODIUJWN73YB6FW6UOAMNY4C4SA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.450900413 (103h10m50.299507502s) [noise=true , address=77.90.31.17:4772 , piece_id=63V72ATK3KI3ADRC4N5HTRIXSEB3DRM7N73MVZNDPZJ7EVO3RZJA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.450963993 (103h10m50.299571083s) [noise=true , address=45.91.116.8:4048 , piece_id=AROG4HVUX7R6MHG3DT3N2FIQJBBEVVEVKQHSAGZCWMISWTFSQMAQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.451375091 (103h10m50.299982178s) [noise=true , address=45.91.116.13:4155 , piece_id=KMDEG3QMJNHRF7TETZAFVN3TOZWTQNW7IRTCVI3WLNE7NJAAAPCA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.451400410 (103h10m50.300007509s) [noise=true , address=77.90.31.12:4592 , piece_id=NAWAKPN4LUPCFZIXDIEPN2GKO3CSGX35LZNHI4QDCGYBYAMWBGOA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.451837258 (103h10m50.300444352s) [noise=true , address=77.90.31.17:4771 , piece_id=VCHLVD4PCT6SGJNCFA4LSEDIXSRMX27JH7VLUXDBM2PMIGZ54CNA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.452344092 (103h10m50.300951186s) [noise=true , address=45.91.116.4:4937 , piece_id=DZILXASXCQNDQGTAIYQIY4LCXHENKF6LLWHME5ILDWL3IZOSWMCQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.452773889 (103h10m50.301380982s) [noise=true , address=45.91.116.11:4119 , piece_id=LOMOQU77BVHAPGZ23ZY6CG63FZFQX2IG5QRY2P46WKSGB5AF5O3Q, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.453168858 (103h10m50.301775945s) [noise=true , address=45.91.116.5:4298 , piece_id=AIURH3V65NZD3QI3A5YGBJ222ANC4LTHEYFWTOFIC6SPVH3O4CKA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.453559736 (103h10m50.302166813s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQJDPJRZQEJHTED4UJBNSJ2CGAL6FPTLLIIRFTOVBUK65VSPLGA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.460854171 (103h10m50.309461265s) [noise=true , address=77.90.31.14:4661 , piece_id=MA3CRNAREYTT6ZDO7Q7QRSAOVQQ3EM3CHZTWYQYE6QOWNMYQBBQQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.462111720 (103h10m50.310718803s) [noise=true , address=45.91.116.12:4225 , piece_id=MFM7FUANQ2WU4GK46NPKKKFDUXSVVQ3MGK7GM3QBR7GMOD563PTQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.462819102 (103h10m50.311426189s) [noise=true , address=104.36.82.13:4186 , piece_id=O5BVVJBIWINWK2GXWWJDETED6T3Z462AAUQWCG6LGC7PVVFSHZBA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.463600977 (103h10m50.312208063s) [noise=true , address=45.91.116.7:4011 , piece_id=GKVDRTZ2TY2ESEKU2FKIJND4LXDMITNWQORE4Z47OQ454UZMM7BA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.464356397 (103h10m50.312963485s) [noise=true , address=77.90.31.13:4626 , piece_id=D63E7JGO5RRTEJGXVUMNEC3UYXHS45HBOB7S67VPCWLHJXNH7OSQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.587088382 (103h10m50.435695454s) [upload=82463]: Next returned nil segment
Jul 23 06:01:08.587107670 (103h10m50.435714757s) [upload=82463]: Waiting for error group managing segments...
Jul 23 06:01:08.609523184 (103h10m50.458130271s) [noise=true , address=77.90.31.14:4661 , piece_id=MA3CRNAREYTT6ZDO7Q7QRSAOVQQ3EM3CHZTWYQYE6QOWNMYQBBQQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.612539671 (103h10m50.46114674s) [noise=true , address=45.91.116.5:4299 , piece_id=K3URA4ZO4R2CAGZLTO6YJTZQOZ6YIOB7XOJ3AQAVGDFIMVU63BIA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.617758941 (103h10m50.466366025s) [noise=true , address=45.91.116.13:4155 , piece_id=KMDEG3QMJNHRF7TETZAFVN3TOZWTQNW7IRTCVI3WLNE7NJAAAPCA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.618513849 (103h10m50.467120922s) [noise=true , address=45.91.116.8:4048 , piece_id=AROG4HVUX7R6MHG3DT3N2FIQJBBEVVEVKQHSAGZCWMISWTFSQMAQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.624184177 (103h10m50.47279126s) [noise=true , address=45.91.116.7:4010 , piece_id=WVMOM2PPTVXY7NYI5UH25PFW6RODIUJWN73YB6FW6UOAMNY4C4SA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.625290831 (103h10m50.473897916s) [noise=true , address=45.91.116.7:4011 , piece_id=GKVDRTZ2TY2ESEKU2FKIJND4LXDMITNWQORE4Z47OQ454UZMM7BA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.628897649 (103h10m50.477504746s) [noise=true , address=45.91.116.5:4298 , piece_id=AIURH3V65NZD3QI3A5YGBJ222ANC4LTHEYFWTOFIC6SPVH3O4CKA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.629542622 (103h10m50.478149697s) [noise=true , address=45.91.116.12:4225 , piece_id=MFM7FUANQ2WU4GK46NPKKKFDUXSVVQ3MGK7GM3QBR7GMOD563PTQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.630003986 (103h10m50.478611073s) [noise=true , address=77.90.31.17:4772 , piece_id=63V72ATK3KI3ADRC4N5HTRIXSEB3DRM7N73MVZNDPZJ7EVO3RZJA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.630441959 (103h10m50.479049047s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQJDPJRZQEJHTED4UJBNSJ2CGAL6FPTLLIIRFTOVBUK65VSPLGA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.630941221 (103h10m50.479548305s) [noise=true , address=77.90.31.12:4592 , piece_id=NAWAKPN4LUPCFZIXDIEPN2GKO3CSGX35LZNHI4QDCGYBYAMWBGOA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.631378970 (103h10m50.47998605s) [noise=true , address=77.90.31.13:4626 , piece_id=D63E7JGO5RRTEJGXVUMNEC3UYXHS45HBOB7S67VPCWLHJXNH7OSQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.631840658 (103h10m50.480447733s) [noise=true , address=77.90.31.17:4771 , piece_id=VCHLVD4PCT6SGJNCFA4LSEDIXSRMX27JH7VLUXDBM2PMIGZ54CNA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.631850523 (103h10m50.480457598s) [seg_pos={0 0}, upload=82463]: Segment reached optimal threshold of 13 pieces.
Jul 23 06:01:08.631982956 (103h10m50.480590031s) [noise=true , address=104.36.82.13:4186 , piece_id=O5BVVJBIWINWK2GXWWJDETED6T3Z462AAUQWCG6LGC7PVVFSHZBA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF
Jul 23 06:01:08.632087803 (103h10m50.480694892s) [noise=true , address=45.91.116.4:4937 , piece_id=DZILXASXCQNDQGTAIYQIY4LCXHENKF6LLWHME5ILDWL3IZOSWMCQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF
Jul 23 06:01:08.633101069 (103h10m50.481708157s) [noise=true , address=45.91.116.11:4119 , piece_id=LOMOQU77BVHAPGZ23ZY6CG63FZFQX2IG5QRY2P46WKSGB5AF5O3Q, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.633132335 (103h10m50.481739428s) [seg_pos={0 0}, upload=82463]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 23 06:01:08.633221956 (103h10m50.481829049s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 23 06:01:08.633272252 (103h10m50.481879341s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 23 06:01:08.710605841 (103h10m50.559212933s) [upload=82463]: Done uploading segments...
Jul 23 06:01:08.710632085 (103h10m50.559239157s) [upload=82463]: Upload finished. err: <nil>
Duration: 563.812327ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............] duration:
954.127716ms
FAIL (2025-07-23 06:12:45 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
s) [upload=82448]: Got next segment. Inline: false
Jul 23 05:59:58.732247463 (103h9m40.580854536s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 23 05:59:58.732253833 (103h9m40.580860918s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 23 05:59:58.787057645 (103h9m40.635664731s) [seg_pos={0 0}, upload=82448]: Begin upload segment...
Jul 23 05:59:58.787102519 (103h9m40.635709584s) [seg_pos={0 0}, upload=82448]: Done begin upload segment.
Jul 23 05:59:58.787136403 (103h9m40.635743482s) [noise=true , address=77.90.31.12:4622 , piece_id=EZJAG75JRZTYILS7545KYAI37YOUTHHIOOSH26KIWMUQ4BCG556A, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787169354 (103h9m40.635776438s) [noise=true , address=79.127.205.235:4680 , piece_id=T3V7PMZXJOHWT77MQ52PUMFNAIAX4DLMTIIVL6XNVTNA565O5WCQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787544550 (103h9m40.636151681s) [noise=true , address=104.36.82.10:4105 , piece_id=GY72JPHTZOGEVFDG3FM5W5ZSYOPSY3BZXEME2Y37VVR2LR37SIUA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787598825 (103h9m40.636205915s) [noise=true , address=79.127.205.246:4280 , piece_id=NPBN75MFQYLCG4XKRJPLHJU5JCYBNQYIFQ7MKKB3JKXLT4TKF7AA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787946084 (103h9m40.636553167s) [noise=true , address=77.90.31.17:4799 , piece_id=RUTSDNZEXKJ5Y7WQBIVTMFUIJRYLL6DDNNJEXZAL5S7CIATWYHBQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788041731 (103h9m40.636648823s) [noise=true , address=77.90.31.11:4569 , piece_id=MTNLU22JHOYH6VSRMWLKQGSCDI5GIADPIXBVVSC2RIJLTHPMXTOA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788383218 (103h9m40.636990305s) [noise=true , address=104.36.82.42:4818 , piece_id=A6U7TDLTAZHE5IILHHCZKIHCQINHCJSWTBYNXJV7GLCWTIBNB2VA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788633554 (103h9m40.637240644s) [noise=true , address=79.127.219.51:4202 , piece_id=LJZ5A5HAHG4CGXT5J2WNSMXD7HCYSCBUEYGAGOES3NWIVV34BXEQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788735479 (103h9m40.637342568s) [noise=true , address=79.127.219.47:4040 , piece_id=BKWXBAVFXQ2RKG27WGPEAWG5UTHBVODW2BB4EGKHV6SN4EFI7KIQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789072444 (103h9m40.637679533s) [noise=true , address=77.90.31.15:4706 , piece_id=C3JOUPYB77PA2EHPSNPMDETERE724PBALBCSZ6MJCTZ6YRPUC2UA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789122202 (103h9m40.637729291s) [noise=true , address=104.36.82.9:4035 , piece_id=74BSWBLVWYQXPUI2PGV5NIJDSE6MSFKWK65TCVV6CNZKVQBQVNEA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789393961 (103h9m40.638001047s) [noise=true , address=45.91.116.10:4103 , piece_id=UGRDLKKFJRNCCDNDQTAY2LLOIS7CGI6KLNDZ7KJDOUNYWFNUAYQQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789591397 (103h9m40.638198479s) [noise=true , address=104.36.82.5:4870 , piece_id=AYTLB2OL7OAI5EO2ZE6LXIQLCGSGDXKDKT6V5HXIPAREK656MDPQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789716087 (103h9m40.638323176s) [noise=true , address=79.127.205.233:4645 , piece_id=6YAS5E7QYEZAJXB6HUY3R4PLHC4Z7EYTMSFWNFWD5AP7HKXOKTUQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789937996 (103h9m40.638545078s) [noise=true , address=104.36.82.53:4121 , piece_id=GJLXLW2A5RZUF4MBUP5AZAWFKTWGBDADIIWVIUBXPQPPGPKYZ4YA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.790288745 (103h9m40.638895835s) [noise=true , address=77.90.31.15:4708 , piece_id=67LX4NZJUKLYHRTSW3RKHHXZRKMGWJ2KV32FDA2KE3DY364CJ7OA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.790730467 (103h9m40.639337554s) [noise=true , address=45.91.116.14:4206 , piece_id=OVX63SL5OGJTT7HWDRIUV4BDVEKN2MD4ORUUK5M4JIEWX4J6VHFA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.791174576 (103h9m40.639781656s) [noise=true , address=45.91.116.13:4176 , piece_id=GOWXIBOORGZWFIBLMG46HQ4J632B3E6I3MVXNLTWPP2TY3IF2JUQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.898860128 (103h9m40.747467214s) [upload=82448]: Next returned nil segment
Jul 23 05:59:58.898881813 (103h9m40.747488906s) [upload=82448]: Waiting for error group managing segments...
Jul 23 05:59:58.905078785 (103h9m40.753685854s) [noise=true , address=45.91.116.14:4206 , piece_id=OVX63SL5OGJTT7HWDRIUV4BDVEKN2MD4ORUUK5M4JIEWX4J6VHFA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.906441982 (103h9m40.755049063s) [noise=true , address=77.90.31.15:4708 , piece_id=67LX4NZJUKLYHRTSW3RKHHXZRKMGWJ2KV32FDA2KE3DY364CJ7OA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.910115252 (103h9m40.758722337s) [noise=true , address=77.90.31.15:4706 , piece_id=C3JOUPYB77PA2EHPSNPMDETERE724PBALBCSZ6MJCTZ6YRPUC2UA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.910652695 (103h9m40.759259762s) [noise=true , address=104.36.82.9:4035 , piece_id=74BSWBLVWYQXPUI2PGV5NIJDSE6MSFKWK65TCVV6CNZKVQBQVNEA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.910672036 (103h9m40.759279117s) [noise=true , address=77.90.31.11:4569 , piece_id=MTNLU22JHOYH6VSRMWLKQGSCDI5GIADPIXBVVSC2RIJLTHPMXTOA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.911483080 (103h9m40.760090146s) [noise=true , address=45.91.116.13:4176 , piece_id=GOWXIBOORGZWFIBLMG46HQ4J632B3E6I3MVXNLTWPP2TY3IF2JUQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.911951970 (103h9m40.760559052s) [noise=true , address=45.91.116.10:4103 , piece_id=UGRDLKKFJRNCCDNDQTAY2LLOIS7CGI6KLNDZ7KJDOUNYWFNUAYQQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.912044937 (103h9m40.760652006s) [noise=true , address=77.90.31.12:4622 , piece_id=EZJAG75JRZTYILS7545KYAI37YOUTHHIOOSH26KIWMUQ4BCG556A, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.912649865 (103h9m40.761256931s) [noise=true , address=77.90.31.17:4799 , piece_id=RUTSDNZEXKJ5Y7WQBIVTMFUIJRYLL6DDNNJEXZAL5S7CIATWYHBQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.913487977 (103h9m40.76209505s) [noise=true , address=79.127.219.47:4040 , piece_id=BKWXBAVFXQ2RKG27WGPEAWG5UTHBVODW2BB4EGKHV6SN4EFI7KIQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.914486860 (103h9m40.763093927s) [noise=true , address=104.36.82.53:4121 , piece_id=GJLXLW2A5RZUF4MBUP5AZAWFKTWGBDADIIWVIUBXPQPPGPKYZ4YA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.918080770 (103h9m40.766687852s) [noise=true , address=104.36.82.5:4870 , piece_id=AYTLB2OL7OAI5EO2ZE6LXIQLCGSGDXKDKT6V5HXIPAREK656MDPQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.924012392 (103h9m40.772619461s) [noise=true , address=79.127.219.51:4202 , piece_id=LJZ5A5HAHG4CGXT5J2WNSMXD7HCYSCBUEYGAGOES3NWIVV34BXEQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.932892483 (103h9m40.78149955s) [noise=true , address=104.36.82.42:4818 , piece_id=A6U7TDLTAZHE5IILHHCZKIHCQINHCJSWTBYNXJV7GLCWTIBNB2VA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.932913099 (103h9m40.781520167s) [seg_pos={0 0}, upload=82448]: Segment reached optimal threshold of 14 pieces.
Jul 23 05:59:58.933104957 (103h9m40.781712037s) [noise=true , address=104.36.82.10:4105 , piece_id=GY72JPHTZOGEVFDG3FM5W5ZSYOPSY3BZXEME2Y37VVR2LR37SIUA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GhDrKMcDEg2RxamqKtiHphpgNdvtZYdHXrjzGsNJFjBFYeEen): protocol: expected piece hash; context canceled; EOF
Jul 23 05:59:58.933505265 (103h9m40.782112345s) [noise=true , address=79.127.205.233:4645 , piece_id=6YAS5E7QYEZAJXB6HUY3R4PLHC4Z7EYTMSFWNFWD5AP7HKXOKTUQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12NR52pweTqZNNMTyc1EHSbifDKB4awrnnb4megtZHwANSrR18T): protocol: read tcp 159.223.143.39:46566->79.127.205.233:4645: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 23 05:59:58.933568113 (103h9m40.782175194s) [noise=true , address=79.127.205.235:4680 , piece_id=T3V7PMZXJOHWT77MQ52PUMFNAIAX4DLMTIIVL6XNVTNA565O5WCQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GGvQQLoBcTniZST8Hz9h8eCLWbU4DrXn66DGy33iGLXaFgJLj): protocol: read tcp 159.223.143.39:45394->79.127.205.235:4680: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 23 05:59:58.933618915 (103h9m40.782225986s) [noise=true , address=79.127.205.246:4280 , piece_id=NPBN75MFQYLCG4XKRJPLHJU5JCYBNQYIFQ7MKKB3JKXLT4TKF7AA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rZDBvhrfNrYQB6h9MGhx1tNtQ7AH9ehUF4SfNMCJRpv4SKrDX): protocol: read tcp 159.223.143.39:45458->79.127.205.246:4280: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 23 05:59:58.933631191 (103h9m40.78223826s) [seg_pos={0 0}, upload=82448]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jul 23 05:59:58.933756265 (103h9m40.782363345s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 23 05:59:58.933762566 (103h9m40.782369645s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 23 05:59:59.025408726 (103h9m40.874015808s) [upload=82448]: Done uploading segments...
Jul 23 05:59:59.025427547 (103h9m40.874034627s) [upload=82448]: Upload finished. err: <nil>
Duration: 954.164763ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............] duration:
503.548788ms
FAIL (2025-07-23 06:12:22 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
23 06:00:19.742341858 (103h10m1.590948949s) [noise=true , address=104.36.82.4:4804 , piece_id=SMCQKI5GIWKEHYLNSVUGHWQNBHP3S4C5RBB4E3K5CFMXE54Q4WAQ, seg_pos={0 0}, upload=82455]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g): protocol: expected piece hash; context canceled; EOF
Jul 23 06:00:19.742959346 (103h10m1.59156642s) [noise=true , address=77.90.31.16:4734 , piece_id=NFCPPB6P25YMO5UZRM4CQKMNRV5Q2FXMSR6M4VSTEERBMVNLYJJA, seg_pos={0 0}, upload=82455]: Done uploading piece. err: <nil>
Jul 23 06:00:19.743002232 (103h10m1.591609306s) [seg_pos={0 0}, upload=82455]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 23 06:00:19.743286680 (103h10m1.591893764s) [upload=82455]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 23 06:00:19.743293769 (103h10m1.591900841s) [upload=82455]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 23 06:00:19.815641707 (103h10m1.664248798s) [upload=82455]: Done uploading segments...
Jul 23 06:00:19.815666887 (103h10m1.664273954s) [upload=82455]: Upload finished. err: <nil>
Jul 23 06:01:08.401614022 (103h10m50.250221093s) [upload=82463]: Starting upload
Jul 23 06:01:08.401709823 (103h10m50.250316908s) [upload=82463]: Done starting upload
Jul 23 06:01:08.402138436 (103h10m50.250745505s) [upload=82463]: Uploading segments...
Jul 23 06:01:08.402189044 (103h10m50.250796137s) [upload=82463]: Got next segment. Inline: false
Jul 23 06:01:08.402231416 (103h10m50.250838487s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 23 06:01:08.402237731 (103h10m50.250844804s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 23 06:01:08.450301042 (103h10m50.298908146s) [seg_pos={0 0}, upload=82463]: Begin upload segment...
Jul 23 06:01:08.450364720 (103h10m50.298971798s) [seg_pos={0 0}, upload=82463]: Done begin upload segment.
Jul 23 06:01:08.450425015 (103h10m50.2990321s) [noise=true , address=45.91.116.5:4299 , piece_id=K3URA4ZO4R2CAGZLTO6YJTZQOZ6YIOB7XOJ3AQAVGDFIMVU63BIA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.450431954 (103h10m50.299039031s) [noise=true , address=45.91.116.7:4010 , piece_id=WVMOM2PPTVXY7NYI5UH25PFW6RODIUJWN73YB6FW6UOAMNY4C4SA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.450900413 (103h10m50.299507502s) [noise=true , address=77.90.31.17:4772 , piece_id=63V72ATK3KI3ADRC4N5HTRIXSEB3DRM7N73MVZNDPZJ7EVO3RZJA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.450963993 (103h10m50.299571083s) [noise=true , address=45.91.116.8:4048 , piece_id=AROG4HVUX7R6MHG3DT3N2FIQJBBEVVEVKQHSAGZCWMISWTFSQMAQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.451375091 (103h10m50.299982178s) [noise=true , address=45.91.116.13:4155 , piece_id=KMDEG3QMJNHRF7TETZAFVN3TOZWTQNW7IRTCVI3WLNE7NJAAAPCA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.451400410 (103h10m50.300007509s) [noise=true , address=77.90.31.12:4592 , piece_id=NAWAKPN4LUPCFZIXDIEPN2GKO3CSGX35LZNHI4QDCGYBYAMWBGOA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.451837258 (103h10m50.300444352s) [noise=true , address=77.90.31.17:4771 , piece_id=VCHLVD4PCT6SGJNCFA4LSEDIXSRMX27JH7VLUXDBM2PMIGZ54CNA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.452344092 (103h10m50.300951186s) [noise=true , address=45.91.116.4:4937 , piece_id=DZILXASXCQNDQGTAIYQIY4LCXHENKF6LLWHME5ILDWL3IZOSWMCQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.452773889 (103h10m50.301380982s) [noise=true , address=45.91.116.11:4119 , piece_id=LOMOQU77BVHAPGZ23ZY6CG63FZFQX2IG5QRY2P46WKSGB5AF5O3Q, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.453168858 (103h10m50.301775945s) [noise=true , address=45.91.116.5:4298 , piece_id=AIURH3V65NZD3QI3A5YGBJ222ANC4LTHEYFWTOFIC6SPVH3O4CKA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.453559736 (103h10m50.302166813s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQJDPJRZQEJHTED4UJBNSJ2CGAL6FPTLLIIRFTOVBUK65VSPLGA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.460854171 (103h10m50.309461265s) [noise=true , address=77.90.31.14:4661 , piece_id=MA3CRNAREYTT6ZDO7Q7QRSAOVQQ3EM3CHZTWYQYE6QOWNMYQBBQQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.462111720 (103h10m50.310718803s) [noise=true , address=45.91.116.12:4225 , piece_id=MFM7FUANQ2WU4GK46NPKKKFDUXSVVQ3MGK7GM3QBR7GMOD563PTQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.462819102 (103h10m50.311426189s) [noise=true , address=104.36.82.13:4186 , piece_id=O5BVVJBIWINWK2GXWWJDETED6T3Z462AAUQWCG6LGC7PVVFSHZBA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.463600977 (103h10m50.312208063s) [noise=true , address=45.91.116.7:4011 , piece_id=GKVDRTZ2TY2ESEKU2FKIJND4LXDMITNWQORE4Z47OQ454UZMM7BA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.464356397 (103h10m50.312963485s) [noise=true , address=77.90.31.13:4626 , piece_id=D63E7JGO5RRTEJGXVUMNEC3UYXHS45HBOB7S67VPCWLHJXNH7OSQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.587088382 (103h10m50.435695454s) [upload=82463]: Next returned nil segment
Jul 23 06:01:08.587107670 (103h10m50.435714757s) [upload=82463]: Waiting for error group managing segments...
Jul 23 06:01:08.609523184 (103h10m50.458130271s) [noise=true , address=77.90.31.14:4661 , piece_id=MA3CRNAREYTT6ZDO7Q7QRSAOVQQ3EM3CHZTWYQYE6QOWNMYQBBQQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.612539671 (103h10m50.46114674s) [noise=true , address=45.91.116.5:4299 , piece_id=K3URA4ZO4R2CAGZLTO6YJTZQOZ6YIOB7XOJ3AQAVGDFIMVU63BIA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.617758941 (103h10m50.466366025s) [noise=true , address=45.91.116.13:4155 , piece_id=KMDEG3QMJNHRF7TETZAFVN3TOZWTQNW7IRTCVI3WLNE7NJAAAPCA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.618513849 (103h10m50.467120922s) [noise=true , address=45.91.116.8:4048 , piece_id=AROG4HVUX7R6MHG3DT3N2FIQJBBEVVEVKQHSAGZCWMISWTFSQMAQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.624184177 (103h10m50.47279126s) [noise=true , address=45.91.116.7:4010 , piece_id=WVMOM2PPTVXY7NYI5UH25PFW6RODIUJWN73YB6FW6UOAMNY4C4SA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.625290831 (103h10m50.473897916s) [noise=true , address=45.91.116.7:4011 , piece_id=GKVDRTZ2TY2ESEKU2FKIJND4LXDMITNWQORE4Z47OQ454UZMM7BA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.628897649 (103h10m50.477504746s) [noise=true , address=45.91.116.5:4298 , piece_id=AIURH3V65NZD3QI3A5YGBJ222ANC4LTHEYFWTOFIC6SPVH3O4CKA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.629542622 (103h10m50.478149697s) [noise=true , address=45.91.116.12:4225 , piece_id=MFM7FUANQ2WU4GK46NPKKKFDUXSVVQ3MGK7GM3QBR7GMOD563PTQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.630003986 (103h10m50.478611073s) [noise=true , address=77.90.31.17:4772 , piece_id=63V72ATK3KI3ADRC4N5HTRIXSEB3DRM7N73MVZNDPZJ7EVO3RZJA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.630441959 (103h10m50.479049047s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQJDPJRZQEJHTED4UJBNSJ2CGAL6FPTLLIIRFTOVBUK65VSPLGA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.630941221 (103h10m50.479548305s) [noise=true , address=77.90.31.12:4592 , piece_id=NAWAKPN4LUPCFZIXDIEPN2GKO3CSGX35LZNHI4QDCGYBYAMWBGOA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.631378970 (103h10m50.47998605s) [noise=true , address=77.90.31.13:4626 , piece_id=D63E7JGO5RRTEJGXVUMNEC3UYXHS45HBOB7S67VPCWLHJXNH7OSQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.631840658 (103h10m50.480447733s) [noise=true , address=77.90.31.17:4771 , piece_id=VCHLVD4PCT6SGJNCFA4LSEDIXSRMX27JH7VLUXDBM2PMIGZ54CNA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.631850523 (103h10m50.480457598s) [seg_pos={0 0}, upload=82463]: Segment reached optimal threshold of 13 pieces.
Jul 23 06:01:08.631982956 (103h10m50.480590031s) [noise=true , address=104.36.82.13:4186 , piece_id=O5BVVJBIWINWK2GXWWJDETED6T3Z462AAUQWCG6LGC7PVVFSHZBA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF
Jul 23 06:01:08.632087803 (103h10m50.480694892s) [noise=true , address=45.91.116.4:4937 , piece_id=DZILXASXCQNDQGTAIYQIY4LCXHENKF6LLWHME5ILDWL3IZOSWMCQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF
Jul 23 06:01:08.633101069 (103h10m50.481708157s) [noise=true , address=45.91.116.11:4119 , piece_id=LOMOQU77BVHAPGZ23ZY6CG63FZFQX2IG5QRY2P46WKSGB5AF5O3Q, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.633132335 (103h10m50.481739428s) [seg_pos={0 0}, upload=82463]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 23 06:01:08.633221956 (103h10m50.481829049s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 23 06:01:08.633272252 (103h10m50.481879341s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 23 06:01:08.710605841 (103h10m50.559212933s) [upload=82463]: Done uploading segments...
Jul 23 06:01:08.710632085 (103h10m50.559239157s) [upload=82463]: Upload finished. err: <nil>
Duration: 503.579309ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............................................X.............] duration:
502.342095ms
FAIL (2025-07-23 06:12:06 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
23 06:00:19.742341858 (103h10m1.590948949s) [noise=true , address=104.36.82.4:4804 , piece_id=SMCQKI5GIWKEHYLNSVUGHWQNBHP3S4C5RBB4E3K5CFMXE54Q4WAQ, seg_pos={0 0}, upload=82455]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g): protocol: expected piece hash; context canceled; EOF
Jul 23 06:00:19.742959346 (103h10m1.59156642s) [noise=true , address=77.90.31.16:4734 , piece_id=NFCPPB6P25YMO5UZRM4CQKMNRV5Q2FXMSR6M4VSTEERBMVNLYJJA, seg_pos={0 0}, upload=82455]: Done uploading piece. err: <nil>
Jul 23 06:00:19.743002232 (103h10m1.591609306s) [seg_pos={0 0}, upload=82455]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 23 06:00:19.743286680 (103h10m1.591893764s) [upload=82455]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 23 06:00:19.743293769 (103h10m1.591900841s) [upload=82455]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 23 06:00:19.815641707 (103h10m1.664248798s) [upload=82455]: Done uploading segments...
Jul 23 06:00:19.815666887 (103h10m1.664273954s) [upload=82455]: Upload finished. err: <nil>
Jul 23 06:01:08.401614022 (103h10m50.250221093s) [upload=82463]: Starting upload
Jul 23 06:01:08.401709823 (103h10m50.250316908s) [upload=82463]: Done starting upload
Jul 23 06:01:08.402138436 (103h10m50.250745505s) [upload=82463]: Uploading segments...
Jul 23 06:01:08.402189044 (103h10m50.250796137s) [upload=82463]: Got next segment. Inline: false
Jul 23 06:01:08.402231416 (103h10m50.250838487s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 23 06:01:08.402237731 (103h10m50.250844804s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 23 06:01:08.450301042 (103h10m50.298908146s) [seg_pos={0 0}, upload=82463]: Begin upload segment...
Jul 23 06:01:08.450364720 (103h10m50.298971798s) [seg_pos={0 0}, upload=82463]: Done begin upload segment.
Jul 23 06:01:08.450425015 (103h10m50.2990321s) [noise=true , address=45.91.116.5:4299 , piece_id=K3URA4ZO4R2CAGZLTO6YJTZQOZ6YIOB7XOJ3AQAVGDFIMVU63BIA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.450431954 (103h10m50.299039031s) [noise=true , address=45.91.116.7:4010 , piece_id=WVMOM2PPTVXY7NYI5UH25PFW6RODIUJWN73YB6FW6UOAMNY4C4SA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.450900413 (103h10m50.299507502s) [noise=true , address=77.90.31.17:4772 , piece_id=63V72ATK3KI3ADRC4N5HTRIXSEB3DRM7N73MVZNDPZJ7EVO3RZJA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.450963993 (103h10m50.299571083s) [noise=true , address=45.91.116.8:4048 , piece_id=AROG4HVUX7R6MHG3DT3N2FIQJBBEVVEVKQHSAGZCWMISWTFSQMAQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.451375091 (103h10m50.299982178s) [noise=true , address=45.91.116.13:4155 , piece_id=KMDEG3QMJNHRF7TETZAFVN3TOZWTQNW7IRTCVI3WLNE7NJAAAPCA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.451400410 (103h10m50.300007509s) [noise=true , address=77.90.31.12:4592 , piece_id=NAWAKPN4LUPCFZIXDIEPN2GKO3CSGX35LZNHI4QDCGYBYAMWBGOA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.451837258 (103h10m50.300444352s) [noise=true , address=77.90.31.17:4771 , piece_id=VCHLVD4PCT6SGJNCFA4LSEDIXSRMX27JH7VLUXDBM2PMIGZ54CNA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.452344092 (103h10m50.300951186s) [noise=true , address=45.91.116.4:4937 , piece_id=DZILXASXCQNDQGTAIYQIY4LCXHENKF6LLWHME5ILDWL3IZOSWMCQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.452773889 (103h10m50.301380982s) [noise=true , address=45.91.116.11:4119 , piece_id=LOMOQU77BVHAPGZ23ZY6CG63FZFQX2IG5QRY2P46WKSGB5AF5O3Q, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.453168858 (103h10m50.301775945s) [noise=true , address=45.91.116.5:4298 , piece_id=AIURH3V65NZD3QI3A5YGBJ222ANC4LTHEYFWTOFIC6SPVH3O4CKA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.453559736 (103h10m50.302166813s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQJDPJRZQEJHTED4UJBNSJ2CGAL6FPTLLIIRFTOVBUK65VSPLGA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.460854171 (103h10m50.309461265s) [noise=true , address=77.90.31.14:4661 , piece_id=MA3CRNAREYTT6ZDO7Q7QRSAOVQQ3EM3CHZTWYQYE6QOWNMYQBBQQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.462111720 (103h10m50.310718803s) [noise=true , address=45.91.116.12:4225 , piece_id=MFM7FUANQ2WU4GK46NPKKKFDUXSVVQ3MGK7GM3QBR7GMOD563PTQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.462819102 (103h10m50.311426189s) [noise=true , address=104.36.82.13:4186 , piece_id=O5BVVJBIWINWK2GXWWJDETED6T3Z462AAUQWCG6LGC7PVVFSHZBA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.463600977 (103h10m50.312208063s) [noise=true , address=45.91.116.7:4011 , piece_id=GKVDRTZ2TY2ESEKU2FKIJND4LXDMITNWQORE4Z47OQ454UZMM7BA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.464356397 (103h10m50.312963485s) [noise=true , address=77.90.31.13:4626 , piece_id=D63E7JGO5RRTEJGXVUMNEC3UYXHS45HBOB7S67VPCWLHJXNH7OSQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.587088382 (103h10m50.435695454s) [upload=82463]: Next returned nil segment
Jul 23 06:01:08.587107670 (103h10m50.435714757s) [upload=82463]: Waiting for error group managing segments...
Jul 23 06:01:08.609523184 (103h10m50.458130271s) [noise=true , address=77.90.31.14:4661 , piece_id=MA3CRNAREYTT6ZDO7Q7QRSAOVQQ3EM3CHZTWYQYE6QOWNMYQBBQQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.612539671 (103h10m50.46114674s) [noise=true , address=45.91.116.5:4299 , piece_id=K3URA4ZO4R2CAGZLTO6YJTZQOZ6YIOB7XOJ3AQAVGDFIMVU63BIA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.617758941 (103h10m50.466366025s) [noise=true , address=45.91.116.13:4155 , piece_id=KMDEG3QMJNHRF7TETZAFVN3TOZWTQNW7IRTCVI3WLNE7NJAAAPCA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.618513849 (103h10m50.467120922s) [noise=true , address=45.91.116.8:4048 , piece_id=AROG4HVUX7R6MHG3DT3N2FIQJBBEVVEVKQHSAGZCWMISWTFSQMAQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.624184177 (103h10m50.47279126s) [noise=true , address=45.91.116.7:4010 , piece_id=WVMOM2PPTVXY7NYI5UH25PFW6RODIUJWN73YB6FW6UOAMNY4C4SA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.625290831 (103h10m50.473897916s) [noise=true , address=45.91.116.7:4011 , piece_id=GKVDRTZ2TY2ESEKU2FKIJND4LXDMITNWQORE4Z47OQ454UZMM7BA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.628897649 (103h10m50.477504746s) [noise=true , address=45.91.116.5:4298 , piece_id=AIURH3V65NZD3QI3A5YGBJ222ANC4LTHEYFWTOFIC6SPVH3O4CKA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.629542622 (103h10m50.478149697s) [noise=true , address=45.91.116.12:4225 , piece_id=MFM7FUANQ2WU4GK46NPKKKFDUXSVVQ3MGK7GM3QBR7GMOD563PTQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.630003986 (103h10m50.478611073s) [noise=true , address=77.90.31.17:4772 , piece_id=63V72ATK3KI3ADRC4N5HTRIXSEB3DRM7N73MVZNDPZJ7EVO3RZJA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.630441959 (103h10m50.479049047s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQJDPJRZQEJHTED4UJBNSJ2CGAL6FPTLLIIRFTOVBUK65VSPLGA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.630941221 (103h10m50.479548305s) [noise=true , address=77.90.31.12:4592 , piece_id=NAWAKPN4LUPCFZIXDIEPN2GKO3CSGX35LZNHI4QDCGYBYAMWBGOA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.631378970 (103h10m50.47998605s) [noise=true , address=77.90.31.13:4626 , piece_id=D63E7JGO5RRTEJGXVUMNEC3UYXHS45HBOB7S67VPCWLHJXNH7OSQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.631840658 (103h10m50.480447733s) [noise=true , address=77.90.31.17:4771 , piece_id=VCHLVD4PCT6SGJNCFA4LSEDIXSRMX27JH7VLUXDBM2PMIGZ54CNA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.631850523 (103h10m50.480457598s) [seg_pos={0 0}, upload=82463]: Segment reached optimal threshold of 13 pieces.
Jul 23 06:01:08.631982956 (103h10m50.480590031s) [noise=true , address=104.36.82.13:4186 , piece_id=O5BVVJBIWINWK2GXWWJDETED6T3Z462AAUQWCG6LGC7PVVFSHZBA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF
Jul 23 06:01:08.632087803 (103h10m50.480694892s) [noise=true , address=45.91.116.4:4937 , piece_id=DZILXASXCQNDQGTAIYQIY4LCXHENKF6LLWHME5ILDWL3IZOSWMCQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF
Jul 23 06:01:08.633101069 (103h10m50.481708157s) [noise=true , address=45.91.116.11:4119 , piece_id=LOMOQU77BVHAPGZ23ZY6CG63FZFQX2IG5QRY2P46WKSGB5AF5O3Q, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.633132335 (103h10m50.481739428s) [seg_pos={0 0}, upload=82463]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 23 06:01:08.633221956 (103h10m50.481829049s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 23 06:01:08.633272252 (103h10m50.481879341s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 23 06:01:08.710605841 (103h10m50.559212933s) [upload=82463]: Done uploading segments...
Jul 23 06:01:08.710632085 (103h10m50.559239157s) [upload=82463]: Upload finished. err: <nil>
Duration: 502.669952ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [..............................................X.............] duration:
949.195728ms
FAIL (2025-07-23 06:12:05 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
s) [upload=82448]: Got next segment. Inline: false
Jul 23 05:59:58.732247463 (103h9m40.580854536s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 23 05:59:58.732253833 (103h9m40.580860918s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 23 05:59:58.787057645 (103h9m40.635664731s) [seg_pos={0 0}, upload=82448]: Begin upload segment...
Jul 23 05:59:58.787102519 (103h9m40.635709584s) [seg_pos={0 0}, upload=82448]: Done begin upload segment.
Jul 23 05:59:58.787136403 (103h9m40.635743482s) [noise=true , address=77.90.31.12:4622 , piece_id=EZJAG75JRZTYILS7545KYAI37YOUTHHIOOSH26KIWMUQ4BCG556A, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787169354 (103h9m40.635776438s) [noise=true , address=79.127.205.235:4680 , piece_id=T3V7PMZXJOHWT77MQ52PUMFNAIAX4DLMTIIVL6XNVTNA565O5WCQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787544550 (103h9m40.636151681s) [noise=true , address=104.36.82.10:4105 , piece_id=GY72JPHTZOGEVFDG3FM5W5ZSYOPSY3BZXEME2Y37VVR2LR37SIUA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787598825 (103h9m40.636205915s) [noise=true , address=79.127.205.246:4280 , piece_id=NPBN75MFQYLCG4XKRJPLHJU5JCYBNQYIFQ7MKKB3JKXLT4TKF7AA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787946084 (103h9m40.636553167s) [noise=true , address=77.90.31.17:4799 , piece_id=RUTSDNZEXKJ5Y7WQBIVTMFUIJRYLL6DDNNJEXZAL5S7CIATWYHBQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788041731 (103h9m40.636648823s) [noise=true , address=77.90.31.11:4569 , piece_id=MTNLU22JHOYH6VSRMWLKQGSCDI5GIADPIXBVVSC2RIJLTHPMXTOA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788383218 (103h9m40.636990305s) [noise=true , address=104.36.82.42:4818 , piece_id=A6U7TDLTAZHE5IILHHCZKIHCQINHCJSWTBYNXJV7GLCWTIBNB2VA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788633554 (103h9m40.637240644s) [noise=true , address=79.127.219.51:4202 , piece_id=LJZ5A5HAHG4CGXT5J2WNSMXD7HCYSCBUEYGAGOES3NWIVV34BXEQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788735479 (103h9m40.637342568s) [noise=true , address=79.127.219.47:4040 , piece_id=BKWXBAVFXQ2RKG27WGPEAWG5UTHBVODW2BB4EGKHV6SN4EFI7KIQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789072444 (103h9m40.637679533s) [noise=true , address=77.90.31.15:4706 , piece_id=C3JOUPYB77PA2EHPSNPMDETERE724PBALBCSZ6MJCTZ6YRPUC2UA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789122202 (103h9m40.637729291s) [noise=true , address=104.36.82.9:4035 , piece_id=74BSWBLVWYQXPUI2PGV5NIJDSE6MSFKWK65TCVV6CNZKVQBQVNEA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789393961 (103h9m40.638001047s) [noise=true , address=45.91.116.10:4103 , piece_id=UGRDLKKFJRNCCDNDQTAY2LLOIS7CGI6KLNDZ7KJDOUNYWFNUAYQQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789591397 (103h9m40.638198479s) [noise=true , address=104.36.82.5:4870 , piece_id=AYTLB2OL7OAI5EO2ZE6LXIQLCGSGDXKDKT6V5HXIPAREK656MDPQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789716087 (103h9m40.638323176s) [noise=true , address=79.127.205.233:4645 , piece_id=6YAS5E7QYEZAJXB6HUY3R4PLHC4Z7EYTMSFWNFWD5AP7HKXOKTUQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789937996 (103h9m40.638545078s) [noise=true , address=104.36.82.53:4121 , piece_id=GJLXLW2A5RZUF4MBUP5AZAWFKTWGBDADIIWVIUBXPQPPGPKYZ4YA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.790288745 (103h9m40.638895835s) [noise=true , address=77.90.31.15:4708 , piece_id=67LX4NZJUKLYHRTSW3RKHHXZRKMGWJ2KV32FDA2KE3DY364CJ7OA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.790730467 (103h9m40.639337554s) [noise=true , address=45.91.116.14:4206 , piece_id=OVX63SL5OGJTT7HWDRIUV4BDVEKN2MD4ORUUK5M4JIEWX4J6VHFA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.791174576 (103h9m40.639781656s) [noise=true , address=45.91.116.13:4176 , piece_id=GOWXIBOORGZWFIBLMG46HQ4J632B3E6I3MVXNLTWPP2TY3IF2JUQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.898860128 (103h9m40.747467214s) [upload=82448]: Next returned nil segment
Jul 23 05:59:58.898881813 (103h9m40.747488906s) [upload=82448]: Waiting for error group managing segments...
Jul 23 05:59:58.905078785 (103h9m40.753685854s) [noise=true , address=45.91.116.14:4206 , piece_id=OVX63SL5OGJTT7HWDRIUV4BDVEKN2MD4ORUUK5M4JIEWX4J6VHFA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.906441982 (103h9m40.755049063s) [noise=true , address=77.90.31.15:4708 , piece_id=67LX4NZJUKLYHRTSW3RKHHXZRKMGWJ2KV32FDA2KE3DY364CJ7OA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.910115252 (103h9m40.758722337s) [noise=true , address=77.90.31.15:4706 , piece_id=C3JOUPYB77PA2EHPSNPMDETERE724PBALBCSZ6MJCTZ6YRPUC2UA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.910652695 (103h9m40.759259762s) [noise=true , address=104.36.82.9:4035 , piece_id=74BSWBLVWYQXPUI2PGV5NIJDSE6MSFKWK65TCVV6CNZKVQBQVNEA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.910672036 (103h9m40.759279117s) [noise=true , address=77.90.31.11:4569 , piece_id=MTNLU22JHOYH6VSRMWLKQGSCDI5GIADPIXBVVSC2RIJLTHPMXTOA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.911483080 (103h9m40.760090146s) [noise=true , address=45.91.116.13:4176 , piece_id=GOWXIBOORGZWFIBLMG46HQ4J632B3E6I3MVXNLTWPP2TY3IF2JUQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.911951970 (103h9m40.760559052s) [noise=true , address=45.91.116.10:4103 , piece_id=UGRDLKKFJRNCCDNDQTAY2LLOIS7CGI6KLNDZ7KJDOUNYWFNUAYQQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.912044937 (103h9m40.760652006s) [noise=true , address=77.90.31.12:4622 , piece_id=EZJAG75JRZTYILS7545KYAI37YOUTHHIOOSH26KIWMUQ4BCG556A, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.912649865 (103h9m40.761256931s) [noise=true , address=77.90.31.17:4799 , piece_id=RUTSDNZEXKJ5Y7WQBIVTMFUIJRYLL6DDNNJEXZAL5S7CIATWYHBQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.913487977 (103h9m40.76209505s) [noise=true , address=79.127.219.47:4040 , piece_id=BKWXBAVFXQ2RKG27WGPEAWG5UTHBVODW2BB4EGKHV6SN4EFI7KIQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.914486860 (103h9m40.763093927s) [noise=true , address=104.36.82.53:4121 , piece_id=GJLXLW2A5RZUF4MBUP5AZAWFKTWGBDADIIWVIUBXPQPPGPKYZ4YA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.918080770 (103h9m40.766687852s) [noise=true , address=104.36.82.5:4870 , piece_id=AYTLB2OL7OAI5EO2ZE6LXIQLCGSGDXKDKT6V5HXIPAREK656MDPQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.924012392 (103h9m40.772619461s) [noise=true , address=79.127.219.51:4202 , piece_id=LJZ5A5HAHG4CGXT5J2WNSMXD7HCYSCBUEYGAGOES3NWIVV34BXEQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.932892483 (103h9m40.78149955s) [noise=true , address=104.36.82.42:4818 , piece_id=A6U7TDLTAZHE5IILHHCZKIHCQINHCJSWTBYNXJV7GLCWTIBNB2VA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.932913099 (103h9m40.781520167s) [seg_pos={0 0}, upload=82448]: Segment reached optimal threshold of 14 pieces.
Jul 23 05:59:58.933104957 (103h9m40.781712037s) [noise=true , address=104.36.82.10:4105 , piece_id=GY72JPHTZOGEVFDG3FM5W5ZSYOPSY3BZXEME2Y37VVR2LR37SIUA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GhDrKMcDEg2RxamqKtiHphpgNdvtZYdHXrjzGsNJFjBFYeEen): protocol: expected piece hash; context canceled; EOF
Jul 23 05:59:58.933505265 (103h9m40.782112345s) [noise=true , address=79.127.205.233:4645 , piece_id=6YAS5E7QYEZAJXB6HUY3R4PLHC4Z7EYTMSFWNFWD5AP7HKXOKTUQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12NR52pweTqZNNMTyc1EHSbifDKB4awrnnb4megtZHwANSrR18T): protocol: read tcp 159.223.143.39:46566->79.127.205.233:4645: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 23 05:59:58.933568113 (103h9m40.782175194s) [noise=true , address=79.127.205.235:4680 , piece_id=T3V7PMZXJOHWT77MQ52PUMFNAIAX4DLMTIIVL6XNVTNA565O5WCQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GGvQQLoBcTniZST8Hz9h8eCLWbU4DrXn66DGy33iGLXaFgJLj): protocol: read tcp 159.223.143.39:45394->79.127.205.235:4680: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 23 05:59:58.933618915 (103h9m40.782225986s) [noise=true , address=79.127.205.246:4280 , piece_id=NPBN75MFQYLCG4XKRJPLHJU5JCYBNQYIFQ7MKKB3JKXLT4TKF7AA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rZDBvhrfNrYQB6h9MGhx1tNtQ7AH9ehUF4SfNMCJRpv4SKrDX): protocol: read tcp 159.223.143.39:45458->79.127.205.246:4280: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 23 05:59:58.933631191 (103h9m40.78223826s) [seg_pos={0 0}, upload=82448]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jul 23 05:59:58.933756265 (103h9m40.782363345s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 23 05:59:58.933762566 (103h9m40.782369645s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 23 05:59:59.025408726 (103h9m40.874015808s) [upload=82448]: Done uploading segments...
Jul 23 05:59:59.025427547 (103h9m40.874034627s) [upload=82448]: Upload finished. err: <nil>
Duration: 949.224259ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............] duration:
576.317242ms
FAIL (2025-07-23 06:11:10 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
23 06:00:19.742341858 (103h10m1.590948949s) [noise=true , address=104.36.82.4:4804 , piece_id=SMCQKI5GIWKEHYLNSVUGHWQNBHP3S4C5RBB4E3K5CFMXE54Q4WAQ, seg_pos={0 0}, upload=82455]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g): protocol: expected piece hash; context canceled; EOF
Jul 23 06:00:19.742959346 (103h10m1.59156642s) [noise=true , address=77.90.31.16:4734 , piece_id=NFCPPB6P25YMO5UZRM4CQKMNRV5Q2FXMSR6M4VSTEERBMVNLYJJA, seg_pos={0 0}, upload=82455]: Done uploading piece. err: <nil>
Jul 23 06:00:19.743002232 (103h10m1.591609306s) [seg_pos={0 0}, upload=82455]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 23 06:00:19.743286680 (103h10m1.591893764s) [upload=82455]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 23 06:00:19.743293769 (103h10m1.591900841s) [upload=82455]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 23 06:00:19.815641707 (103h10m1.664248798s) [upload=82455]: Done uploading segments...
Jul 23 06:00:19.815666887 (103h10m1.664273954s) [upload=82455]: Upload finished. err: <nil>
Jul 23 06:01:08.401614022 (103h10m50.250221093s) [upload=82463]: Starting upload
Jul 23 06:01:08.401709823 (103h10m50.250316908s) [upload=82463]: Done starting upload
Jul 23 06:01:08.402138436 (103h10m50.250745505s) [upload=82463]: Uploading segments...
Jul 23 06:01:08.402189044 (103h10m50.250796137s) [upload=82463]: Got next segment. Inline: false
Jul 23 06:01:08.402231416 (103h10m50.250838487s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 23 06:01:08.402237731 (103h10m50.250844804s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 23 06:01:08.450301042 (103h10m50.298908146s) [seg_pos={0 0}, upload=82463]: Begin upload segment...
Jul 23 06:01:08.450364720 (103h10m50.298971798s) [seg_pos={0 0}, upload=82463]: Done begin upload segment.
Jul 23 06:01:08.450425015 (103h10m50.2990321s) [noise=true , address=45.91.116.5:4299 , piece_id=K3URA4ZO4R2CAGZLTO6YJTZQOZ6YIOB7XOJ3AQAVGDFIMVU63BIA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.450431954 (103h10m50.299039031s) [noise=true , address=45.91.116.7:4010 , piece_id=WVMOM2PPTVXY7NYI5UH25PFW6RODIUJWN73YB6FW6UOAMNY4C4SA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.450900413 (103h10m50.299507502s) [noise=true , address=77.90.31.17:4772 , piece_id=63V72ATK3KI3ADRC4N5HTRIXSEB3DRM7N73MVZNDPZJ7EVO3RZJA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.450963993 (103h10m50.299571083s) [noise=true , address=45.91.116.8:4048 , piece_id=AROG4HVUX7R6MHG3DT3N2FIQJBBEVVEVKQHSAGZCWMISWTFSQMAQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.451375091 (103h10m50.299982178s) [noise=true , address=45.91.116.13:4155 , piece_id=KMDEG3QMJNHRF7TETZAFVN3TOZWTQNW7IRTCVI3WLNE7NJAAAPCA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.451400410 (103h10m50.300007509s) [noise=true , address=77.90.31.12:4592 , piece_id=NAWAKPN4LUPCFZIXDIEPN2GKO3CSGX35LZNHI4QDCGYBYAMWBGOA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.451837258 (103h10m50.300444352s) [noise=true , address=77.90.31.17:4771 , piece_id=VCHLVD4PCT6SGJNCFA4LSEDIXSRMX27JH7VLUXDBM2PMIGZ54CNA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.452344092 (103h10m50.300951186s) [noise=true , address=45.91.116.4:4937 , piece_id=DZILXASXCQNDQGTAIYQIY4LCXHENKF6LLWHME5ILDWL3IZOSWMCQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.452773889 (103h10m50.301380982s) [noise=true , address=45.91.116.11:4119 , piece_id=LOMOQU77BVHAPGZ23ZY6CG63FZFQX2IG5QRY2P46WKSGB5AF5O3Q, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.453168858 (103h10m50.301775945s) [noise=true , address=45.91.116.5:4298 , piece_id=AIURH3V65NZD3QI3A5YGBJ222ANC4LTHEYFWTOFIC6SPVH3O4CKA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.453559736 (103h10m50.302166813s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQJDPJRZQEJHTED4UJBNSJ2CGAL6FPTLLIIRFTOVBUK65VSPLGA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.460854171 (103h10m50.309461265s) [noise=true , address=77.90.31.14:4661 , piece_id=MA3CRNAREYTT6ZDO7Q7QRSAOVQQ3EM3CHZTWYQYE6QOWNMYQBBQQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.462111720 (103h10m50.310718803s) [noise=true , address=45.91.116.12:4225 , piece_id=MFM7FUANQ2WU4GK46NPKKKFDUXSVVQ3MGK7GM3QBR7GMOD563PTQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.462819102 (103h10m50.311426189s) [noise=true , address=104.36.82.13:4186 , piece_id=O5BVVJBIWINWK2GXWWJDETED6T3Z462AAUQWCG6LGC7PVVFSHZBA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.463600977 (103h10m50.312208063s) [noise=true , address=45.91.116.7:4011 , piece_id=GKVDRTZ2TY2ESEKU2FKIJND4LXDMITNWQORE4Z47OQ454UZMM7BA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.464356397 (103h10m50.312963485s) [noise=true , address=77.90.31.13:4626 , piece_id=D63E7JGO5RRTEJGXVUMNEC3UYXHS45HBOB7S67VPCWLHJXNH7OSQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.587088382 (103h10m50.435695454s) [upload=82463]: Next returned nil segment
Jul 23 06:01:08.587107670 (103h10m50.435714757s) [upload=82463]: Waiting for error group managing segments...
Jul 23 06:01:08.609523184 (103h10m50.458130271s) [noise=true , address=77.90.31.14:4661 , piece_id=MA3CRNAREYTT6ZDO7Q7QRSAOVQQ3EM3CHZTWYQYE6QOWNMYQBBQQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.612539671 (103h10m50.46114674s) [noise=true , address=45.91.116.5:4299 , piece_id=K3URA4ZO4R2CAGZLTO6YJTZQOZ6YIOB7XOJ3AQAVGDFIMVU63BIA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.617758941 (103h10m50.466366025s) [noise=true , address=45.91.116.13:4155 , piece_id=KMDEG3QMJNHRF7TETZAFVN3TOZWTQNW7IRTCVI3WLNE7NJAAAPCA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.618513849 (103h10m50.467120922s) [noise=true , address=45.91.116.8:4048 , piece_id=AROG4HVUX7R6MHG3DT3N2FIQJBBEVVEVKQHSAGZCWMISWTFSQMAQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.624184177 (103h10m50.47279126s) [noise=true , address=45.91.116.7:4010 , piece_id=WVMOM2PPTVXY7NYI5UH25PFW6RODIUJWN73YB6FW6UOAMNY4C4SA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.625290831 (103h10m50.473897916s) [noise=true , address=45.91.116.7:4011 , piece_id=GKVDRTZ2TY2ESEKU2FKIJND4LXDMITNWQORE4Z47OQ454UZMM7BA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.628897649 (103h10m50.477504746s) [noise=true , address=45.91.116.5:4298 , piece_id=AIURH3V65NZD3QI3A5YGBJ222ANC4LTHEYFWTOFIC6SPVH3O4CKA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.629542622 (103h10m50.478149697s) [noise=true , address=45.91.116.12:4225 , piece_id=MFM7FUANQ2WU4GK46NPKKKFDUXSVVQ3MGK7GM3QBR7GMOD563PTQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.630003986 (103h10m50.478611073s) [noise=true , address=77.90.31.17:4772 , piece_id=63V72ATK3KI3ADRC4N5HTRIXSEB3DRM7N73MVZNDPZJ7EVO3RZJA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.630441959 (103h10m50.479049047s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQJDPJRZQEJHTED4UJBNSJ2CGAL6FPTLLIIRFTOVBUK65VSPLGA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.630941221 (103h10m50.479548305s) [noise=true , address=77.90.31.12:4592 , piece_id=NAWAKPN4LUPCFZIXDIEPN2GKO3CSGX35LZNHI4QDCGYBYAMWBGOA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.631378970 (103h10m50.47998605s) [noise=true , address=77.90.31.13:4626 , piece_id=D63E7JGO5RRTEJGXVUMNEC3UYXHS45HBOB7S67VPCWLHJXNH7OSQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.631840658 (103h10m50.480447733s) [noise=true , address=77.90.31.17:4771 , piece_id=VCHLVD4PCT6SGJNCFA4LSEDIXSRMX27JH7VLUXDBM2PMIGZ54CNA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.631850523 (103h10m50.480457598s) [seg_pos={0 0}, upload=82463]: Segment reached optimal threshold of 13 pieces.
Jul 23 06:01:08.631982956 (103h10m50.480590031s) [noise=true , address=104.36.82.13:4186 , piece_id=O5BVVJBIWINWK2GXWWJDETED6T3Z462AAUQWCG6LGC7PVVFSHZBA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF
Jul 23 06:01:08.632087803 (103h10m50.480694892s) [noise=true , address=45.91.116.4:4937 , piece_id=DZILXASXCQNDQGTAIYQIY4LCXHENKF6LLWHME5ILDWL3IZOSWMCQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF
Jul 23 06:01:08.633101069 (103h10m50.481708157s) [noise=true , address=45.91.116.11:4119 , piece_id=LOMOQU77BVHAPGZ23ZY6CG63FZFQX2IG5QRY2P46WKSGB5AF5O3Q, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.633132335 (103h10m50.481739428s) [seg_pos={0 0}, upload=82463]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 23 06:01:08.633221956 (103h10m50.481829049s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 23 06:01:08.633272252 (103h10m50.481879341s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 23 06:01:08.710605841 (103h10m50.559212933s) [upload=82463]: Done uploading segments...
Jul 23 06:01:08.710632085 (103h10m50.559239157s) [upload=82463]: Upload finished. err: <nil>
Duration: 576.348491ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............] duration:
554.54556ms
FAIL (2025-07-23 06:11:09 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
s) [upload=82448]: Got next segment. Inline: false
Jul 23 05:59:58.732247463 (103h9m40.580854536s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 23 05:59:58.732253833 (103h9m40.580860918s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 23 05:59:58.787057645 (103h9m40.635664731s) [seg_pos={0 0}, upload=82448]: Begin upload segment...
Jul 23 05:59:58.787102519 (103h9m40.635709584s) [seg_pos={0 0}, upload=82448]: Done begin upload segment.
Jul 23 05:59:58.787136403 (103h9m40.635743482s) [noise=true , address=77.90.31.12:4622 , piece_id=EZJAG75JRZTYILS7545KYAI37YOUTHHIOOSH26KIWMUQ4BCG556A, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787169354 (103h9m40.635776438s) [noise=true , address=79.127.205.235:4680 , piece_id=T3V7PMZXJOHWT77MQ52PUMFNAIAX4DLMTIIVL6XNVTNA565O5WCQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787544550 (103h9m40.636151681s) [noise=true , address=104.36.82.10:4105 , piece_id=GY72JPHTZOGEVFDG3FM5W5ZSYOPSY3BZXEME2Y37VVR2LR37SIUA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787598825 (103h9m40.636205915s) [noise=true , address=79.127.205.246:4280 , piece_id=NPBN75MFQYLCG4XKRJPLHJU5JCYBNQYIFQ7MKKB3JKXLT4TKF7AA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787946084 (103h9m40.636553167s) [noise=true , address=77.90.31.17:4799 , piece_id=RUTSDNZEXKJ5Y7WQBIVTMFUIJRYLL6DDNNJEXZAL5S7CIATWYHBQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788041731 (103h9m40.636648823s) [noise=true , address=77.90.31.11:4569 , piece_id=MTNLU22JHOYH6VSRMWLKQGSCDI5GIADPIXBVVSC2RIJLTHPMXTOA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788383218 (103h9m40.636990305s) [noise=true , address=104.36.82.42:4818 , piece_id=A6U7TDLTAZHE5IILHHCZKIHCQINHCJSWTBYNXJV7GLCWTIBNB2VA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788633554 (103h9m40.637240644s) [noise=true , address=79.127.219.51:4202 , piece_id=LJZ5A5HAHG4CGXT5J2WNSMXD7HCYSCBUEYGAGOES3NWIVV34BXEQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788735479 (103h9m40.637342568s) [noise=true , address=79.127.219.47:4040 , piece_id=BKWXBAVFXQ2RKG27WGPEAWG5UTHBVODW2BB4EGKHV6SN4EFI7KIQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789072444 (103h9m40.637679533s) [noise=true , address=77.90.31.15:4706 , piece_id=C3JOUPYB77PA2EHPSNPMDETERE724PBALBCSZ6MJCTZ6YRPUC2UA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789122202 (103h9m40.637729291s) [noise=true , address=104.36.82.9:4035 , piece_id=74BSWBLVWYQXPUI2PGV5NIJDSE6MSFKWK65TCVV6CNZKVQBQVNEA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789393961 (103h9m40.638001047s) [noise=true , address=45.91.116.10:4103 , piece_id=UGRDLKKFJRNCCDNDQTAY2LLOIS7CGI6KLNDZ7KJDOUNYWFNUAYQQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789591397 (103h9m40.638198479s) [noise=true , address=104.36.82.5:4870 , piece_id=AYTLB2OL7OAI5EO2ZE6LXIQLCGSGDXKDKT6V5HXIPAREK656MDPQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789716087 (103h9m40.638323176s) [noise=true , address=79.127.205.233:4645 , piece_id=6YAS5E7QYEZAJXB6HUY3R4PLHC4Z7EYTMSFWNFWD5AP7HKXOKTUQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789937996 (103h9m40.638545078s) [noise=true , address=104.36.82.53:4121 , piece_id=GJLXLW2A5RZUF4MBUP5AZAWFKTWGBDADIIWVIUBXPQPPGPKYZ4YA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.790288745 (103h9m40.638895835s) [noise=true , address=77.90.31.15:4708 , piece_id=67LX4NZJUKLYHRTSW3RKHHXZRKMGWJ2KV32FDA2KE3DY364CJ7OA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.790730467 (103h9m40.639337554s) [noise=true , address=45.91.116.14:4206 , piece_id=OVX63SL5OGJTT7HWDRIUV4BDVEKN2MD4ORUUK5M4JIEWX4J6VHFA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.791174576 (103h9m40.639781656s) [noise=true , address=45.91.116.13:4176 , piece_id=GOWXIBOORGZWFIBLMG46HQ4J632B3E6I3MVXNLTWPP2TY3IF2JUQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.898860128 (103h9m40.747467214s) [upload=82448]: Next returned nil segment
Jul 23 05:59:58.898881813 (103h9m40.747488906s) [upload=82448]: Waiting for error group managing segments...
Jul 23 05:59:58.905078785 (103h9m40.753685854s) [noise=true , address=45.91.116.14:4206 , piece_id=OVX63SL5OGJTT7HWDRIUV4BDVEKN2MD4ORUUK5M4JIEWX4J6VHFA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.906441982 (103h9m40.755049063s) [noise=true , address=77.90.31.15:4708 , piece_id=67LX4NZJUKLYHRTSW3RKHHXZRKMGWJ2KV32FDA2KE3DY364CJ7OA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.910115252 (103h9m40.758722337s) [noise=true , address=77.90.31.15:4706 , piece_id=C3JOUPYB77PA2EHPSNPMDETERE724PBALBCSZ6MJCTZ6YRPUC2UA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.910652695 (103h9m40.759259762s) [noise=true , address=104.36.82.9:4035 , piece_id=74BSWBLVWYQXPUI2PGV5NIJDSE6MSFKWK65TCVV6CNZKVQBQVNEA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.910672036 (103h9m40.759279117s) [noise=true , address=77.90.31.11:4569 , piece_id=MTNLU22JHOYH6VSRMWLKQGSCDI5GIADPIXBVVSC2RIJLTHPMXTOA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.911483080 (103h9m40.760090146s) [noise=true , address=45.91.116.13:4176 , piece_id=GOWXIBOORGZWFIBLMG46HQ4J632B3E6I3MVXNLTWPP2TY3IF2JUQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.911951970 (103h9m40.760559052s) [noise=true , address=45.91.116.10:4103 , piece_id=UGRDLKKFJRNCCDNDQTAY2LLOIS7CGI6KLNDZ7KJDOUNYWFNUAYQQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.912044937 (103h9m40.760652006s) [noise=true , address=77.90.31.12:4622 , piece_id=EZJAG75JRZTYILS7545KYAI37YOUTHHIOOSH26KIWMUQ4BCG556A, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.912649865 (103h9m40.761256931s) [noise=true , address=77.90.31.17:4799 , piece_id=RUTSDNZEXKJ5Y7WQBIVTMFUIJRYLL6DDNNJEXZAL5S7CIATWYHBQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.913487977 (103h9m40.76209505s) [noise=true , address=79.127.219.47:4040 , piece_id=BKWXBAVFXQ2RKG27WGPEAWG5UTHBVODW2BB4EGKHV6SN4EFI7KIQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.914486860 (103h9m40.763093927s) [noise=true , address=104.36.82.53:4121 , piece_id=GJLXLW2A5RZUF4MBUP5AZAWFKTWGBDADIIWVIUBXPQPPGPKYZ4YA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.918080770 (103h9m40.766687852s) [noise=true , address=104.36.82.5:4870 , piece_id=AYTLB2OL7OAI5EO2ZE6LXIQLCGSGDXKDKT6V5HXIPAREK656MDPQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.924012392 (103h9m40.772619461s) [noise=true , address=79.127.219.51:4202 , piece_id=LJZ5A5HAHG4CGXT5J2WNSMXD7HCYSCBUEYGAGOES3NWIVV34BXEQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.932892483 (103h9m40.78149955s) [noise=true , address=104.36.82.42:4818 , piece_id=A6U7TDLTAZHE5IILHHCZKIHCQINHCJSWTBYNXJV7GLCWTIBNB2VA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.932913099 (103h9m40.781520167s) [seg_pos={0 0}, upload=82448]: Segment reached optimal threshold of 14 pieces.
Jul 23 05:59:58.933104957 (103h9m40.781712037s) [noise=true , address=104.36.82.10:4105 , piece_id=GY72JPHTZOGEVFDG3FM5W5ZSYOPSY3BZXEME2Y37VVR2LR37SIUA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GhDrKMcDEg2RxamqKtiHphpgNdvtZYdHXrjzGsNJFjBFYeEen): protocol: expected piece hash; context canceled; EOF
Jul 23 05:59:58.933505265 (103h9m40.782112345s) [noise=true , address=79.127.205.233:4645 , piece_id=6YAS5E7QYEZAJXB6HUY3R4PLHC4Z7EYTMSFWNFWD5AP7HKXOKTUQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12NR52pweTqZNNMTyc1EHSbifDKB4awrnnb4megtZHwANSrR18T): protocol: read tcp 159.223.143.39:46566->79.127.205.233:4645: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 23 05:59:58.933568113 (103h9m40.782175194s) [noise=true , address=79.127.205.235:4680 , piece_id=T3V7PMZXJOHWT77MQ52PUMFNAIAX4DLMTIIVL6XNVTNA565O5WCQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GGvQQLoBcTniZST8Hz9h8eCLWbU4DrXn66DGy33iGLXaFgJLj): protocol: read tcp 159.223.143.39:45394->79.127.205.235:4680: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 23 05:59:58.933618915 (103h9m40.782225986s) [noise=true , address=79.127.205.246:4280 , piece_id=NPBN75MFQYLCG4XKRJPLHJU5JCYBNQYIFQ7MKKB3JKXLT4TKF7AA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rZDBvhrfNrYQB6h9MGhx1tNtQ7AH9ehUF4SfNMCJRpv4SKrDX): protocol: read tcp 159.223.143.39:45458->79.127.205.246:4280: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 23 05:59:58.933631191 (103h9m40.78223826s) [seg_pos={0 0}, upload=82448]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jul 23 05:59:58.933756265 (103h9m40.782363345s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 23 05:59:58.933762566 (103h9m40.782369645s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 23 05:59:59.025408726 (103h9m40.874015808s) [upload=82448]: Done uploading segments...
Jul 23 05:59:59.025427547 (103h9m40.874034627s) [upload=82448]: Upload finished. err: <nil>
Duration: 554.577924ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........] duration:
700.422311ms
FAIL (2025-07-23 06:10:40 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
s) [upload=82448]: Got next segment. Inline: false
Jul 23 05:59:58.732247463 (103h9m40.580854536s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 23 05:59:58.732253833 (103h9m40.580860918s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 23 05:59:58.787057645 (103h9m40.635664731s) [seg_pos={0 0}, upload=82448]: Begin upload segment...
Jul 23 05:59:58.787102519 (103h9m40.635709584s) [seg_pos={0 0}, upload=82448]: Done begin upload segment.
Jul 23 05:59:58.787136403 (103h9m40.635743482s) [noise=true , address=77.90.31.12:4622 , piece_id=EZJAG75JRZTYILS7545KYAI37YOUTHHIOOSH26KIWMUQ4BCG556A, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787169354 (103h9m40.635776438s) [noise=true , address=79.127.205.235:4680 , piece_id=T3V7PMZXJOHWT77MQ52PUMFNAIAX4DLMTIIVL6XNVTNA565O5WCQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787544550 (103h9m40.636151681s) [noise=true , address=104.36.82.10:4105 , piece_id=GY72JPHTZOGEVFDG3FM5W5ZSYOPSY3BZXEME2Y37VVR2LR37SIUA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787598825 (103h9m40.636205915s) [noise=true , address=79.127.205.246:4280 , piece_id=NPBN75MFQYLCG4XKRJPLHJU5JCYBNQYIFQ7MKKB3JKXLT4TKF7AA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787946084 (103h9m40.636553167s) [noise=true , address=77.90.31.17:4799 , piece_id=RUTSDNZEXKJ5Y7WQBIVTMFUIJRYLL6DDNNJEXZAL5S7CIATWYHBQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788041731 (103h9m40.636648823s) [noise=true , address=77.90.31.11:4569 , piece_id=MTNLU22JHOYH6VSRMWLKQGSCDI5GIADPIXBVVSC2RIJLTHPMXTOA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788383218 (103h9m40.636990305s) [noise=true , address=104.36.82.42:4818 , piece_id=A6U7TDLTAZHE5IILHHCZKIHCQINHCJSWTBYNXJV7GLCWTIBNB2VA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788633554 (103h9m40.637240644s) [noise=true , address=79.127.219.51:4202 , piece_id=LJZ5A5HAHG4CGXT5J2WNSMXD7HCYSCBUEYGAGOES3NWIVV34BXEQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788735479 (103h9m40.637342568s) [noise=true , address=79.127.219.47:4040 , piece_id=BKWXBAVFXQ2RKG27WGPEAWG5UTHBVODW2BB4EGKHV6SN4EFI7KIQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789072444 (103h9m40.637679533s) [noise=true , address=77.90.31.15:4706 , piece_id=C3JOUPYB77PA2EHPSNPMDETERE724PBALBCSZ6MJCTZ6YRPUC2UA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789122202 (103h9m40.637729291s) [noise=true , address=104.36.82.9:4035 , piece_id=74BSWBLVWYQXPUI2PGV5NIJDSE6MSFKWK65TCVV6CNZKVQBQVNEA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789393961 (103h9m40.638001047s) [noise=true , address=45.91.116.10:4103 , piece_id=UGRDLKKFJRNCCDNDQTAY2LLOIS7CGI6KLNDZ7KJDOUNYWFNUAYQQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789591397 (103h9m40.638198479s) [noise=true , address=104.36.82.5:4870 , piece_id=AYTLB2OL7OAI5EO2ZE6LXIQLCGSGDXKDKT6V5HXIPAREK656MDPQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789716087 (103h9m40.638323176s) [noise=true , address=79.127.205.233:4645 , piece_id=6YAS5E7QYEZAJXB6HUY3R4PLHC4Z7EYTMSFWNFWD5AP7HKXOKTUQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789937996 (103h9m40.638545078s) [noise=true , address=104.36.82.53:4121 , piece_id=GJLXLW2A5RZUF4MBUP5AZAWFKTWGBDADIIWVIUBXPQPPGPKYZ4YA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.790288745 (103h9m40.638895835s) [noise=true , address=77.90.31.15:4708 , piece_id=67LX4NZJUKLYHRTSW3RKHHXZRKMGWJ2KV32FDA2KE3DY364CJ7OA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.790730467 (103h9m40.639337554s) [noise=true , address=45.91.116.14:4206 , piece_id=OVX63SL5OGJTT7HWDRIUV4BDVEKN2MD4ORUUK5M4JIEWX4J6VHFA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.791174576 (103h9m40.639781656s) [noise=true , address=45.91.116.13:4176 , piece_id=GOWXIBOORGZWFIBLMG46HQ4J632B3E6I3MVXNLTWPP2TY3IF2JUQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.898860128 (103h9m40.747467214s) [upload=82448]: Next returned nil segment
Jul 23 05:59:58.898881813 (103h9m40.747488906s) [upload=82448]: Waiting for error group managing segments...
Jul 23 05:59:58.905078785 (103h9m40.753685854s) [noise=true , address=45.91.116.14:4206 , piece_id=OVX63SL5OGJTT7HWDRIUV4BDVEKN2MD4ORUUK5M4JIEWX4J6VHFA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.906441982 (103h9m40.755049063s) [noise=true , address=77.90.31.15:4708 , piece_id=67LX4NZJUKLYHRTSW3RKHHXZRKMGWJ2KV32FDA2KE3DY364CJ7OA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.910115252 (103h9m40.758722337s) [noise=true , address=77.90.31.15:4706 , piece_id=C3JOUPYB77PA2EHPSNPMDETERE724PBALBCSZ6MJCTZ6YRPUC2UA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.910652695 (103h9m40.759259762s) [noise=true , address=104.36.82.9:4035 , piece_id=74BSWBLVWYQXPUI2PGV5NIJDSE6MSFKWK65TCVV6CNZKVQBQVNEA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.910672036 (103h9m40.759279117s) [noise=true , address=77.90.31.11:4569 , piece_id=MTNLU22JHOYH6VSRMWLKQGSCDI5GIADPIXBVVSC2RIJLTHPMXTOA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.911483080 (103h9m40.760090146s) [noise=true , address=45.91.116.13:4176 , piece_id=GOWXIBOORGZWFIBLMG46HQ4J632B3E6I3MVXNLTWPP2TY3IF2JUQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.911951970 (103h9m40.760559052s) [noise=true , address=45.91.116.10:4103 , piece_id=UGRDLKKFJRNCCDNDQTAY2LLOIS7CGI6KLNDZ7KJDOUNYWFNUAYQQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.912044937 (103h9m40.760652006s) [noise=true , address=77.90.31.12:4622 , piece_id=EZJAG75JRZTYILS7545KYAI37YOUTHHIOOSH26KIWMUQ4BCG556A, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.912649865 (103h9m40.761256931s) [noise=true , address=77.90.31.17:4799 , piece_id=RUTSDNZEXKJ5Y7WQBIVTMFUIJRYLL6DDNNJEXZAL5S7CIATWYHBQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.913487977 (103h9m40.76209505s) [noise=true , address=79.127.219.47:4040 , piece_id=BKWXBAVFXQ2RKG27WGPEAWG5UTHBVODW2BB4EGKHV6SN4EFI7KIQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.914486860 (103h9m40.763093927s) [noise=true , address=104.36.82.53:4121 , piece_id=GJLXLW2A5RZUF4MBUP5AZAWFKTWGBDADIIWVIUBXPQPPGPKYZ4YA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.918080770 (103h9m40.766687852s) [noise=true , address=104.36.82.5:4870 , piece_id=AYTLB2OL7OAI5EO2ZE6LXIQLCGSGDXKDKT6V5HXIPAREK656MDPQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.924012392 (103h9m40.772619461s) [noise=true , address=79.127.219.51:4202 , piece_id=LJZ5A5HAHG4CGXT5J2WNSMXD7HCYSCBUEYGAGOES3NWIVV34BXEQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.932892483 (103h9m40.78149955s) [noise=true , address=104.36.82.42:4818 , piece_id=A6U7TDLTAZHE5IILHHCZKIHCQINHCJSWTBYNXJV7GLCWTIBNB2VA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.932913099 (103h9m40.781520167s) [seg_pos={0 0}, upload=82448]: Segment reached optimal threshold of 14 pieces.
Jul 23 05:59:58.933104957 (103h9m40.781712037s) [noise=true , address=104.36.82.10:4105 , piece_id=GY72JPHTZOGEVFDG3FM5W5ZSYOPSY3BZXEME2Y37VVR2LR37SIUA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GhDrKMcDEg2RxamqKtiHphpgNdvtZYdHXrjzGsNJFjBFYeEen): protocol: expected piece hash; context canceled; EOF
Jul 23 05:59:58.933505265 (103h9m40.782112345s) [noise=true , address=79.127.205.233:4645 , piece_id=6YAS5E7QYEZAJXB6HUY3R4PLHC4Z7EYTMSFWNFWD5AP7HKXOKTUQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12NR52pweTqZNNMTyc1EHSbifDKB4awrnnb4megtZHwANSrR18T): protocol: read tcp 159.223.143.39:46566->79.127.205.233:4645: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 23 05:59:58.933568113 (103h9m40.782175194s) [noise=true , address=79.127.205.235:4680 , piece_id=T3V7PMZXJOHWT77MQ52PUMFNAIAX4DLMTIIVL6XNVTNA565O5WCQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GGvQQLoBcTniZST8Hz9h8eCLWbU4DrXn66DGy33iGLXaFgJLj): protocol: read tcp 159.223.143.39:45394->79.127.205.235:4680: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 23 05:59:58.933618915 (103h9m40.782225986s) [noise=true , address=79.127.205.246:4280 , piece_id=NPBN75MFQYLCG4XKRJPLHJU5JCYBNQYIFQ7MKKB3JKXLT4TKF7AA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rZDBvhrfNrYQB6h9MGhx1tNtQ7AH9ehUF4SfNMCJRpv4SKrDX): protocol: read tcp 159.223.143.39:45458->79.127.205.246:4280: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 23 05:59:58.933631191 (103h9m40.78223826s) [seg_pos={0 0}, upload=82448]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jul 23 05:59:58.933756265 (103h9m40.782363345s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 23 05:59:58.933762566 (103h9m40.782369645s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 23 05:59:59.025408726 (103h9m40.874015808s) [upload=82448]: Done uploading segments...
Jul 23 05:59:59.025427547 (103h9m40.874034627s) [upload=82448]: Upload finished. err: <nil>
Duration: 700.463296ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........] duration:
804.322604ms
FAIL (2025-07-23 06:10:23 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
23 06:00:19.742341858 (103h10m1.590948949s) [noise=true , address=104.36.82.4:4804 , piece_id=SMCQKI5GIWKEHYLNSVUGHWQNBHP3S4C5RBB4E3K5CFMXE54Q4WAQ, seg_pos={0 0}, upload=82455]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g): protocol: expected piece hash; context canceled; EOF
Jul 23 06:00:19.742959346 (103h10m1.59156642s) [noise=true , address=77.90.31.16:4734 , piece_id=NFCPPB6P25YMO5UZRM4CQKMNRV5Q2FXMSR6M4VSTEERBMVNLYJJA, seg_pos={0 0}, upload=82455]: Done uploading piece. err: <nil>
Jul 23 06:00:19.743002232 (103h10m1.591609306s) [seg_pos={0 0}, upload=82455]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 23 06:00:19.743286680 (103h10m1.591893764s) [upload=82455]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 23 06:00:19.743293769 (103h10m1.591900841s) [upload=82455]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 23 06:00:19.815641707 (103h10m1.664248798s) [upload=82455]: Done uploading segments...
Jul 23 06:00:19.815666887 (103h10m1.664273954s) [upload=82455]: Upload finished. err: <nil>
Jul 23 06:01:08.401614022 (103h10m50.250221093s) [upload=82463]: Starting upload
Jul 23 06:01:08.401709823 (103h10m50.250316908s) [upload=82463]: Done starting upload
Jul 23 06:01:08.402138436 (103h10m50.250745505s) [upload=82463]: Uploading segments...
Jul 23 06:01:08.402189044 (103h10m50.250796137s) [upload=82463]: Got next segment. Inline: false
Jul 23 06:01:08.402231416 (103h10m50.250838487s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 23 06:01:08.402237731 (103h10m50.250844804s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 23 06:01:08.450301042 (103h10m50.298908146s) [seg_pos={0 0}, upload=82463]: Begin upload segment...
Jul 23 06:01:08.450364720 (103h10m50.298971798s) [seg_pos={0 0}, upload=82463]: Done begin upload segment.
Jul 23 06:01:08.450425015 (103h10m50.2990321s) [noise=true , address=45.91.116.5:4299 , piece_id=K3URA4ZO4R2CAGZLTO6YJTZQOZ6YIOB7XOJ3AQAVGDFIMVU63BIA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.450431954 (103h10m50.299039031s) [noise=true , address=45.91.116.7:4010 , piece_id=WVMOM2PPTVXY7NYI5UH25PFW6RODIUJWN73YB6FW6UOAMNY4C4SA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.450900413 (103h10m50.299507502s) [noise=true , address=77.90.31.17:4772 , piece_id=63V72ATK3KI3ADRC4N5HTRIXSEB3DRM7N73MVZNDPZJ7EVO3RZJA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.450963993 (103h10m50.299571083s) [noise=true , address=45.91.116.8:4048 , piece_id=AROG4HVUX7R6MHG3DT3N2FIQJBBEVVEVKQHSAGZCWMISWTFSQMAQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.451375091 (103h10m50.299982178s) [noise=true , address=45.91.116.13:4155 , piece_id=KMDEG3QMJNHRF7TETZAFVN3TOZWTQNW7IRTCVI3WLNE7NJAAAPCA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.451400410 (103h10m50.300007509s) [noise=true , address=77.90.31.12:4592 , piece_id=NAWAKPN4LUPCFZIXDIEPN2GKO3CSGX35LZNHI4QDCGYBYAMWBGOA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.451837258 (103h10m50.300444352s) [noise=true , address=77.90.31.17:4771 , piece_id=VCHLVD4PCT6SGJNCFA4LSEDIXSRMX27JH7VLUXDBM2PMIGZ54CNA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.452344092 (103h10m50.300951186s) [noise=true , address=45.91.116.4:4937 , piece_id=DZILXASXCQNDQGTAIYQIY4LCXHENKF6LLWHME5ILDWL3IZOSWMCQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.452773889 (103h10m50.301380982s) [noise=true , address=45.91.116.11:4119 , piece_id=LOMOQU77BVHAPGZ23ZY6CG63FZFQX2IG5QRY2P46WKSGB5AF5O3Q, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.453168858 (103h10m50.301775945s) [noise=true , address=45.91.116.5:4298 , piece_id=AIURH3V65NZD3QI3A5YGBJ222ANC4LTHEYFWTOFIC6SPVH3O4CKA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.453559736 (103h10m50.302166813s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQJDPJRZQEJHTED4UJBNSJ2CGAL6FPTLLIIRFTOVBUK65VSPLGA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.460854171 (103h10m50.309461265s) [noise=true , address=77.90.31.14:4661 , piece_id=MA3CRNAREYTT6ZDO7Q7QRSAOVQQ3EM3CHZTWYQYE6QOWNMYQBBQQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.462111720 (103h10m50.310718803s) [noise=true , address=45.91.116.12:4225 , piece_id=MFM7FUANQ2WU4GK46NPKKKFDUXSVVQ3MGK7GM3QBR7GMOD563PTQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.462819102 (103h10m50.311426189s) [noise=true , address=104.36.82.13:4186 , piece_id=O5BVVJBIWINWK2GXWWJDETED6T3Z462AAUQWCG6LGC7PVVFSHZBA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.463600977 (103h10m50.312208063s) [noise=true , address=45.91.116.7:4011 , piece_id=GKVDRTZ2TY2ESEKU2FKIJND4LXDMITNWQORE4Z47OQ454UZMM7BA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.464356397 (103h10m50.312963485s) [noise=true , address=77.90.31.13:4626 , piece_id=D63E7JGO5RRTEJGXVUMNEC3UYXHS45HBOB7S67VPCWLHJXNH7OSQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.587088382 (103h10m50.435695454s) [upload=82463]: Next returned nil segment
Jul 23 06:01:08.587107670 (103h10m50.435714757s) [upload=82463]: Waiting for error group managing segments...
Jul 23 06:01:08.609523184 (103h10m50.458130271s) [noise=true , address=77.90.31.14:4661 , piece_id=MA3CRNAREYTT6ZDO7Q7QRSAOVQQ3EM3CHZTWYQYE6QOWNMYQBBQQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.612539671 (103h10m50.46114674s) [noise=true , address=45.91.116.5:4299 , piece_id=K3URA4ZO4R2CAGZLTO6YJTZQOZ6YIOB7XOJ3AQAVGDFIMVU63BIA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.617758941 (103h10m50.466366025s) [noise=true , address=45.91.116.13:4155 , piece_id=KMDEG3QMJNHRF7TETZAFVN3TOZWTQNW7IRTCVI3WLNE7NJAAAPCA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.618513849 (103h10m50.467120922s) [noise=true , address=45.91.116.8:4048 , piece_id=AROG4HVUX7R6MHG3DT3N2FIQJBBEVVEVKQHSAGZCWMISWTFSQMAQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.624184177 (103h10m50.47279126s) [noise=true , address=45.91.116.7:4010 , piece_id=WVMOM2PPTVXY7NYI5UH25PFW6RODIUJWN73YB6FW6UOAMNY4C4SA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.625290831 (103h10m50.473897916s) [noise=true , address=45.91.116.7:4011 , piece_id=GKVDRTZ2TY2ESEKU2FKIJND4LXDMITNWQORE4Z47OQ454UZMM7BA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.628897649 (103h10m50.477504746s) [noise=true , address=45.91.116.5:4298 , piece_id=AIURH3V65NZD3QI3A5YGBJ222ANC4LTHEYFWTOFIC6SPVH3O4CKA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.629542622 (103h10m50.478149697s) [noise=true , address=45.91.116.12:4225 , piece_id=MFM7FUANQ2WU4GK46NPKKKFDUXSVVQ3MGK7GM3QBR7GMOD563PTQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.630003986 (103h10m50.478611073s) [noise=true , address=77.90.31.17:4772 , piece_id=63V72ATK3KI3ADRC4N5HTRIXSEB3DRM7N73MVZNDPZJ7EVO3RZJA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.630441959 (103h10m50.479049047s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQJDPJRZQEJHTED4UJBNSJ2CGAL6FPTLLIIRFTOVBUK65VSPLGA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.630941221 (103h10m50.479548305s) [noise=true , address=77.90.31.12:4592 , piece_id=NAWAKPN4LUPCFZIXDIEPN2GKO3CSGX35LZNHI4QDCGYBYAMWBGOA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.631378970 (103h10m50.47998605s) [noise=true , address=77.90.31.13:4626 , piece_id=D63E7JGO5RRTEJGXVUMNEC3UYXHS45HBOB7S67VPCWLHJXNH7OSQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.631840658 (103h10m50.480447733s) [noise=true , address=77.90.31.17:4771 , piece_id=VCHLVD4PCT6SGJNCFA4LSEDIXSRMX27JH7VLUXDBM2PMIGZ54CNA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.631850523 (103h10m50.480457598s) [seg_pos={0 0}, upload=82463]: Segment reached optimal threshold of 13 pieces.
Jul 23 06:01:08.631982956 (103h10m50.480590031s) [noise=true , address=104.36.82.13:4186 , piece_id=O5BVVJBIWINWK2GXWWJDETED6T3Z462AAUQWCG6LGC7PVVFSHZBA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF
Jul 23 06:01:08.632087803 (103h10m50.480694892s) [noise=true , address=45.91.116.4:4937 , piece_id=DZILXASXCQNDQGTAIYQIY4LCXHENKF6LLWHME5ILDWL3IZOSWMCQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF
Jul 23 06:01:08.633101069 (103h10m50.481708157s) [noise=true , address=45.91.116.11:4119 , piece_id=LOMOQU77BVHAPGZ23ZY6CG63FZFQX2IG5QRY2P46WKSGB5AF5O3Q, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.633132335 (103h10m50.481739428s) [seg_pos={0 0}, upload=82463]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 23 06:01:08.633221956 (103h10m50.481829049s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 23 06:01:08.633272252 (103h10m50.481879341s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 23 06:01:08.710605841 (103h10m50.559212933s) [upload=82463]: Done uploading segments...
Jul 23 06:01:08.710632085 (103h10m50.559239157s) [upload=82463]: Upload finished. err: <nil>
Duration: 804.361824ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........] duration:
603.799955ms
FAIL (2025-07-23 06:10:01 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
23 06:00:19.742341858 (103h10m1.590948949s) [noise=true , address=104.36.82.4:4804 , piece_id=SMCQKI5GIWKEHYLNSVUGHWQNBHP3S4C5RBB4E3K5CFMXE54Q4WAQ, seg_pos={0 0}, upload=82455]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g): protocol: expected piece hash; context canceled; EOF
Jul 23 06:00:19.742959346 (103h10m1.59156642s) [noise=true , address=77.90.31.16:4734 , piece_id=NFCPPB6P25YMO5UZRM4CQKMNRV5Q2FXMSR6M4VSTEERBMVNLYJJA, seg_pos={0 0}, upload=82455]: Done uploading piece. err: <nil>
Jul 23 06:00:19.743002232 (103h10m1.591609306s) [seg_pos={0 0}, upload=82455]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 23 06:00:19.743286680 (103h10m1.591893764s) [upload=82455]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 23 06:00:19.743293769 (103h10m1.591900841s) [upload=82455]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 23 06:00:19.815641707 (103h10m1.664248798s) [upload=82455]: Done uploading segments...
Jul 23 06:00:19.815666887 (103h10m1.664273954s) [upload=82455]: Upload finished. err: <nil>
Jul 23 06:01:08.401614022 (103h10m50.250221093s) [upload=82463]: Starting upload
Jul 23 06:01:08.401709823 (103h10m50.250316908s) [upload=82463]: Done starting upload
Jul 23 06:01:08.402138436 (103h10m50.250745505s) [upload=82463]: Uploading segments...
Jul 23 06:01:08.402189044 (103h10m50.250796137s) [upload=82463]: Got next segment. Inline: false
Jul 23 06:01:08.402231416 (103h10m50.250838487s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 23 06:01:08.402237731 (103h10m50.250844804s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 23 06:01:08.450301042 (103h10m50.298908146s) [seg_pos={0 0}, upload=82463]: Begin upload segment...
Jul 23 06:01:08.450364720 (103h10m50.298971798s) [seg_pos={0 0}, upload=82463]: Done begin upload segment.
Jul 23 06:01:08.450425015 (103h10m50.2990321s) [noise=true , address=45.91.116.5:4299 , piece_id=K3URA4ZO4R2CAGZLTO6YJTZQOZ6YIOB7XOJ3AQAVGDFIMVU63BIA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.450431954 (103h10m50.299039031s) [noise=true , address=45.91.116.7:4010 , piece_id=WVMOM2PPTVXY7NYI5UH25PFW6RODIUJWN73YB6FW6UOAMNY4C4SA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.450900413 (103h10m50.299507502s) [noise=true , address=77.90.31.17:4772 , piece_id=63V72ATK3KI3ADRC4N5HTRIXSEB3DRM7N73MVZNDPZJ7EVO3RZJA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.450963993 (103h10m50.299571083s) [noise=true , address=45.91.116.8:4048 , piece_id=AROG4HVUX7R6MHG3DT3N2FIQJBBEVVEVKQHSAGZCWMISWTFSQMAQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.451375091 (103h10m50.299982178s) [noise=true , address=45.91.116.13:4155 , piece_id=KMDEG3QMJNHRF7TETZAFVN3TOZWTQNW7IRTCVI3WLNE7NJAAAPCA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.451400410 (103h10m50.300007509s) [noise=true , address=77.90.31.12:4592 , piece_id=NAWAKPN4LUPCFZIXDIEPN2GKO3CSGX35LZNHI4QDCGYBYAMWBGOA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.451837258 (103h10m50.300444352s) [noise=true , address=77.90.31.17:4771 , piece_id=VCHLVD4PCT6SGJNCFA4LSEDIXSRMX27JH7VLUXDBM2PMIGZ54CNA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.452344092 (103h10m50.300951186s) [noise=true , address=45.91.116.4:4937 , piece_id=DZILXASXCQNDQGTAIYQIY4LCXHENKF6LLWHME5ILDWL3IZOSWMCQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.452773889 (103h10m50.301380982s) [noise=true , address=45.91.116.11:4119 , piece_id=LOMOQU77BVHAPGZ23ZY6CG63FZFQX2IG5QRY2P46WKSGB5AF5O3Q, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.453168858 (103h10m50.301775945s) [noise=true , address=45.91.116.5:4298 , piece_id=AIURH3V65NZD3QI3A5YGBJ222ANC4LTHEYFWTOFIC6SPVH3O4CKA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.453559736 (103h10m50.302166813s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQJDPJRZQEJHTED4UJBNSJ2CGAL6FPTLLIIRFTOVBUK65VSPLGA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.460854171 (103h10m50.309461265s) [noise=true , address=77.90.31.14:4661 , piece_id=MA3CRNAREYTT6ZDO7Q7QRSAOVQQ3EM3CHZTWYQYE6QOWNMYQBBQQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.462111720 (103h10m50.310718803s) [noise=true , address=45.91.116.12:4225 , piece_id=MFM7FUANQ2WU4GK46NPKKKFDUXSVVQ3MGK7GM3QBR7GMOD563PTQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.462819102 (103h10m50.311426189s) [noise=true , address=104.36.82.13:4186 , piece_id=O5BVVJBIWINWK2GXWWJDETED6T3Z462AAUQWCG6LGC7PVVFSHZBA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.463600977 (103h10m50.312208063s) [noise=true , address=45.91.116.7:4011 , piece_id=GKVDRTZ2TY2ESEKU2FKIJND4LXDMITNWQORE4Z47OQ454UZMM7BA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.464356397 (103h10m50.312963485s) [noise=true , address=77.90.31.13:4626 , piece_id=D63E7JGO5RRTEJGXVUMNEC3UYXHS45HBOB7S67VPCWLHJXNH7OSQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.587088382 (103h10m50.435695454s) [upload=82463]: Next returned nil segment
Jul 23 06:01:08.587107670 (103h10m50.435714757s) [upload=82463]: Waiting for error group managing segments...
Jul 23 06:01:08.609523184 (103h10m50.458130271s) [noise=true , address=77.90.31.14:4661 , piece_id=MA3CRNAREYTT6ZDO7Q7QRSAOVQQ3EM3CHZTWYQYE6QOWNMYQBBQQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.612539671 (103h10m50.46114674s) [noise=true , address=45.91.116.5:4299 , piece_id=K3URA4ZO4R2CAGZLTO6YJTZQOZ6YIOB7XOJ3AQAVGDFIMVU63BIA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.617758941 (103h10m50.466366025s) [noise=true , address=45.91.116.13:4155 , piece_id=KMDEG3QMJNHRF7TETZAFVN3TOZWTQNW7IRTCVI3WLNE7NJAAAPCA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.618513849 (103h10m50.467120922s) [noise=true , address=45.91.116.8:4048 , piece_id=AROG4HVUX7R6MHG3DT3N2FIQJBBEVVEVKQHSAGZCWMISWTFSQMAQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.624184177 (103h10m50.47279126s) [noise=true , address=45.91.116.7:4010 , piece_id=WVMOM2PPTVXY7NYI5UH25PFW6RODIUJWN73YB6FW6UOAMNY4C4SA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.625290831 (103h10m50.473897916s) [noise=true , address=45.91.116.7:4011 , piece_id=GKVDRTZ2TY2ESEKU2FKIJND4LXDMITNWQORE4Z47OQ454UZMM7BA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.628897649 (103h10m50.477504746s) [noise=true , address=45.91.116.5:4298 , piece_id=AIURH3V65NZD3QI3A5YGBJ222ANC4LTHEYFWTOFIC6SPVH3O4CKA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.629542622 (103h10m50.478149697s) [noise=true , address=45.91.116.12:4225 , piece_id=MFM7FUANQ2WU4GK46NPKKKFDUXSVVQ3MGK7GM3QBR7GMOD563PTQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.630003986 (103h10m50.478611073s) [noise=true , address=77.90.31.17:4772 , piece_id=63V72ATK3KI3ADRC4N5HTRIXSEB3DRM7N73MVZNDPZJ7EVO3RZJA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.630441959 (103h10m50.479049047s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQJDPJRZQEJHTED4UJBNSJ2CGAL6FPTLLIIRFTOVBUK65VSPLGA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.630941221 (103h10m50.479548305s) [noise=true , address=77.90.31.12:4592 , piece_id=NAWAKPN4LUPCFZIXDIEPN2GKO3CSGX35LZNHI4QDCGYBYAMWBGOA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.631378970 (103h10m50.47998605s) [noise=true , address=77.90.31.13:4626 , piece_id=D63E7JGO5RRTEJGXVUMNEC3UYXHS45HBOB7S67VPCWLHJXNH7OSQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.631840658 (103h10m50.480447733s) [noise=true , address=77.90.31.17:4771 , piece_id=VCHLVD4PCT6SGJNCFA4LSEDIXSRMX27JH7VLUXDBM2PMIGZ54CNA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.631850523 (103h10m50.480457598s) [seg_pos={0 0}, upload=82463]: Segment reached optimal threshold of 13 pieces.
Jul 23 06:01:08.631982956 (103h10m50.480590031s) [noise=true , address=104.36.82.13:4186 , piece_id=O5BVVJBIWINWK2GXWWJDETED6T3Z462AAUQWCG6LGC7PVVFSHZBA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF
Jul 23 06:01:08.632087803 (103h10m50.480694892s) [noise=true , address=45.91.116.4:4937 , piece_id=DZILXASXCQNDQGTAIYQIY4LCXHENKF6LLWHME5ILDWL3IZOSWMCQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF
Jul 23 06:01:08.633101069 (103h10m50.481708157s) [noise=true , address=45.91.116.11:4119 , piece_id=LOMOQU77BVHAPGZ23ZY6CG63FZFQX2IG5QRY2P46WKSGB5AF5O3Q, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.633132335 (103h10m50.481739428s) [seg_pos={0 0}, upload=82463]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 23 06:01:08.633221956 (103h10m50.481829049s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 23 06:01:08.633272252 (103h10m50.481879341s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 23 06:01:08.710605841 (103h10m50.559212933s) [upload=82463]: Done uploading segments...
Jul 23 06:01:08.710632085 (103h10m50.559239157s) [upload=82463]: Upload finished. err: <nil>
Duration: 603.839229ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........] duration:
575.585646ms
FAIL (2025-07-23 06:09:50 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
s) [upload=82448]: Got next segment. Inline: false
Jul 23 05:59:58.732247463 (103h9m40.580854536s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 23 05:59:58.732253833 (103h9m40.580860918s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 23 05:59:58.787057645 (103h9m40.635664731s) [seg_pos={0 0}, upload=82448]: Begin upload segment...
Jul 23 05:59:58.787102519 (103h9m40.635709584s) [seg_pos={0 0}, upload=82448]: Done begin upload segment.
Jul 23 05:59:58.787136403 (103h9m40.635743482s) [noise=true , address=77.90.31.12:4622 , piece_id=EZJAG75JRZTYILS7545KYAI37YOUTHHIOOSH26KIWMUQ4BCG556A, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787169354 (103h9m40.635776438s) [noise=true , address=79.127.205.235:4680 , piece_id=T3V7PMZXJOHWT77MQ52PUMFNAIAX4DLMTIIVL6XNVTNA565O5WCQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787544550 (103h9m40.636151681s) [noise=true , address=104.36.82.10:4105 , piece_id=GY72JPHTZOGEVFDG3FM5W5ZSYOPSY3BZXEME2Y37VVR2LR37SIUA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787598825 (103h9m40.636205915s) [noise=true , address=79.127.205.246:4280 , piece_id=NPBN75MFQYLCG4XKRJPLHJU5JCYBNQYIFQ7MKKB3JKXLT4TKF7AA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787946084 (103h9m40.636553167s) [noise=true , address=77.90.31.17:4799 , piece_id=RUTSDNZEXKJ5Y7WQBIVTMFUIJRYLL6DDNNJEXZAL5S7CIATWYHBQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788041731 (103h9m40.636648823s) [noise=true , address=77.90.31.11:4569 , piece_id=MTNLU22JHOYH6VSRMWLKQGSCDI5GIADPIXBVVSC2RIJLTHPMXTOA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788383218 (103h9m40.636990305s) [noise=true , address=104.36.82.42:4818 , piece_id=A6U7TDLTAZHE5IILHHCZKIHCQINHCJSWTBYNXJV7GLCWTIBNB2VA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788633554 (103h9m40.637240644s) [noise=true , address=79.127.219.51:4202 , piece_id=LJZ5A5HAHG4CGXT5J2WNSMXD7HCYSCBUEYGAGOES3NWIVV34BXEQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788735479 (103h9m40.637342568s) [noise=true , address=79.127.219.47:4040 , piece_id=BKWXBAVFXQ2RKG27WGPEAWG5UTHBVODW2BB4EGKHV6SN4EFI7KIQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789072444 (103h9m40.637679533s) [noise=true , address=77.90.31.15:4706 , piece_id=C3JOUPYB77PA2EHPSNPMDETERE724PBALBCSZ6MJCTZ6YRPUC2UA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789122202 (103h9m40.637729291s) [noise=true , address=104.36.82.9:4035 , piece_id=74BSWBLVWYQXPUI2PGV5NIJDSE6MSFKWK65TCVV6CNZKVQBQVNEA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789393961 (103h9m40.638001047s) [noise=true , address=45.91.116.10:4103 , piece_id=UGRDLKKFJRNCCDNDQTAY2LLOIS7CGI6KLNDZ7KJDOUNYWFNUAYQQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789591397 (103h9m40.638198479s) [noise=true , address=104.36.82.5:4870 , piece_id=AYTLB2OL7OAI5EO2ZE6LXIQLCGSGDXKDKT6V5HXIPAREK656MDPQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789716087 (103h9m40.638323176s) [noise=true , address=79.127.205.233:4645 , piece_id=6YAS5E7QYEZAJXB6HUY3R4PLHC4Z7EYTMSFWNFWD5AP7HKXOKTUQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789937996 (103h9m40.638545078s) [noise=true , address=104.36.82.53:4121 , piece_id=GJLXLW2A5RZUF4MBUP5AZAWFKTWGBDADIIWVIUBXPQPPGPKYZ4YA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.790288745 (103h9m40.638895835s) [noise=true , address=77.90.31.15:4708 , piece_id=67LX4NZJUKLYHRTSW3RKHHXZRKMGWJ2KV32FDA2KE3DY364CJ7OA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.790730467 (103h9m40.639337554s) [noise=true , address=45.91.116.14:4206 , piece_id=OVX63SL5OGJTT7HWDRIUV4BDVEKN2MD4ORUUK5M4JIEWX4J6VHFA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.791174576 (103h9m40.639781656s) [noise=true , address=45.91.116.13:4176 , piece_id=GOWXIBOORGZWFIBLMG46HQ4J632B3E6I3MVXNLTWPP2TY3IF2JUQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.898860128 (103h9m40.747467214s) [upload=82448]: Next returned nil segment
Jul 23 05:59:58.898881813 (103h9m40.747488906s) [upload=82448]: Waiting for error group managing segments...
Jul 23 05:59:58.905078785 (103h9m40.753685854s) [noise=true , address=45.91.116.14:4206 , piece_id=OVX63SL5OGJTT7HWDRIUV4BDVEKN2MD4ORUUK5M4JIEWX4J6VHFA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.906441982 (103h9m40.755049063s) [noise=true , address=77.90.31.15:4708 , piece_id=67LX4NZJUKLYHRTSW3RKHHXZRKMGWJ2KV32FDA2KE3DY364CJ7OA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.910115252 (103h9m40.758722337s) [noise=true , address=77.90.31.15:4706 , piece_id=C3JOUPYB77PA2EHPSNPMDETERE724PBALBCSZ6MJCTZ6YRPUC2UA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.910652695 (103h9m40.759259762s) [noise=true , address=104.36.82.9:4035 , piece_id=74BSWBLVWYQXPUI2PGV5NIJDSE6MSFKWK65TCVV6CNZKVQBQVNEA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.910672036 (103h9m40.759279117s) [noise=true , address=77.90.31.11:4569 , piece_id=MTNLU22JHOYH6VSRMWLKQGSCDI5GIADPIXBVVSC2RIJLTHPMXTOA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.911483080 (103h9m40.760090146s) [noise=true , address=45.91.116.13:4176 , piece_id=GOWXIBOORGZWFIBLMG46HQ4J632B3E6I3MVXNLTWPP2TY3IF2JUQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.911951970 (103h9m40.760559052s) [noise=true , address=45.91.116.10:4103 , piece_id=UGRDLKKFJRNCCDNDQTAY2LLOIS7CGI6KLNDZ7KJDOUNYWFNUAYQQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.912044937 (103h9m40.760652006s) [noise=true , address=77.90.31.12:4622 , piece_id=EZJAG75JRZTYILS7545KYAI37YOUTHHIOOSH26KIWMUQ4BCG556A, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.912649865 (103h9m40.761256931s) [noise=true , address=77.90.31.17:4799 , piece_id=RUTSDNZEXKJ5Y7WQBIVTMFUIJRYLL6DDNNJEXZAL5S7CIATWYHBQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.913487977 (103h9m40.76209505s) [noise=true , address=79.127.219.47:4040 , piece_id=BKWXBAVFXQ2RKG27WGPEAWG5UTHBVODW2BB4EGKHV6SN4EFI7KIQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.914486860 (103h9m40.763093927s) [noise=true , address=104.36.82.53:4121 , piece_id=GJLXLW2A5RZUF4MBUP5AZAWFKTWGBDADIIWVIUBXPQPPGPKYZ4YA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.918080770 (103h9m40.766687852s) [noise=true , address=104.36.82.5:4870 , piece_id=AYTLB2OL7OAI5EO2ZE6LXIQLCGSGDXKDKT6V5HXIPAREK656MDPQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.924012392 (103h9m40.772619461s) [noise=true , address=79.127.219.51:4202 , piece_id=LJZ5A5HAHG4CGXT5J2WNSMXD7HCYSCBUEYGAGOES3NWIVV34BXEQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.932892483 (103h9m40.78149955s) [noise=true , address=104.36.82.42:4818 , piece_id=A6U7TDLTAZHE5IILHHCZKIHCQINHCJSWTBYNXJV7GLCWTIBNB2VA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.932913099 (103h9m40.781520167s) [seg_pos={0 0}, upload=82448]: Segment reached optimal threshold of 14 pieces.
Jul 23 05:59:58.933104957 (103h9m40.781712037s) [noise=true , address=104.36.82.10:4105 , piece_id=GY72JPHTZOGEVFDG3FM5W5ZSYOPSY3BZXEME2Y37VVR2LR37SIUA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GhDrKMcDEg2RxamqKtiHphpgNdvtZYdHXrjzGsNJFjBFYeEen): protocol: expected piece hash; context canceled; EOF
Jul 23 05:59:58.933505265 (103h9m40.782112345s) [noise=true , address=79.127.205.233:4645 , piece_id=6YAS5E7QYEZAJXB6HUY3R4PLHC4Z7EYTMSFWNFWD5AP7HKXOKTUQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12NR52pweTqZNNMTyc1EHSbifDKB4awrnnb4megtZHwANSrR18T): protocol: read tcp 159.223.143.39:46566->79.127.205.233:4645: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 23 05:59:58.933568113 (103h9m40.782175194s) [noise=true , address=79.127.205.235:4680 , piece_id=T3V7PMZXJOHWT77MQ52PUMFNAIAX4DLMTIIVL6XNVTNA565O5WCQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GGvQQLoBcTniZST8Hz9h8eCLWbU4DrXn66DGy33iGLXaFgJLj): protocol: read tcp 159.223.143.39:45394->79.127.205.235:4680: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 23 05:59:58.933618915 (103h9m40.782225986s) [noise=true , address=79.127.205.246:4280 , piece_id=NPBN75MFQYLCG4XKRJPLHJU5JCYBNQYIFQ7MKKB3JKXLT4TKF7AA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rZDBvhrfNrYQB6h9MGhx1tNtQ7AH9ehUF4SfNMCJRpv4SKrDX): protocol: read tcp 159.223.143.39:45458->79.127.205.246:4280: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 23 05:59:58.933631191 (103h9m40.78223826s) [seg_pos={0 0}, upload=82448]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jul 23 05:59:58.933756265 (103h9m40.782363345s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 23 05:59:58.933762566 (103h9m40.782369645s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 23 05:59:59.025408726 (103h9m40.874015808s) [upload=82448]: Done uploading segments...
Jul 23 05:59:59.025427547 (103h9m40.874034627s) [upload=82448]: Upload finished. err: <nil>
Duration: 575.691649ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
535.895278ms
FAIL (2025-07-23 06:09:19 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
23 06:00:19.742341858 (103h10m1.590948949s) [noise=true , address=104.36.82.4:4804 , piece_id=SMCQKI5GIWKEHYLNSVUGHWQNBHP3S4C5RBB4E3K5CFMXE54Q4WAQ, seg_pos={0 0}, upload=82455]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g): protocol: expected piece hash; context canceled; EOF
Jul 23 06:00:19.742959346 (103h10m1.59156642s) [noise=true , address=77.90.31.16:4734 , piece_id=NFCPPB6P25YMO5UZRM4CQKMNRV5Q2FXMSR6M4VSTEERBMVNLYJJA, seg_pos={0 0}, upload=82455]: Done uploading piece. err: <nil>
Jul 23 06:00:19.743002232 (103h10m1.591609306s) [seg_pos={0 0}, upload=82455]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 23 06:00:19.743286680 (103h10m1.591893764s) [upload=82455]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 23 06:00:19.743293769 (103h10m1.591900841s) [upload=82455]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 23 06:00:19.815641707 (103h10m1.664248798s) [upload=82455]: Done uploading segments...
Jul 23 06:00:19.815666887 (103h10m1.664273954s) [upload=82455]: Upload finished. err: <nil>
Jul 23 06:01:08.401614022 (103h10m50.250221093s) [upload=82463]: Starting upload
Jul 23 06:01:08.401709823 (103h10m50.250316908s) [upload=82463]: Done starting upload
Jul 23 06:01:08.402138436 (103h10m50.250745505s) [upload=82463]: Uploading segments...
Jul 23 06:01:08.402189044 (103h10m50.250796137s) [upload=82463]: Got next segment. Inline: false
Jul 23 06:01:08.402231416 (103h10m50.250838487s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 23 06:01:08.402237731 (103h10m50.250844804s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 23 06:01:08.450301042 (103h10m50.298908146s) [seg_pos={0 0}, upload=82463]: Begin upload segment...
Jul 23 06:01:08.450364720 (103h10m50.298971798s) [seg_pos={0 0}, upload=82463]: Done begin upload segment.
Jul 23 06:01:08.450425015 (103h10m50.2990321s) [noise=true , address=45.91.116.5:4299 , piece_id=K3URA4ZO4R2CAGZLTO6YJTZQOZ6YIOB7XOJ3AQAVGDFIMVU63BIA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.450431954 (103h10m50.299039031s) [noise=true , address=45.91.116.7:4010 , piece_id=WVMOM2PPTVXY7NYI5UH25PFW6RODIUJWN73YB6FW6UOAMNY4C4SA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.450900413 (103h10m50.299507502s) [noise=true , address=77.90.31.17:4772 , piece_id=63V72ATK3KI3ADRC4N5HTRIXSEB3DRM7N73MVZNDPZJ7EVO3RZJA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.450963993 (103h10m50.299571083s) [noise=true , address=45.91.116.8:4048 , piece_id=AROG4HVUX7R6MHG3DT3N2FIQJBBEVVEVKQHSAGZCWMISWTFSQMAQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.451375091 (103h10m50.299982178s) [noise=true , address=45.91.116.13:4155 , piece_id=KMDEG3QMJNHRF7TETZAFVN3TOZWTQNW7IRTCVI3WLNE7NJAAAPCA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.451400410 (103h10m50.300007509s) [noise=true , address=77.90.31.12:4592 , piece_id=NAWAKPN4LUPCFZIXDIEPN2GKO3CSGX35LZNHI4QDCGYBYAMWBGOA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.451837258 (103h10m50.300444352s) [noise=true , address=77.90.31.17:4771 , piece_id=VCHLVD4PCT6SGJNCFA4LSEDIXSRMX27JH7VLUXDBM2PMIGZ54CNA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.452344092 (103h10m50.300951186s) [noise=true , address=45.91.116.4:4937 , piece_id=DZILXASXCQNDQGTAIYQIY4LCXHENKF6LLWHME5ILDWL3IZOSWMCQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.452773889 (103h10m50.301380982s) [noise=true , address=45.91.116.11:4119 , piece_id=LOMOQU77BVHAPGZ23ZY6CG63FZFQX2IG5QRY2P46WKSGB5AF5O3Q, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.453168858 (103h10m50.301775945s) [noise=true , address=45.91.116.5:4298 , piece_id=AIURH3V65NZD3QI3A5YGBJ222ANC4LTHEYFWTOFIC6SPVH3O4CKA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.453559736 (103h10m50.302166813s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQJDPJRZQEJHTED4UJBNSJ2CGAL6FPTLLIIRFTOVBUK65VSPLGA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.460854171 (103h10m50.309461265s) [noise=true , address=77.90.31.14:4661 , piece_id=MA3CRNAREYTT6ZDO7Q7QRSAOVQQ3EM3CHZTWYQYE6QOWNMYQBBQQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.462111720 (103h10m50.310718803s) [noise=true , address=45.91.116.12:4225 , piece_id=MFM7FUANQ2WU4GK46NPKKKFDUXSVVQ3MGK7GM3QBR7GMOD563PTQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.462819102 (103h10m50.311426189s) [noise=true , address=104.36.82.13:4186 , piece_id=O5BVVJBIWINWK2GXWWJDETED6T3Z462AAUQWCG6LGC7PVVFSHZBA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.463600977 (103h10m50.312208063s) [noise=true , address=45.91.116.7:4011 , piece_id=GKVDRTZ2TY2ESEKU2FKIJND4LXDMITNWQORE4Z47OQ454UZMM7BA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.464356397 (103h10m50.312963485s) [noise=true , address=77.90.31.13:4626 , piece_id=D63E7JGO5RRTEJGXVUMNEC3UYXHS45HBOB7S67VPCWLHJXNH7OSQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.587088382 (103h10m50.435695454s) [upload=82463]: Next returned nil segment
Jul 23 06:01:08.587107670 (103h10m50.435714757s) [upload=82463]: Waiting for error group managing segments...
Jul 23 06:01:08.609523184 (103h10m50.458130271s) [noise=true , address=77.90.31.14:4661 , piece_id=MA3CRNAREYTT6ZDO7Q7QRSAOVQQ3EM3CHZTWYQYE6QOWNMYQBBQQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.612539671 (103h10m50.46114674s) [noise=true , address=45.91.116.5:4299 , piece_id=K3URA4ZO4R2CAGZLTO6YJTZQOZ6YIOB7XOJ3AQAVGDFIMVU63BIA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.617758941 (103h10m50.466366025s) [noise=true , address=45.91.116.13:4155 , piece_id=KMDEG3QMJNHRF7TETZAFVN3TOZWTQNW7IRTCVI3WLNE7NJAAAPCA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.618513849 (103h10m50.467120922s) [noise=true , address=45.91.116.8:4048 , piece_id=AROG4HVUX7R6MHG3DT3N2FIQJBBEVVEVKQHSAGZCWMISWTFSQMAQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.624184177 (103h10m50.47279126s) [noise=true , address=45.91.116.7:4010 , piece_id=WVMOM2PPTVXY7NYI5UH25PFW6RODIUJWN73YB6FW6UOAMNY4C4SA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.625290831 (103h10m50.473897916s) [noise=true , address=45.91.116.7:4011 , piece_id=GKVDRTZ2TY2ESEKU2FKIJND4LXDMITNWQORE4Z47OQ454UZMM7BA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.628897649 (103h10m50.477504746s) [noise=true , address=45.91.116.5:4298 , piece_id=AIURH3V65NZD3QI3A5YGBJ222ANC4LTHEYFWTOFIC6SPVH3O4CKA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.629542622 (103h10m50.478149697s) [noise=true , address=45.91.116.12:4225 , piece_id=MFM7FUANQ2WU4GK46NPKKKFDUXSVVQ3MGK7GM3QBR7GMOD563PTQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.630003986 (103h10m50.478611073s) [noise=true , address=77.90.31.17:4772 , piece_id=63V72ATK3KI3ADRC4N5HTRIXSEB3DRM7N73MVZNDPZJ7EVO3RZJA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.630441959 (103h10m50.479049047s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQJDPJRZQEJHTED4UJBNSJ2CGAL6FPTLLIIRFTOVBUK65VSPLGA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.630941221 (103h10m50.479548305s) [noise=true , address=77.90.31.12:4592 , piece_id=NAWAKPN4LUPCFZIXDIEPN2GKO3CSGX35LZNHI4QDCGYBYAMWBGOA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.631378970 (103h10m50.47998605s) [noise=true , address=77.90.31.13:4626 , piece_id=D63E7JGO5RRTEJGXVUMNEC3UYXHS45HBOB7S67VPCWLHJXNH7OSQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.631840658 (103h10m50.480447733s) [noise=true , address=77.90.31.17:4771 , piece_id=VCHLVD4PCT6SGJNCFA4LSEDIXSRMX27JH7VLUXDBM2PMIGZ54CNA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.631850523 (103h10m50.480457598s) [seg_pos={0 0}, upload=82463]: Segment reached optimal threshold of 13 pieces.
Jul 23 06:01:08.631982956 (103h10m50.480590031s) [noise=true , address=104.36.82.13:4186 , piece_id=O5BVVJBIWINWK2GXWWJDETED6T3Z462AAUQWCG6LGC7PVVFSHZBA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF
Jul 23 06:01:08.632087803 (103h10m50.480694892s) [noise=true , address=45.91.116.4:4937 , piece_id=DZILXASXCQNDQGTAIYQIY4LCXHENKF6LLWHME5ILDWL3IZOSWMCQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF
Jul 23 06:01:08.633101069 (103h10m50.481708157s) [noise=true , address=45.91.116.11:4119 , piece_id=LOMOQU77BVHAPGZ23ZY6CG63FZFQX2IG5QRY2P46WKSGB5AF5O3Q, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.633132335 (103h10m50.481739428s) [seg_pos={0 0}, upload=82463]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 23 06:01:08.633221956 (103h10m50.481829049s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 23 06:01:08.633272252 (103h10m50.481879341s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 23 06:01:08.710605841 (103h10m50.559212933s) [upload=82463]: Done uploading segments...
Jul 23 06:01:08.710632085 (103h10m50.559239157s) [upload=82463]: Upload finished. err: <nil>
Duration: 535.990824ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
1.076734263s
FAIL (2025-07-23 06:09:14 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
s) [upload=82448]: Got next segment. Inline: false
Jul 23 05:59:58.732247463 (103h9m40.580854536s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 23 05:59:58.732253833 (103h9m40.580860918s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 23 05:59:58.787057645 (103h9m40.635664731s) [seg_pos={0 0}, upload=82448]: Begin upload segment...
Jul 23 05:59:58.787102519 (103h9m40.635709584s) [seg_pos={0 0}, upload=82448]: Done begin upload segment.
Jul 23 05:59:58.787136403 (103h9m40.635743482s) [noise=true , address=77.90.31.12:4622 , piece_id=EZJAG75JRZTYILS7545KYAI37YOUTHHIOOSH26KIWMUQ4BCG556A, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787169354 (103h9m40.635776438s) [noise=true , address=79.127.205.235:4680 , piece_id=T3V7PMZXJOHWT77MQ52PUMFNAIAX4DLMTIIVL6XNVTNA565O5WCQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787544550 (103h9m40.636151681s) [noise=true , address=104.36.82.10:4105 , piece_id=GY72JPHTZOGEVFDG3FM5W5ZSYOPSY3BZXEME2Y37VVR2LR37SIUA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787598825 (103h9m40.636205915s) [noise=true , address=79.127.205.246:4280 , piece_id=NPBN75MFQYLCG4XKRJPLHJU5JCYBNQYIFQ7MKKB3JKXLT4TKF7AA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787946084 (103h9m40.636553167s) [noise=true , address=77.90.31.17:4799 , piece_id=RUTSDNZEXKJ5Y7WQBIVTMFUIJRYLL6DDNNJEXZAL5S7CIATWYHBQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788041731 (103h9m40.636648823s) [noise=true , address=77.90.31.11:4569 , piece_id=MTNLU22JHOYH6VSRMWLKQGSCDI5GIADPIXBVVSC2RIJLTHPMXTOA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788383218 (103h9m40.636990305s) [noise=true , address=104.36.82.42:4818 , piece_id=A6U7TDLTAZHE5IILHHCZKIHCQINHCJSWTBYNXJV7GLCWTIBNB2VA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788633554 (103h9m40.637240644s) [noise=true , address=79.127.219.51:4202 , piece_id=LJZ5A5HAHG4CGXT5J2WNSMXD7HCYSCBUEYGAGOES3NWIVV34BXEQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788735479 (103h9m40.637342568s) [noise=true , address=79.127.219.47:4040 , piece_id=BKWXBAVFXQ2RKG27WGPEAWG5UTHBVODW2BB4EGKHV6SN4EFI7KIQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789072444 (103h9m40.637679533s) [noise=true , address=77.90.31.15:4706 , piece_id=C3JOUPYB77PA2EHPSNPMDETERE724PBALBCSZ6MJCTZ6YRPUC2UA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789122202 (103h9m40.637729291s) [noise=true , address=104.36.82.9:4035 , piece_id=74BSWBLVWYQXPUI2PGV5NIJDSE6MSFKWK65TCVV6CNZKVQBQVNEA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789393961 (103h9m40.638001047s) [noise=true , address=45.91.116.10:4103 , piece_id=UGRDLKKFJRNCCDNDQTAY2LLOIS7CGI6KLNDZ7KJDOUNYWFNUAYQQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789591397 (103h9m40.638198479s) [noise=true , address=104.36.82.5:4870 , piece_id=AYTLB2OL7OAI5EO2ZE6LXIQLCGSGDXKDKT6V5HXIPAREK656MDPQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789716087 (103h9m40.638323176s) [noise=true , address=79.127.205.233:4645 , piece_id=6YAS5E7QYEZAJXB6HUY3R4PLHC4Z7EYTMSFWNFWD5AP7HKXOKTUQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789937996 (103h9m40.638545078s) [noise=true , address=104.36.82.53:4121 , piece_id=GJLXLW2A5RZUF4MBUP5AZAWFKTWGBDADIIWVIUBXPQPPGPKYZ4YA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.790288745 (103h9m40.638895835s) [noise=true , address=77.90.31.15:4708 , piece_id=67LX4NZJUKLYHRTSW3RKHHXZRKMGWJ2KV32FDA2KE3DY364CJ7OA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.790730467 (103h9m40.639337554s) [noise=true , address=45.91.116.14:4206 , piece_id=OVX63SL5OGJTT7HWDRIUV4BDVEKN2MD4ORUUK5M4JIEWX4J6VHFA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.791174576 (103h9m40.639781656s) [noise=true , address=45.91.116.13:4176 , piece_id=GOWXIBOORGZWFIBLMG46HQ4J632B3E6I3MVXNLTWPP2TY3IF2JUQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.898860128 (103h9m40.747467214s) [upload=82448]: Next returned nil segment
Jul 23 05:59:58.898881813 (103h9m40.747488906s) [upload=82448]: Waiting for error group managing segments...
Jul 23 05:59:58.905078785 (103h9m40.753685854s) [noise=true , address=45.91.116.14:4206 , piece_id=OVX63SL5OGJTT7HWDRIUV4BDVEKN2MD4ORUUK5M4JIEWX4J6VHFA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.906441982 (103h9m40.755049063s) [noise=true , address=77.90.31.15:4708 , piece_id=67LX4NZJUKLYHRTSW3RKHHXZRKMGWJ2KV32FDA2KE3DY364CJ7OA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.910115252 (103h9m40.758722337s) [noise=true , address=77.90.31.15:4706 , piece_id=C3JOUPYB77PA2EHPSNPMDETERE724PBALBCSZ6MJCTZ6YRPUC2UA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.910652695 (103h9m40.759259762s) [noise=true , address=104.36.82.9:4035 , piece_id=74BSWBLVWYQXPUI2PGV5NIJDSE6MSFKWK65TCVV6CNZKVQBQVNEA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.910672036 (103h9m40.759279117s) [noise=true , address=77.90.31.11:4569 , piece_id=MTNLU22JHOYH6VSRMWLKQGSCDI5GIADPIXBVVSC2RIJLTHPMXTOA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.911483080 (103h9m40.760090146s) [noise=true , address=45.91.116.13:4176 , piece_id=GOWXIBOORGZWFIBLMG46HQ4J632B3E6I3MVXNLTWPP2TY3IF2JUQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.911951970 (103h9m40.760559052s) [noise=true , address=45.91.116.10:4103 , piece_id=UGRDLKKFJRNCCDNDQTAY2LLOIS7CGI6KLNDZ7KJDOUNYWFNUAYQQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.912044937 (103h9m40.760652006s) [noise=true , address=77.90.31.12:4622 , piece_id=EZJAG75JRZTYILS7545KYAI37YOUTHHIOOSH26KIWMUQ4BCG556A, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.912649865 (103h9m40.761256931s) [noise=true , address=77.90.31.17:4799 , piece_id=RUTSDNZEXKJ5Y7WQBIVTMFUIJRYLL6DDNNJEXZAL5S7CIATWYHBQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.913487977 (103h9m40.76209505s) [noise=true , address=79.127.219.47:4040 , piece_id=BKWXBAVFXQ2RKG27WGPEAWG5UTHBVODW2BB4EGKHV6SN4EFI7KIQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.914486860 (103h9m40.763093927s) [noise=true , address=104.36.82.53:4121 , piece_id=GJLXLW2A5RZUF4MBUP5AZAWFKTWGBDADIIWVIUBXPQPPGPKYZ4YA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.918080770 (103h9m40.766687852s) [noise=true , address=104.36.82.5:4870 , piece_id=AYTLB2OL7OAI5EO2ZE6LXIQLCGSGDXKDKT6V5HXIPAREK656MDPQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.924012392 (103h9m40.772619461s) [noise=true , address=79.127.219.51:4202 , piece_id=LJZ5A5HAHG4CGXT5J2WNSMXD7HCYSCBUEYGAGOES3NWIVV34BXEQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.932892483 (103h9m40.78149955s) [noise=true , address=104.36.82.42:4818 , piece_id=A6U7TDLTAZHE5IILHHCZKIHCQINHCJSWTBYNXJV7GLCWTIBNB2VA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.932913099 (103h9m40.781520167s) [seg_pos={0 0}, upload=82448]: Segment reached optimal threshold of 14 pieces.
Jul 23 05:59:58.933104957 (103h9m40.781712037s) [noise=true , address=104.36.82.10:4105 , piece_id=GY72JPHTZOGEVFDG3FM5W5ZSYOPSY3BZXEME2Y37VVR2LR37SIUA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GhDrKMcDEg2RxamqKtiHphpgNdvtZYdHXrjzGsNJFjBFYeEen): protocol: expected piece hash; context canceled; EOF
Jul 23 05:59:58.933505265 (103h9m40.782112345s) [noise=true , address=79.127.205.233:4645 , piece_id=6YAS5E7QYEZAJXB6HUY3R4PLHC4Z7EYTMSFWNFWD5AP7HKXOKTUQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12NR52pweTqZNNMTyc1EHSbifDKB4awrnnb4megtZHwANSrR18T): protocol: read tcp 159.223.143.39:46566->79.127.205.233:4645: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 23 05:59:58.933568113 (103h9m40.782175194s) [noise=true , address=79.127.205.235:4680 , piece_id=T3V7PMZXJOHWT77MQ52PUMFNAIAX4DLMTIIVL6XNVTNA565O5WCQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GGvQQLoBcTniZST8Hz9h8eCLWbU4DrXn66DGy33iGLXaFgJLj): protocol: read tcp 159.223.143.39:45394->79.127.205.235:4680: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 23 05:59:58.933618915 (103h9m40.782225986s) [noise=true , address=79.127.205.246:4280 , piece_id=NPBN75MFQYLCG4XKRJPLHJU5JCYBNQYIFQ7MKKB3JKXLT4TKF7AA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rZDBvhrfNrYQB6h9MGhx1tNtQ7AH9ehUF4SfNMCJRpv4SKrDX): protocol: read tcp 159.223.143.39:45458->79.127.205.246:4280: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 23 05:59:58.933631191 (103h9m40.78223826s) [seg_pos={0 0}, upload=82448]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jul 23 05:59:58.933756265 (103h9m40.782363345s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 23 05:59:58.933762566 (103h9m40.782369645s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 23 05:59:59.025408726 (103h9m40.874015808s) [upload=82448]: Done uploading segments...
Jul 23 05:59:59.025427547 (103h9m40.874034627s) [upload=82448]: Upload finished. err: <nil>
Duration: 1.076768914s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........] duration:
507.839291ms
FAIL (2025-07-23 06:08:40 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
23 06:00:19.742341858 (103h10m1.590948949s) [noise=true , address=104.36.82.4:4804 , piece_id=SMCQKI5GIWKEHYLNSVUGHWQNBHP3S4C5RBB4E3K5CFMXE54Q4WAQ, seg_pos={0 0}, upload=82455]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g): protocol: expected piece hash; context canceled; EOF
Jul 23 06:00:19.742959346 (103h10m1.59156642s) [noise=true , address=77.90.31.16:4734 , piece_id=NFCPPB6P25YMO5UZRM4CQKMNRV5Q2FXMSR6M4VSTEERBMVNLYJJA, seg_pos={0 0}, upload=82455]: Done uploading piece. err: <nil>
Jul 23 06:00:19.743002232 (103h10m1.591609306s) [seg_pos={0 0}, upload=82455]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 23 06:00:19.743286680 (103h10m1.591893764s) [upload=82455]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 23 06:00:19.743293769 (103h10m1.591900841s) [upload=82455]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 23 06:00:19.815641707 (103h10m1.664248798s) [upload=82455]: Done uploading segments...
Jul 23 06:00:19.815666887 (103h10m1.664273954s) [upload=82455]: Upload finished. err: <nil>
Jul 23 06:01:08.401614022 (103h10m50.250221093s) [upload=82463]: Starting upload
Jul 23 06:01:08.401709823 (103h10m50.250316908s) [upload=82463]: Done starting upload
Jul 23 06:01:08.402138436 (103h10m50.250745505s) [upload=82463]: Uploading segments...
Jul 23 06:01:08.402189044 (103h10m50.250796137s) [upload=82463]: Got next segment. Inline: false
Jul 23 06:01:08.402231416 (103h10m50.250838487s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 23 06:01:08.402237731 (103h10m50.250844804s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 23 06:01:08.450301042 (103h10m50.298908146s) [seg_pos={0 0}, upload=82463]: Begin upload segment...
Jul 23 06:01:08.450364720 (103h10m50.298971798s) [seg_pos={0 0}, upload=82463]: Done begin upload segment.
Jul 23 06:01:08.450425015 (103h10m50.2990321s) [noise=true , address=45.91.116.5:4299 , piece_id=K3URA4ZO4R2CAGZLTO6YJTZQOZ6YIOB7XOJ3AQAVGDFIMVU63BIA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.450431954 (103h10m50.299039031s) [noise=true , address=45.91.116.7:4010 , piece_id=WVMOM2PPTVXY7NYI5UH25PFW6RODIUJWN73YB6FW6UOAMNY4C4SA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.450900413 (103h10m50.299507502s) [noise=true , address=77.90.31.17:4772 , piece_id=63V72ATK3KI3ADRC4N5HTRIXSEB3DRM7N73MVZNDPZJ7EVO3RZJA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.450963993 (103h10m50.299571083s) [noise=true , address=45.91.116.8:4048 , piece_id=AROG4HVUX7R6MHG3DT3N2FIQJBBEVVEVKQHSAGZCWMISWTFSQMAQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.451375091 (103h10m50.299982178s) [noise=true , address=45.91.116.13:4155 , piece_id=KMDEG3QMJNHRF7TETZAFVN3TOZWTQNW7IRTCVI3WLNE7NJAAAPCA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.451400410 (103h10m50.300007509s) [noise=true , address=77.90.31.12:4592 , piece_id=NAWAKPN4LUPCFZIXDIEPN2GKO3CSGX35LZNHI4QDCGYBYAMWBGOA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.451837258 (103h10m50.300444352s) [noise=true , address=77.90.31.17:4771 , piece_id=VCHLVD4PCT6SGJNCFA4LSEDIXSRMX27JH7VLUXDBM2PMIGZ54CNA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.452344092 (103h10m50.300951186s) [noise=true , address=45.91.116.4:4937 , piece_id=DZILXASXCQNDQGTAIYQIY4LCXHENKF6LLWHME5ILDWL3IZOSWMCQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.452773889 (103h10m50.301380982s) [noise=true , address=45.91.116.11:4119 , piece_id=LOMOQU77BVHAPGZ23ZY6CG63FZFQX2IG5QRY2P46WKSGB5AF5O3Q, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.453168858 (103h10m50.301775945s) [noise=true , address=45.91.116.5:4298 , piece_id=AIURH3V65NZD3QI3A5YGBJ222ANC4LTHEYFWTOFIC6SPVH3O4CKA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.453559736 (103h10m50.302166813s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQJDPJRZQEJHTED4UJBNSJ2CGAL6FPTLLIIRFTOVBUK65VSPLGA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.460854171 (103h10m50.309461265s) [noise=true , address=77.90.31.14:4661 , piece_id=MA3CRNAREYTT6ZDO7Q7QRSAOVQQ3EM3CHZTWYQYE6QOWNMYQBBQQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.462111720 (103h10m50.310718803s) [noise=true , address=45.91.116.12:4225 , piece_id=MFM7FUANQ2WU4GK46NPKKKFDUXSVVQ3MGK7GM3QBR7GMOD563PTQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.462819102 (103h10m50.311426189s) [noise=true , address=104.36.82.13:4186 , piece_id=O5BVVJBIWINWK2GXWWJDETED6T3Z462AAUQWCG6LGC7PVVFSHZBA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.463600977 (103h10m50.312208063s) [noise=true , address=45.91.116.7:4011 , piece_id=GKVDRTZ2TY2ESEKU2FKIJND4LXDMITNWQORE4Z47OQ454UZMM7BA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.464356397 (103h10m50.312963485s) [noise=true , address=77.90.31.13:4626 , piece_id=D63E7JGO5RRTEJGXVUMNEC3UYXHS45HBOB7S67VPCWLHJXNH7OSQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.587088382 (103h10m50.435695454s) [upload=82463]: Next returned nil segment
Jul 23 06:01:08.587107670 (103h10m50.435714757s) [upload=82463]: Waiting for error group managing segments...
Jul 23 06:01:08.609523184 (103h10m50.458130271s) [noise=true , address=77.90.31.14:4661 , piece_id=MA3CRNAREYTT6ZDO7Q7QRSAOVQQ3EM3CHZTWYQYE6QOWNMYQBBQQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.612539671 (103h10m50.46114674s) [noise=true , address=45.91.116.5:4299 , piece_id=K3URA4ZO4R2CAGZLTO6YJTZQOZ6YIOB7XOJ3AQAVGDFIMVU63BIA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.617758941 (103h10m50.466366025s) [noise=true , address=45.91.116.13:4155 , piece_id=KMDEG3QMJNHRF7TETZAFVN3TOZWTQNW7IRTCVI3WLNE7NJAAAPCA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.618513849 (103h10m50.467120922s) [noise=true , address=45.91.116.8:4048 , piece_id=AROG4HVUX7R6MHG3DT3N2FIQJBBEVVEVKQHSAGZCWMISWTFSQMAQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.624184177 (103h10m50.47279126s) [noise=true , address=45.91.116.7:4010 , piece_id=WVMOM2PPTVXY7NYI5UH25PFW6RODIUJWN73YB6FW6UOAMNY4C4SA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.625290831 (103h10m50.473897916s) [noise=true , address=45.91.116.7:4011 , piece_id=GKVDRTZ2TY2ESEKU2FKIJND4LXDMITNWQORE4Z47OQ454UZMM7BA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.628897649 (103h10m50.477504746s) [noise=true , address=45.91.116.5:4298 , piece_id=AIURH3V65NZD3QI3A5YGBJ222ANC4LTHEYFWTOFIC6SPVH3O4CKA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.629542622 (103h10m50.478149697s) [noise=true , address=45.91.116.12:4225 , piece_id=MFM7FUANQ2WU4GK46NPKKKFDUXSVVQ3MGK7GM3QBR7GMOD563PTQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.630003986 (103h10m50.478611073s) [noise=true , address=77.90.31.17:4772 , piece_id=63V72ATK3KI3ADRC4N5HTRIXSEB3DRM7N73MVZNDPZJ7EVO3RZJA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.630441959 (103h10m50.479049047s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQJDPJRZQEJHTED4UJBNSJ2CGAL6FPTLLIIRFTOVBUK65VSPLGA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.630941221 (103h10m50.479548305s) [noise=true , address=77.90.31.12:4592 , piece_id=NAWAKPN4LUPCFZIXDIEPN2GKO3CSGX35LZNHI4QDCGYBYAMWBGOA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.631378970 (103h10m50.47998605s) [noise=true , address=77.90.31.13:4626 , piece_id=D63E7JGO5RRTEJGXVUMNEC3UYXHS45HBOB7S67VPCWLHJXNH7OSQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.631840658 (103h10m50.480447733s) [noise=true , address=77.90.31.17:4771 , piece_id=VCHLVD4PCT6SGJNCFA4LSEDIXSRMX27JH7VLUXDBM2PMIGZ54CNA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.631850523 (103h10m50.480457598s) [seg_pos={0 0}, upload=82463]: Segment reached optimal threshold of 13 pieces.
Jul 23 06:01:08.631982956 (103h10m50.480590031s) [noise=true , address=104.36.82.13:4186 , piece_id=O5BVVJBIWINWK2GXWWJDETED6T3Z462AAUQWCG6LGC7PVVFSHZBA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF
Jul 23 06:01:08.632087803 (103h10m50.480694892s) [noise=true , address=45.91.116.4:4937 , piece_id=DZILXASXCQNDQGTAIYQIY4LCXHENKF6LLWHME5ILDWL3IZOSWMCQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF
Jul 23 06:01:08.633101069 (103h10m50.481708157s) [noise=true , address=45.91.116.11:4119 , piece_id=LOMOQU77BVHAPGZ23ZY6CG63FZFQX2IG5QRY2P46WKSGB5AF5O3Q, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.633132335 (103h10m50.481739428s) [seg_pos={0 0}, upload=82463]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 23 06:01:08.633221956 (103h10m50.481829049s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 23 06:01:08.633272252 (103h10m50.481879341s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 23 06:01:08.710605841 (103h10m50.559212933s) [upload=82463]: Done uploading segments...
Jul 23 06:01:08.710632085 (103h10m50.559239157s) [upload=82463]: Upload finished. err: <nil>
Duration: 507.873327ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......] duration:
751.570883ms
FAIL (2025-07-23 06:08:03 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
23 06:00:19.742341858 (103h10m1.590948949s) [noise=true , address=104.36.82.4:4804 , piece_id=SMCQKI5GIWKEHYLNSVUGHWQNBHP3S4C5RBB4E3K5CFMXE54Q4WAQ, seg_pos={0 0}, upload=82455]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g): protocol: expected piece hash; context canceled; EOF
Jul 23 06:00:19.742959346 (103h10m1.59156642s) [noise=true , address=77.90.31.16:4734 , piece_id=NFCPPB6P25YMO5UZRM4CQKMNRV5Q2FXMSR6M4VSTEERBMVNLYJJA, seg_pos={0 0}, upload=82455]: Done uploading piece. err: <nil>
Jul 23 06:00:19.743002232 (103h10m1.591609306s) [seg_pos={0 0}, upload=82455]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 23 06:00:19.743286680 (103h10m1.591893764s) [upload=82455]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 23 06:00:19.743293769 (103h10m1.591900841s) [upload=82455]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 23 06:00:19.815641707 (103h10m1.664248798s) [upload=82455]: Done uploading segments...
Jul 23 06:00:19.815666887 (103h10m1.664273954s) [upload=82455]: Upload finished. err: <nil>
Jul 23 06:01:08.401614022 (103h10m50.250221093s) [upload=82463]: Starting upload
Jul 23 06:01:08.401709823 (103h10m50.250316908s) [upload=82463]: Done starting upload
Jul 23 06:01:08.402138436 (103h10m50.250745505s) [upload=82463]: Uploading segments...
Jul 23 06:01:08.402189044 (103h10m50.250796137s) [upload=82463]: Got next segment. Inline: false
Jul 23 06:01:08.402231416 (103h10m50.250838487s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 23 06:01:08.402237731 (103h10m50.250844804s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 23 06:01:08.450301042 (103h10m50.298908146s) [seg_pos={0 0}, upload=82463]: Begin upload segment...
Jul 23 06:01:08.450364720 (103h10m50.298971798s) [seg_pos={0 0}, upload=82463]: Done begin upload segment.
Jul 23 06:01:08.450425015 (103h10m50.2990321s) [noise=true , address=45.91.116.5:4299 , piece_id=K3URA4ZO4R2CAGZLTO6YJTZQOZ6YIOB7XOJ3AQAVGDFIMVU63BIA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.450431954 (103h10m50.299039031s) [noise=true , address=45.91.116.7:4010 , piece_id=WVMOM2PPTVXY7NYI5UH25PFW6RODIUJWN73YB6FW6UOAMNY4C4SA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.450900413 (103h10m50.299507502s) [noise=true , address=77.90.31.17:4772 , piece_id=63V72ATK3KI3ADRC4N5HTRIXSEB3DRM7N73MVZNDPZJ7EVO3RZJA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.450963993 (103h10m50.299571083s) [noise=true , address=45.91.116.8:4048 , piece_id=AROG4HVUX7R6MHG3DT3N2FIQJBBEVVEVKQHSAGZCWMISWTFSQMAQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.451375091 (103h10m50.299982178s) [noise=true , address=45.91.116.13:4155 , piece_id=KMDEG3QMJNHRF7TETZAFVN3TOZWTQNW7IRTCVI3WLNE7NJAAAPCA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.451400410 (103h10m50.300007509s) [noise=true , address=77.90.31.12:4592 , piece_id=NAWAKPN4LUPCFZIXDIEPN2GKO3CSGX35LZNHI4QDCGYBYAMWBGOA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.451837258 (103h10m50.300444352s) [noise=true , address=77.90.31.17:4771 , piece_id=VCHLVD4PCT6SGJNCFA4LSEDIXSRMX27JH7VLUXDBM2PMIGZ54CNA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.452344092 (103h10m50.300951186s) [noise=true , address=45.91.116.4:4937 , piece_id=DZILXASXCQNDQGTAIYQIY4LCXHENKF6LLWHME5ILDWL3IZOSWMCQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.452773889 (103h10m50.301380982s) [noise=true , address=45.91.116.11:4119 , piece_id=LOMOQU77BVHAPGZ23ZY6CG63FZFQX2IG5QRY2P46WKSGB5AF5O3Q, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.453168858 (103h10m50.301775945s) [noise=true , address=45.91.116.5:4298 , piece_id=AIURH3V65NZD3QI3A5YGBJ222ANC4LTHEYFWTOFIC6SPVH3O4CKA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.453559736 (103h10m50.302166813s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQJDPJRZQEJHTED4UJBNSJ2CGAL6FPTLLIIRFTOVBUK65VSPLGA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.460854171 (103h10m50.309461265s) [noise=true , address=77.90.31.14:4661 , piece_id=MA3CRNAREYTT6ZDO7Q7QRSAOVQQ3EM3CHZTWYQYE6QOWNMYQBBQQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.462111720 (103h10m50.310718803s) [noise=true , address=45.91.116.12:4225 , piece_id=MFM7FUANQ2WU4GK46NPKKKFDUXSVVQ3MGK7GM3QBR7GMOD563PTQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.462819102 (103h10m50.311426189s) [noise=true , address=104.36.82.13:4186 , piece_id=O5BVVJBIWINWK2GXWWJDETED6T3Z462AAUQWCG6LGC7PVVFSHZBA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.463600977 (103h10m50.312208063s) [noise=true , address=45.91.116.7:4011 , piece_id=GKVDRTZ2TY2ESEKU2FKIJND4LXDMITNWQORE4Z47OQ454UZMM7BA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.464356397 (103h10m50.312963485s) [noise=true , address=77.90.31.13:4626 , piece_id=D63E7JGO5RRTEJGXVUMNEC3UYXHS45HBOB7S67VPCWLHJXNH7OSQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.587088382 (103h10m50.435695454s) [upload=82463]: Next returned nil segment
Jul 23 06:01:08.587107670 (103h10m50.435714757s) [upload=82463]: Waiting for error group managing segments...
Jul 23 06:01:08.609523184 (103h10m50.458130271s) [noise=true , address=77.90.31.14:4661 , piece_id=MA3CRNAREYTT6ZDO7Q7QRSAOVQQ3EM3CHZTWYQYE6QOWNMYQBBQQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.612539671 (103h10m50.46114674s) [noise=true , address=45.91.116.5:4299 , piece_id=K3URA4ZO4R2CAGZLTO6YJTZQOZ6YIOB7XOJ3AQAVGDFIMVU63BIA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.617758941 (103h10m50.466366025s) [noise=true , address=45.91.116.13:4155 , piece_id=KMDEG3QMJNHRF7TETZAFVN3TOZWTQNW7IRTCVI3WLNE7NJAAAPCA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.618513849 (103h10m50.467120922s) [noise=true , address=45.91.116.8:4048 , piece_id=AROG4HVUX7R6MHG3DT3N2FIQJBBEVVEVKQHSAGZCWMISWTFSQMAQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.624184177 (103h10m50.47279126s) [noise=true , address=45.91.116.7:4010 , piece_id=WVMOM2PPTVXY7NYI5UH25PFW6RODIUJWN73YB6FW6UOAMNY4C4SA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.625290831 (103h10m50.473897916s) [noise=true , address=45.91.116.7:4011 , piece_id=GKVDRTZ2TY2ESEKU2FKIJND4LXDMITNWQORE4Z47OQ454UZMM7BA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.628897649 (103h10m50.477504746s) [noise=true , address=45.91.116.5:4298 , piece_id=AIURH3V65NZD3QI3A5YGBJ222ANC4LTHEYFWTOFIC6SPVH3O4CKA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.629542622 (103h10m50.478149697s) [noise=true , address=45.91.116.12:4225 , piece_id=MFM7FUANQ2WU4GK46NPKKKFDUXSVVQ3MGK7GM3QBR7GMOD563PTQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.630003986 (103h10m50.478611073s) [noise=true , address=77.90.31.17:4772 , piece_id=63V72ATK3KI3ADRC4N5HTRIXSEB3DRM7N73MVZNDPZJ7EVO3RZJA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.630441959 (103h10m50.479049047s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQJDPJRZQEJHTED4UJBNSJ2CGAL6FPTLLIIRFTOVBUK65VSPLGA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.630941221 (103h10m50.479548305s) [noise=true , address=77.90.31.12:4592 , piece_id=NAWAKPN4LUPCFZIXDIEPN2GKO3CSGX35LZNHI4QDCGYBYAMWBGOA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.631378970 (103h10m50.47998605s) [noise=true , address=77.90.31.13:4626 , piece_id=D63E7JGO5RRTEJGXVUMNEC3UYXHS45HBOB7S67VPCWLHJXNH7OSQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.631840658 (103h10m50.480447733s) [noise=true , address=77.90.31.17:4771 , piece_id=VCHLVD4PCT6SGJNCFA4LSEDIXSRMX27JH7VLUXDBM2PMIGZ54CNA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.631850523 (103h10m50.480457598s) [seg_pos={0 0}, upload=82463]: Segment reached optimal threshold of 13 pieces.
Jul 23 06:01:08.631982956 (103h10m50.480590031s) [noise=true , address=104.36.82.13:4186 , piece_id=O5BVVJBIWINWK2GXWWJDETED6T3Z462AAUQWCG6LGC7PVVFSHZBA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF
Jul 23 06:01:08.632087803 (103h10m50.480694892s) [noise=true , address=45.91.116.4:4937 , piece_id=DZILXASXCQNDQGTAIYQIY4LCXHENKF6LLWHME5ILDWL3IZOSWMCQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF
Jul 23 06:01:08.633101069 (103h10m50.481708157s) [noise=true , address=45.91.116.11:4119 , piece_id=LOMOQU77BVHAPGZ23ZY6CG63FZFQX2IG5QRY2P46WKSGB5AF5O3Q, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.633132335 (103h10m50.481739428s) [seg_pos={0 0}, upload=82463]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 23 06:01:08.633221956 (103h10m50.481829049s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 23 06:01:08.633272252 (103h10m50.481879341s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 23 06:01:08.710605841 (103h10m50.559212933s) [upload=82463]: Done uploading segments...
Jul 23 06:01:08.710632085 (103h10m50.559239157s) [upload=82463]: Upload finished. err: <nil>
Duration: 751.686429ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........] duration:
556.505917ms
FAIL (2025-07-23 06:07:59 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
s) [upload=82448]: Got next segment. Inline: false
Jul 23 05:59:58.732247463 (103h9m40.580854536s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 23 05:59:58.732253833 (103h9m40.580860918s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 23 05:59:58.787057645 (103h9m40.635664731s) [seg_pos={0 0}, upload=82448]: Begin upload segment...
Jul 23 05:59:58.787102519 (103h9m40.635709584s) [seg_pos={0 0}, upload=82448]: Done begin upload segment.
Jul 23 05:59:58.787136403 (103h9m40.635743482s) [noise=true , address=77.90.31.12:4622 , piece_id=EZJAG75JRZTYILS7545KYAI37YOUTHHIOOSH26KIWMUQ4BCG556A, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787169354 (103h9m40.635776438s) [noise=true , address=79.127.205.235:4680 , piece_id=T3V7PMZXJOHWT77MQ52PUMFNAIAX4DLMTIIVL6XNVTNA565O5WCQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787544550 (103h9m40.636151681s) [noise=true , address=104.36.82.10:4105 , piece_id=GY72JPHTZOGEVFDG3FM5W5ZSYOPSY3BZXEME2Y37VVR2LR37SIUA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787598825 (103h9m40.636205915s) [noise=true , address=79.127.205.246:4280 , piece_id=NPBN75MFQYLCG4XKRJPLHJU5JCYBNQYIFQ7MKKB3JKXLT4TKF7AA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787946084 (103h9m40.636553167s) [noise=true , address=77.90.31.17:4799 , piece_id=RUTSDNZEXKJ5Y7WQBIVTMFUIJRYLL6DDNNJEXZAL5S7CIATWYHBQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788041731 (103h9m40.636648823s) [noise=true , address=77.90.31.11:4569 , piece_id=MTNLU22JHOYH6VSRMWLKQGSCDI5GIADPIXBVVSC2RIJLTHPMXTOA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788383218 (103h9m40.636990305s) [noise=true , address=104.36.82.42:4818 , piece_id=A6U7TDLTAZHE5IILHHCZKIHCQINHCJSWTBYNXJV7GLCWTIBNB2VA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788633554 (103h9m40.637240644s) [noise=true , address=79.127.219.51:4202 , piece_id=LJZ5A5HAHG4CGXT5J2WNSMXD7HCYSCBUEYGAGOES3NWIVV34BXEQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788735479 (103h9m40.637342568s) [noise=true , address=79.127.219.47:4040 , piece_id=BKWXBAVFXQ2RKG27WGPEAWG5UTHBVODW2BB4EGKHV6SN4EFI7KIQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789072444 (103h9m40.637679533s) [noise=true , address=77.90.31.15:4706 , piece_id=C3JOUPYB77PA2EHPSNPMDETERE724PBALBCSZ6MJCTZ6YRPUC2UA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789122202 (103h9m40.637729291s) [noise=true , address=104.36.82.9:4035 , piece_id=74BSWBLVWYQXPUI2PGV5NIJDSE6MSFKWK65TCVV6CNZKVQBQVNEA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789393961 (103h9m40.638001047s) [noise=true , address=45.91.116.10:4103 , piece_id=UGRDLKKFJRNCCDNDQTAY2LLOIS7CGI6KLNDZ7KJDOUNYWFNUAYQQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789591397 (103h9m40.638198479s) [noise=true , address=104.36.82.5:4870 , piece_id=AYTLB2OL7OAI5EO2ZE6LXIQLCGSGDXKDKT6V5HXIPAREK656MDPQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789716087 (103h9m40.638323176s) [noise=true , address=79.127.205.233:4645 , piece_id=6YAS5E7QYEZAJXB6HUY3R4PLHC4Z7EYTMSFWNFWD5AP7HKXOKTUQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789937996 (103h9m40.638545078s) [noise=true , address=104.36.82.53:4121 , piece_id=GJLXLW2A5RZUF4MBUP5AZAWFKTWGBDADIIWVIUBXPQPPGPKYZ4YA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.790288745 (103h9m40.638895835s) [noise=true , address=77.90.31.15:4708 , piece_id=67LX4NZJUKLYHRTSW3RKHHXZRKMGWJ2KV32FDA2KE3DY364CJ7OA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.790730467 (103h9m40.639337554s) [noise=true , address=45.91.116.14:4206 , piece_id=OVX63SL5OGJTT7HWDRIUV4BDVEKN2MD4ORUUK5M4JIEWX4J6VHFA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.791174576 (103h9m40.639781656s) [noise=true , address=45.91.116.13:4176 , piece_id=GOWXIBOORGZWFIBLMG46HQ4J632B3E6I3MVXNLTWPP2TY3IF2JUQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.898860128 (103h9m40.747467214s) [upload=82448]: Next returned nil segment
Jul 23 05:59:58.898881813 (103h9m40.747488906s) [upload=82448]: Waiting for error group managing segments...
Jul 23 05:59:58.905078785 (103h9m40.753685854s) [noise=true , address=45.91.116.14:4206 , piece_id=OVX63SL5OGJTT7HWDRIUV4BDVEKN2MD4ORUUK5M4JIEWX4J6VHFA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.906441982 (103h9m40.755049063s) [noise=true , address=77.90.31.15:4708 , piece_id=67LX4NZJUKLYHRTSW3RKHHXZRKMGWJ2KV32FDA2KE3DY364CJ7OA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.910115252 (103h9m40.758722337s) [noise=true , address=77.90.31.15:4706 , piece_id=C3JOUPYB77PA2EHPSNPMDETERE724PBALBCSZ6MJCTZ6YRPUC2UA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.910652695 (103h9m40.759259762s) [noise=true , address=104.36.82.9:4035 , piece_id=74BSWBLVWYQXPUI2PGV5NIJDSE6MSFKWK65TCVV6CNZKVQBQVNEA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.910672036 (103h9m40.759279117s) [noise=true , address=77.90.31.11:4569 , piece_id=MTNLU22JHOYH6VSRMWLKQGSCDI5GIADPIXBVVSC2RIJLTHPMXTOA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.911483080 (103h9m40.760090146s) [noise=true , address=45.91.116.13:4176 , piece_id=GOWXIBOORGZWFIBLMG46HQ4J632B3E6I3MVXNLTWPP2TY3IF2JUQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.911951970 (103h9m40.760559052s) [noise=true , address=45.91.116.10:4103 , piece_id=UGRDLKKFJRNCCDNDQTAY2LLOIS7CGI6KLNDZ7KJDOUNYWFNUAYQQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.912044937 (103h9m40.760652006s) [noise=true , address=77.90.31.12:4622 , piece_id=EZJAG75JRZTYILS7545KYAI37YOUTHHIOOSH26KIWMUQ4BCG556A, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.912649865 (103h9m40.761256931s) [noise=true , address=77.90.31.17:4799 , piece_id=RUTSDNZEXKJ5Y7WQBIVTMFUIJRYLL6DDNNJEXZAL5S7CIATWYHBQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.913487977 (103h9m40.76209505s) [noise=true , address=79.127.219.47:4040 , piece_id=BKWXBAVFXQ2RKG27WGPEAWG5UTHBVODW2BB4EGKHV6SN4EFI7KIQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.914486860 (103h9m40.763093927s) [noise=true , address=104.36.82.53:4121 , piece_id=GJLXLW2A5RZUF4MBUP5AZAWFKTWGBDADIIWVIUBXPQPPGPKYZ4YA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.918080770 (103h9m40.766687852s) [noise=true , address=104.36.82.5:4870 , piece_id=AYTLB2OL7OAI5EO2ZE6LXIQLCGSGDXKDKT6V5HXIPAREK656MDPQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.924012392 (103h9m40.772619461s) [noise=true , address=79.127.219.51:4202 , piece_id=LJZ5A5HAHG4CGXT5J2WNSMXD7HCYSCBUEYGAGOES3NWIVV34BXEQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.932892483 (103h9m40.78149955s) [noise=true , address=104.36.82.42:4818 , piece_id=A6U7TDLTAZHE5IILHHCZKIHCQINHCJSWTBYNXJV7GLCWTIBNB2VA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.932913099 (103h9m40.781520167s) [seg_pos={0 0}, upload=82448]: Segment reached optimal threshold of 14 pieces.
Jul 23 05:59:58.933104957 (103h9m40.781712037s) [noise=true , address=104.36.82.10:4105 , piece_id=GY72JPHTZOGEVFDG3FM5W5ZSYOPSY3BZXEME2Y37VVR2LR37SIUA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GhDrKMcDEg2RxamqKtiHphpgNdvtZYdHXrjzGsNJFjBFYeEen): protocol: expected piece hash; context canceled; EOF
Jul 23 05:59:58.933505265 (103h9m40.782112345s) [noise=true , address=79.127.205.233:4645 , piece_id=6YAS5E7QYEZAJXB6HUY3R4PLHC4Z7EYTMSFWNFWD5AP7HKXOKTUQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12NR52pweTqZNNMTyc1EHSbifDKB4awrnnb4megtZHwANSrR18T): protocol: read tcp 159.223.143.39:46566->79.127.205.233:4645: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 23 05:59:58.933568113 (103h9m40.782175194s) [noise=true , address=79.127.205.235:4680 , piece_id=T3V7PMZXJOHWT77MQ52PUMFNAIAX4DLMTIIVL6XNVTNA565O5WCQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GGvQQLoBcTniZST8Hz9h8eCLWbU4DrXn66DGy33iGLXaFgJLj): protocol: read tcp 159.223.143.39:45394->79.127.205.235:4680: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 23 05:59:58.933618915 (103h9m40.782225986s) [noise=true , address=79.127.205.246:4280 , piece_id=NPBN75MFQYLCG4XKRJPLHJU5JCYBNQYIFQ7MKKB3JKXLT4TKF7AA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rZDBvhrfNrYQB6h9MGhx1tNtQ7AH9ehUF4SfNMCJRpv4SKrDX): protocol: read tcp 159.223.143.39:45458->79.127.205.246:4280: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 23 05:59:58.933631191 (103h9m40.78223826s) [seg_pos={0 0}, upload=82448]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jul 23 05:59:58.933756265 (103h9m40.782363345s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 23 05:59:58.933762566 (103h9m40.782369645s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 23 05:59:59.025408726 (103h9m40.874015808s) [upload=82448]: Done uploading segments...
Jul 23 05:59:59.025427547 (103h9m40.874034627s) [upload=82448]: Upload finished. err: <nil>
Duration: 556.542359ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......] duration:
487.205555ms
FAIL (2025-07-23 06:07:23 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
23 06:00:19.742341858 (103h10m1.590948949s) [noise=true , address=104.36.82.4:4804 , piece_id=SMCQKI5GIWKEHYLNSVUGHWQNBHP3S4C5RBB4E3K5CFMXE54Q4WAQ, seg_pos={0 0}, upload=82455]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g): protocol: expected piece hash; context canceled; EOF
Jul 23 06:00:19.742959346 (103h10m1.59156642s) [noise=true , address=77.90.31.16:4734 , piece_id=NFCPPB6P25YMO5UZRM4CQKMNRV5Q2FXMSR6M4VSTEERBMVNLYJJA, seg_pos={0 0}, upload=82455]: Done uploading piece. err: <nil>
Jul 23 06:00:19.743002232 (103h10m1.591609306s) [seg_pos={0 0}, upload=82455]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 23 06:00:19.743286680 (103h10m1.591893764s) [upload=82455]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 23 06:00:19.743293769 (103h10m1.591900841s) [upload=82455]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 23 06:00:19.815641707 (103h10m1.664248798s) [upload=82455]: Done uploading segments...
Jul 23 06:00:19.815666887 (103h10m1.664273954s) [upload=82455]: Upload finished. err: <nil>
Jul 23 06:01:08.401614022 (103h10m50.250221093s) [upload=82463]: Starting upload
Jul 23 06:01:08.401709823 (103h10m50.250316908s) [upload=82463]: Done starting upload
Jul 23 06:01:08.402138436 (103h10m50.250745505s) [upload=82463]: Uploading segments...
Jul 23 06:01:08.402189044 (103h10m50.250796137s) [upload=82463]: Got next segment. Inline: false
Jul 23 06:01:08.402231416 (103h10m50.250838487s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 23 06:01:08.402237731 (103h10m50.250844804s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 23 06:01:08.450301042 (103h10m50.298908146s) [seg_pos={0 0}, upload=82463]: Begin upload segment...
Jul 23 06:01:08.450364720 (103h10m50.298971798s) [seg_pos={0 0}, upload=82463]: Done begin upload segment.
Jul 23 06:01:08.450425015 (103h10m50.2990321s) [noise=true , address=45.91.116.5:4299 , piece_id=K3URA4ZO4R2CAGZLTO6YJTZQOZ6YIOB7XOJ3AQAVGDFIMVU63BIA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.450431954 (103h10m50.299039031s) [noise=true , address=45.91.116.7:4010 , piece_id=WVMOM2PPTVXY7NYI5UH25PFW6RODIUJWN73YB6FW6UOAMNY4C4SA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.450900413 (103h10m50.299507502s) [noise=true , address=77.90.31.17:4772 , piece_id=63V72ATK3KI3ADRC4N5HTRIXSEB3DRM7N73MVZNDPZJ7EVO3RZJA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.450963993 (103h10m50.299571083s) [noise=true , address=45.91.116.8:4048 , piece_id=AROG4HVUX7R6MHG3DT3N2FIQJBBEVVEVKQHSAGZCWMISWTFSQMAQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.451375091 (103h10m50.299982178s) [noise=true , address=45.91.116.13:4155 , piece_id=KMDEG3QMJNHRF7TETZAFVN3TOZWTQNW7IRTCVI3WLNE7NJAAAPCA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.451400410 (103h10m50.300007509s) [noise=true , address=77.90.31.12:4592 , piece_id=NAWAKPN4LUPCFZIXDIEPN2GKO3CSGX35LZNHI4QDCGYBYAMWBGOA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.451837258 (103h10m50.300444352s) [noise=true , address=77.90.31.17:4771 , piece_id=VCHLVD4PCT6SGJNCFA4LSEDIXSRMX27JH7VLUXDBM2PMIGZ54CNA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.452344092 (103h10m50.300951186s) [noise=true , address=45.91.116.4:4937 , piece_id=DZILXASXCQNDQGTAIYQIY4LCXHENKF6LLWHME5ILDWL3IZOSWMCQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.452773889 (103h10m50.301380982s) [noise=true , address=45.91.116.11:4119 , piece_id=LOMOQU77BVHAPGZ23ZY6CG63FZFQX2IG5QRY2P46WKSGB5AF5O3Q, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.453168858 (103h10m50.301775945s) [noise=true , address=45.91.116.5:4298 , piece_id=AIURH3V65NZD3QI3A5YGBJ222ANC4LTHEYFWTOFIC6SPVH3O4CKA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.453559736 (103h10m50.302166813s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQJDPJRZQEJHTED4UJBNSJ2CGAL6FPTLLIIRFTOVBUK65VSPLGA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.460854171 (103h10m50.309461265s) [noise=true , address=77.90.31.14:4661 , piece_id=MA3CRNAREYTT6ZDO7Q7QRSAOVQQ3EM3CHZTWYQYE6QOWNMYQBBQQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.462111720 (103h10m50.310718803s) [noise=true , address=45.91.116.12:4225 , piece_id=MFM7FUANQ2WU4GK46NPKKKFDUXSVVQ3MGK7GM3QBR7GMOD563PTQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.462819102 (103h10m50.311426189s) [noise=true , address=104.36.82.13:4186 , piece_id=O5BVVJBIWINWK2GXWWJDETED6T3Z462AAUQWCG6LGC7PVVFSHZBA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.463600977 (103h10m50.312208063s) [noise=true , address=45.91.116.7:4011 , piece_id=GKVDRTZ2TY2ESEKU2FKIJND4LXDMITNWQORE4Z47OQ454UZMM7BA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.464356397 (103h10m50.312963485s) [noise=true , address=77.90.31.13:4626 , piece_id=D63E7JGO5RRTEJGXVUMNEC3UYXHS45HBOB7S67VPCWLHJXNH7OSQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.587088382 (103h10m50.435695454s) [upload=82463]: Next returned nil segment
Jul 23 06:01:08.587107670 (103h10m50.435714757s) [upload=82463]: Waiting for error group managing segments...
Jul 23 06:01:08.609523184 (103h10m50.458130271s) [noise=true , address=77.90.31.14:4661 , piece_id=MA3CRNAREYTT6ZDO7Q7QRSAOVQQ3EM3CHZTWYQYE6QOWNMYQBBQQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.612539671 (103h10m50.46114674s) [noise=true , address=45.91.116.5:4299 , piece_id=K3URA4ZO4R2CAGZLTO6YJTZQOZ6YIOB7XOJ3AQAVGDFIMVU63BIA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.617758941 (103h10m50.466366025s) [noise=true , address=45.91.116.13:4155 , piece_id=KMDEG3QMJNHRF7TETZAFVN3TOZWTQNW7IRTCVI3WLNE7NJAAAPCA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.618513849 (103h10m50.467120922s) [noise=true , address=45.91.116.8:4048 , piece_id=AROG4HVUX7R6MHG3DT3N2FIQJBBEVVEVKQHSAGZCWMISWTFSQMAQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.624184177 (103h10m50.47279126s) [noise=true , address=45.91.116.7:4010 , piece_id=WVMOM2PPTVXY7NYI5UH25PFW6RODIUJWN73YB6FW6UOAMNY4C4SA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.625290831 (103h10m50.473897916s) [noise=true , address=45.91.116.7:4011 , piece_id=GKVDRTZ2TY2ESEKU2FKIJND4LXDMITNWQORE4Z47OQ454UZMM7BA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.628897649 (103h10m50.477504746s) [noise=true , address=45.91.116.5:4298 , piece_id=AIURH3V65NZD3QI3A5YGBJ222ANC4LTHEYFWTOFIC6SPVH3O4CKA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.629542622 (103h10m50.478149697s) [noise=true , address=45.91.116.12:4225 , piece_id=MFM7FUANQ2WU4GK46NPKKKFDUXSVVQ3MGK7GM3QBR7GMOD563PTQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.630003986 (103h10m50.478611073s) [noise=true , address=77.90.31.17:4772 , piece_id=63V72ATK3KI3ADRC4N5HTRIXSEB3DRM7N73MVZNDPZJ7EVO3RZJA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.630441959 (103h10m50.479049047s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQJDPJRZQEJHTED4UJBNSJ2CGAL6FPTLLIIRFTOVBUK65VSPLGA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.630941221 (103h10m50.479548305s) [noise=true , address=77.90.31.12:4592 , piece_id=NAWAKPN4LUPCFZIXDIEPN2GKO3CSGX35LZNHI4QDCGYBYAMWBGOA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.631378970 (103h10m50.47998605s) [noise=true , address=77.90.31.13:4626 , piece_id=D63E7JGO5RRTEJGXVUMNEC3UYXHS45HBOB7S67VPCWLHJXNH7OSQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.631840658 (103h10m50.480447733s) [noise=true , address=77.90.31.17:4771 , piece_id=VCHLVD4PCT6SGJNCFA4LSEDIXSRMX27JH7VLUXDBM2PMIGZ54CNA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.631850523 (103h10m50.480457598s) [seg_pos={0 0}, upload=82463]: Segment reached optimal threshold of 13 pieces.
Jul 23 06:01:08.631982956 (103h10m50.480590031s) [noise=true , address=104.36.82.13:4186 , piece_id=O5BVVJBIWINWK2GXWWJDETED6T3Z462AAUQWCG6LGC7PVVFSHZBA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF
Jul 23 06:01:08.632087803 (103h10m50.480694892s) [noise=true , address=45.91.116.4:4937 , piece_id=DZILXASXCQNDQGTAIYQIY4LCXHENKF6LLWHME5ILDWL3IZOSWMCQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF
Jul 23 06:01:08.633101069 (103h10m50.481708157s) [noise=true , address=45.91.116.11:4119 , piece_id=LOMOQU77BVHAPGZ23ZY6CG63FZFQX2IG5QRY2P46WKSGB5AF5O3Q, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.633132335 (103h10m50.481739428s) [seg_pos={0 0}, upload=82463]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 23 06:01:08.633221956 (103h10m50.481829049s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 23 06:01:08.633272252 (103h10m50.481879341s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 23 06:01:08.710605841 (103h10m50.559212933s) [upload=82463]: Done uploading segments...
Jul 23 06:01:08.710632085 (103h10m50.559239157s) [upload=82463]: Upload finished. err: <nil>
Duration: 487.411171ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......] duration:
604.733411ms
FAIL (2025-07-23 06:06:57 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
s) [upload=82448]: Got next segment. Inline: false
Jul 23 05:59:58.732247463 (103h9m40.580854536s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 23 05:59:58.732253833 (103h9m40.580860918s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 23 05:59:58.787057645 (103h9m40.635664731s) [seg_pos={0 0}, upload=82448]: Begin upload segment...
Jul 23 05:59:58.787102519 (103h9m40.635709584s) [seg_pos={0 0}, upload=82448]: Done begin upload segment.
Jul 23 05:59:58.787136403 (103h9m40.635743482s) [noise=true , address=77.90.31.12:4622 , piece_id=EZJAG75JRZTYILS7545KYAI37YOUTHHIOOSH26KIWMUQ4BCG556A, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787169354 (103h9m40.635776438s) [noise=true , address=79.127.205.235:4680 , piece_id=T3V7PMZXJOHWT77MQ52PUMFNAIAX4DLMTIIVL6XNVTNA565O5WCQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787544550 (103h9m40.636151681s) [noise=true , address=104.36.82.10:4105 , piece_id=GY72JPHTZOGEVFDG3FM5W5ZSYOPSY3BZXEME2Y37VVR2LR37SIUA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787598825 (103h9m40.636205915s) [noise=true , address=79.127.205.246:4280 , piece_id=NPBN75MFQYLCG4XKRJPLHJU5JCYBNQYIFQ7MKKB3JKXLT4TKF7AA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787946084 (103h9m40.636553167s) [noise=true , address=77.90.31.17:4799 , piece_id=RUTSDNZEXKJ5Y7WQBIVTMFUIJRYLL6DDNNJEXZAL5S7CIATWYHBQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788041731 (103h9m40.636648823s) [noise=true , address=77.90.31.11:4569 , piece_id=MTNLU22JHOYH6VSRMWLKQGSCDI5GIADPIXBVVSC2RIJLTHPMXTOA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788383218 (103h9m40.636990305s) [noise=true , address=104.36.82.42:4818 , piece_id=A6U7TDLTAZHE5IILHHCZKIHCQINHCJSWTBYNXJV7GLCWTIBNB2VA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788633554 (103h9m40.637240644s) [noise=true , address=79.127.219.51:4202 , piece_id=LJZ5A5HAHG4CGXT5J2WNSMXD7HCYSCBUEYGAGOES3NWIVV34BXEQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788735479 (103h9m40.637342568s) [noise=true , address=79.127.219.47:4040 , piece_id=BKWXBAVFXQ2RKG27WGPEAWG5UTHBVODW2BB4EGKHV6SN4EFI7KIQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789072444 (103h9m40.637679533s) [noise=true , address=77.90.31.15:4706 , piece_id=C3JOUPYB77PA2EHPSNPMDETERE724PBALBCSZ6MJCTZ6YRPUC2UA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789122202 (103h9m40.637729291s) [noise=true , address=104.36.82.9:4035 , piece_id=74BSWBLVWYQXPUI2PGV5NIJDSE6MSFKWK65TCVV6CNZKVQBQVNEA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789393961 (103h9m40.638001047s) [noise=true , address=45.91.116.10:4103 , piece_id=UGRDLKKFJRNCCDNDQTAY2LLOIS7CGI6KLNDZ7KJDOUNYWFNUAYQQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789591397 (103h9m40.638198479s) [noise=true , address=104.36.82.5:4870 , piece_id=AYTLB2OL7OAI5EO2ZE6LXIQLCGSGDXKDKT6V5HXIPAREK656MDPQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789716087 (103h9m40.638323176s) [noise=true , address=79.127.205.233:4645 , piece_id=6YAS5E7QYEZAJXB6HUY3R4PLHC4Z7EYTMSFWNFWD5AP7HKXOKTUQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789937996 (103h9m40.638545078s) [noise=true , address=104.36.82.53:4121 , piece_id=GJLXLW2A5RZUF4MBUP5AZAWFKTWGBDADIIWVIUBXPQPPGPKYZ4YA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.790288745 (103h9m40.638895835s) [noise=true , address=77.90.31.15:4708 , piece_id=67LX4NZJUKLYHRTSW3RKHHXZRKMGWJ2KV32FDA2KE3DY364CJ7OA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.790730467 (103h9m40.639337554s) [noise=true , address=45.91.116.14:4206 , piece_id=OVX63SL5OGJTT7HWDRIUV4BDVEKN2MD4ORUUK5M4JIEWX4J6VHFA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.791174576 (103h9m40.639781656s) [noise=true , address=45.91.116.13:4176 , piece_id=GOWXIBOORGZWFIBLMG46HQ4J632B3E6I3MVXNLTWPP2TY3IF2JUQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.898860128 (103h9m40.747467214s) [upload=82448]: Next returned nil segment
Jul 23 05:59:58.898881813 (103h9m40.747488906s) [upload=82448]: Waiting for error group managing segments...
Jul 23 05:59:58.905078785 (103h9m40.753685854s) [noise=true , address=45.91.116.14:4206 , piece_id=OVX63SL5OGJTT7HWDRIUV4BDVEKN2MD4ORUUK5M4JIEWX4J6VHFA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.906441982 (103h9m40.755049063s) [noise=true , address=77.90.31.15:4708 , piece_id=67LX4NZJUKLYHRTSW3RKHHXZRKMGWJ2KV32FDA2KE3DY364CJ7OA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.910115252 (103h9m40.758722337s) [noise=true , address=77.90.31.15:4706 , piece_id=C3JOUPYB77PA2EHPSNPMDETERE724PBALBCSZ6MJCTZ6YRPUC2UA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.910652695 (103h9m40.759259762s) [noise=true , address=104.36.82.9:4035 , piece_id=74BSWBLVWYQXPUI2PGV5NIJDSE6MSFKWK65TCVV6CNZKVQBQVNEA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.910672036 (103h9m40.759279117s) [noise=true , address=77.90.31.11:4569 , piece_id=MTNLU22JHOYH6VSRMWLKQGSCDI5GIADPIXBVVSC2RIJLTHPMXTOA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.911483080 (103h9m40.760090146s) [noise=true , address=45.91.116.13:4176 , piece_id=GOWXIBOORGZWFIBLMG46HQ4J632B3E6I3MVXNLTWPP2TY3IF2JUQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.911951970 (103h9m40.760559052s) [noise=true , address=45.91.116.10:4103 , piece_id=UGRDLKKFJRNCCDNDQTAY2LLOIS7CGI6KLNDZ7KJDOUNYWFNUAYQQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.912044937 (103h9m40.760652006s) [noise=true , address=77.90.31.12:4622 , piece_id=EZJAG75JRZTYILS7545KYAI37YOUTHHIOOSH26KIWMUQ4BCG556A, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.912649865 (103h9m40.761256931s) [noise=true , address=77.90.31.17:4799 , piece_id=RUTSDNZEXKJ5Y7WQBIVTMFUIJRYLL6DDNNJEXZAL5S7CIATWYHBQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.913487977 (103h9m40.76209505s) [noise=true , address=79.127.219.47:4040 , piece_id=BKWXBAVFXQ2RKG27WGPEAWG5UTHBVODW2BB4EGKHV6SN4EFI7KIQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.914486860 (103h9m40.763093927s) [noise=true , address=104.36.82.53:4121 , piece_id=GJLXLW2A5RZUF4MBUP5AZAWFKTWGBDADIIWVIUBXPQPPGPKYZ4YA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.918080770 (103h9m40.766687852s) [noise=true , address=104.36.82.5:4870 , piece_id=AYTLB2OL7OAI5EO2ZE6LXIQLCGSGDXKDKT6V5HXIPAREK656MDPQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.924012392 (103h9m40.772619461s) [noise=true , address=79.127.219.51:4202 , piece_id=LJZ5A5HAHG4CGXT5J2WNSMXD7HCYSCBUEYGAGOES3NWIVV34BXEQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.932892483 (103h9m40.78149955s) [noise=true , address=104.36.82.42:4818 , piece_id=A6U7TDLTAZHE5IILHHCZKIHCQINHCJSWTBYNXJV7GLCWTIBNB2VA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.932913099 (103h9m40.781520167s) [seg_pos={0 0}, upload=82448]: Segment reached optimal threshold of 14 pieces.
Jul 23 05:59:58.933104957 (103h9m40.781712037s) [noise=true , address=104.36.82.10:4105 , piece_id=GY72JPHTZOGEVFDG3FM5W5ZSYOPSY3BZXEME2Y37VVR2LR37SIUA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GhDrKMcDEg2RxamqKtiHphpgNdvtZYdHXrjzGsNJFjBFYeEen): protocol: expected piece hash; context canceled; EOF
Jul 23 05:59:58.933505265 (103h9m40.782112345s) [noise=true , address=79.127.205.233:4645 , piece_id=6YAS5E7QYEZAJXB6HUY3R4PLHC4Z7EYTMSFWNFWD5AP7HKXOKTUQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12NR52pweTqZNNMTyc1EHSbifDKB4awrnnb4megtZHwANSrR18T): protocol: read tcp 159.223.143.39:46566->79.127.205.233:4645: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 23 05:59:58.933568113 (103h9m40.782175194s) [noise=true , address=79.127.205.235:4680 , piece_id=T3V7PMZXJOHWT77MQ52PUMFNAIAX4DLMTIIVL6XNVTNA565O5WCQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GGvQQLoBcTniZST8Hz9h8eCLWbU4DrXn66DGy33iGLXaFgJLj): protocol: read tcp 159.223.143.39:45394->79.127.205.235:4680: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 23 05:59:58.933618915 (103h9m40.782225986s) [noise=true , address=79.127.205.246:4280 , piece_id=NPBN75MFQYLCG4XKRJPLHJU5JCYBNQYIFQ7MKKB3JKXLT4TKF7AA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rZDBvhrfNrYQB6h9MGhx1tNtQ7AH9ehUF4SfNMCJRpv4SKrDX): protocol: read tcp 159.223.143.39:45458->79.127.205.246:4280: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 23 05:59:58.933631191 (103h9m40.78223826s) [seg_pos={0 0}, upload=82448]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jul 23 05:59:58.933756265 (103h9m40.782363345s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 23 05:59:58.933762566 (103h9m40.782369645s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 23 05:59:59.025408726 (103h9m40.874015808s) [upload=82448]: Done uploading segments...
Jul 23 05:59:59.025427547 (103h9m40.874034627s) [upload=82448]: Upload finished. err: <nil>
Duration: 604.766252ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......] duration:
588.38903ms
FAIL (2025-07-23 06:06:20 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
s) [upload=82448]: Got next segment. Inline: false
Jul 23 05:59:58.732247463 (103h9m40.580854536s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 23 05:59:58.732253833 (103h9m40.580860918s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 23 05:59:58.787057645 (103h9m40.635664731s) [seg_pos={0 0}, upload=82448]: Begin upload segment...
Jul 23 05:59:58.787102519 (103h9m40.635709584s) [seg_pos={0 0}, upload=82448]: Done begin upload segment.
Jul 23 05:59:58.787136403 (103h9m40.635743482s) [noise=true , address=77.90.31.12:4622 , piece_id=EZJAG75JRZTYILS7545KYAI37YOUTHHIOOSH26KIWMUQ4BCG556A, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787169354 (103h9m40.635776438s) [noise=true , address=79.127.205.235:4680 , piece_id=T3V7PMZXJOHWT77MQ52PUMFNAIAX4DLMTIIVL6XNVTNA565O5WCQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787544550 (103h9m40.636151681s) [noise=true , address=104.36.82.10:4105 , piece_id=GY72JPHTZOGEVFDG3FM5W5ZSYOPSY3BZXEME2Y37VVR2LR37SIUA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787598825 (103h9m40.636205915s) [noise=true , address=79.127.205.246:4280 , piece_id=NPBN75MFQYLCG4XKRJPLHJU5JCYBNQYIFQ7MKKB3JKXLT4TKF7AA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787946084 (103h9m40.636553167s) [noise=true , address=77.90.31.17:4799 , piece_id=RUTSDNZEXKJ5Y7WQBIVTMFUIJRYLL6DDNNJEXZAL5S7CIATWYHBQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788041731 (103h9m40.636648823s) [noise=true , address=77.90.31.11:4569 , piece_id=MTNLU22JHOYH6VSRMWLKQGSCDI5GIADPIXBVVSC2RIJLTHPMXTOA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788383218 (103h9m40.636990305s) [noise=true , address=104.36.82.42:4818 , piece_id=A6U7TDLTAZHE5IILHHCZKIHCQINHCJSWTBYNXJV7GLCWTIBNB2VA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788633554 (103h9m40.637240644s) [noise=true , address=79.127.219.51:4202 , piece_id=LJZ5A5HAHG4CGXT5J2WNSMXD7HCYSCBUEYGAGOES3NWIVV34BXEQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788735479 (103h9m40.637342568s) [noise=true , address=79.127.219.47:4040 , piece_id=BKWXBAVFXQ2RKG27WGPEAWG5UTHBVODW2BB4EGKHV6SN4EFI7KIQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789072444 (103h9m40.637679533s) [noise=true , address=77.90.31.15:4706 , piece_id=C3JOUPYB77PA2EHPSNPMDETERE724PBALBCSZ6MJCTZ6YRPUC2UA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789122202 (103h9m40.637729291s) [noise=true , address=104.36.82.9:4035 , piece_id=74BSWBLVWYQXPUI2PGV5NIJDSE6MSFKWK65TCVV6CNZKVQBQVNEA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789393961 (103h9m40.638001047s) [noise=true , address=45.91.116.10:4103 , piece_id=UGRDLKKFJRNCCDNDQTAY2LLOIS7CGI6KLNDZ7KJDOUNYWFNUAYQQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789591397 (103h9m40.638198479s) [noise=true , address=104.36.82.5:4870 , piece_id=AYTLB2OL7OAI5EO2ZE6LXIQLCGSGDXKDKT6V5HXIPAREK656MDPQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789716087 (103h9m40.638323176s) [noise=true , address=79.127.205.233:4645 , piece_id=6YAS5E7QYEZAJXB6HUY3R4PLHC4Z7EYTMSFWNFWD5AP7HKXOKTUQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789937996 (103h9m40.638545078s) [noise=true , address=104.36.82.53:4121 , piece_id=GJLXLW2A5RZUF4MBUP5AZAWFKTWGBDADIIWVIUBXPQPPGPKYZ4YA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.790288745 (103h9m40.638895835s) [noise=true , address=77.90.31.15:4708 , piece_id=67LX4NZJUKLYHRTSW3RKHHXZRKMGWJ2KV32FDA2KE3DY364CJ7OA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.790730467 (103h9m40.639337554s) [noise=true , address=45.91.116.14:4206 , piece_id=OVX63SL5OGJTT7HWDRIUV4BDVEKN2MD4ORUUK5M4JIEWX4J6VHFA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.791174576 (103h9m40.639781656s) [noise=true , address=45.91.116.13:4176 , piece_id=GOWXIBOORGZWFIBLMG46HQ4J632B3E6I3MVXNLTWPP2TY3IF2JUQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.898860128 (103h9m40.747467214s) [upload=82448]: Next returned nil segment
Jul 23 05:59:58.898881813 (103h9m40.747488906s) [upload=82448]: Waiting for error group managing segments...
Jul 23 05:59:58.905078785 (103h9m40.753685854s) [noise=true , address=45.91.116.14:4206 , piece_id=OVX63SL5OGJTT7HWDRIUV4BDVEKN2MD4ORUUK5M4JIEWX4J6VHFA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.906441982 (103h9m40.755049063s) [noise=true , address=77.90.31.15:4708 , piece_id=67LX4NZJUKLYHRTSW3RKHHXZRKMGWJ2KV32FDA2KE3DY364CJ7OA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.910115252 (103h9m40.758722337s) [noise=true , address=77.90.31.15:4706 , piece_id=C3JOUPYB77PA2EHPSNPMDETERE724PBALBCSZ6MJCTZ6YRPUC2UA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.910652695 (103h9m40.759259762s) [noise=true , address=104.36.82.9:4035 , piece_id=74BSWBLVWYQXPUI2PGV5NIJDSE6MSFKWK65TCVV6CNZKVQBQVNEA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.910672036 (103h9m40.759279117s) [noise=true , address=77.90.31.11:4569 , piece_id=MTNLU22JHOYH6VSRMWLKQGSCDI5GIADPIXBVVSC2RIJLTHPMXTOA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.911483080 (103h9m40.760090146s) [noise=true , address=45.91.116.13:4176 , piece_id=GOWXIBOORGZWFIBLMG46HQ4J632B3E6I3MVXNLTWPP2TY3IF2JUQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.911951970 (103h9m40.760559052s) [noise=true , address=45.91.116.10:4103 , piece_id=UGRDLKKFJRNCCDNDQTAY2LLOIS7CGI6KLNDZ7KJDOUNYWFNUAYQQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.912044937 (103h9m40.760652006s) [noise=true , address=77.90.31.12:4622 , piece_id=EZJAG75JRZTYILS7545KYAI37YOUTHHIOOSH26KIWMUQ4BCG556A, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.912649865 (103h9m40.761256931s) [noise=true , address=77.90.31.17:4799 , piece_id=RUTSDNZEXKJ5Y7WQBIVTMFUIJRYLL6DDNNJEXZAL5S7CIATWYHBQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.913487977 (103h9m40.76209505s) [noise=true , address=79.127.219.47:4040 , piece_id=BKWXBAVFXQ2RKG27WGPEAWG5UTHBVODW2BB4EGKHV6SN4EFI7KIQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.914486860 (103h9m40.763093927s) [noise=true , address=104.36.82.53:4121 , piece_id=GJLXLW2A5RZUF4MBUP5AZAWFKTWGBDADIIWVIUBXPQPPGPKYZ4YA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.918080770 (103h9m40.766687852s) [noise=true , address=104.36.82.5:4870 , piece_id=AYTLB2OL7OAI5EO2ZE6LXIQLCGSGDXKDKT6V5HXIPAREK656MDPQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.924012392 (103h9m40.772619461s) [noise=true , address=79.127.219.51:4202 , piece_id=LJZ5A5HAHG4CGXT5J2WNSMXD7HCYSCBUEYGAGOES3NWIVV34BXEQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.932892483 (103h9m40.78149955s) [noise=true , address=104.36.82.42:4818 , piece_id=A6U7TDLTAZHE5IILHHCZKIHCQINHCJSWTBYNXJV7GLCWTIBNB2VA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.932913099 (103h9m40.781520167s) [seg_pos={0 0}, upload=82448]: Segment reached optimal threshold of 14 pieces.
Jul 23 05:59:58.933104957 (103h9m40.781712037s) [noise=true , address=104.36.82.10:4105 , piece_id=GY72JPHTZOGEVFDG3FM5W5ZSYOPSY3BZXEME2Y37VVR2LR37SIUA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GhDrKMcDEg2RxamqKtiHphpgNdvtZYdHXrjzGsNJFjBFYeEen): protocol: expected piece hash; context canceled; EOF
Jul 23 05:59:58.933505265 (103h9m40.782112345s) [noise=true , address=79.127.205.233:4645 , piece_id=6YAS5E7QYEZAJXB6HUY3R4PLHC4Z7EYTMSFWNFWD5AP7HKXOKTUQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12NR52pweTqZNNMTyc1EHSbifDKB4awrnnb4megtZHwANSrR18T): protocol: read tcp 159.223.143.39:46566->79.127.205.233:4645: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 23 05:59:58.933568113 (103h9m40.782175194s) [noise=true , address=79.127.205.235:4680 , piece_id=T3V7PMZXJOHWT77MQ52PUMFNAIAX4DLMTIIVL6XNVTNA565O5WCQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GGvQQLoBcTniZST8Hz9h8eCLWbU4DrXn66DGy33iGLXaFgJLj): protocol: read tcp 159.223.143.39:45394->79.127.205.235:4680: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 23 05:59:58.933618915 (103h9m40.782225986s) [noise=true , address=79.127.205.246:4280 , piece_id=NPBN75MFQYLCG4XKRJPLHJU5JCYBNQYIFQ7MKKB3JKXLT4TKF7AA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rZDBvhrfNrYQB6h9MGhx1tNtQ7AH9ehUF4SfNMCJRpv4SKrDX): protocol: read tcp 159.223.143.39:45458->79.127.205.246:4280: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 23 05:59:58.933631191 (103h9m40.78223826s) [seg_pos={0 0}, upload=82448]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jul 23 05:59:58.933756265 (103h9m40.782363345s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 23 05:59:58.933762566 (103h9m40.782369645s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 23 05:59:59.025408726 (103h9m40.874015808s) [upload=82448]: Done uploading segments...
Jul 23 05:59:59.025427547 (103h9m40.874034627s) [upload=82448]: Upload finished. err: <nil>
Duration: 588.423001ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....] duration:
876.741426ms
FAIL (2025-07-23 06:06:13 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
23 06:00:19.742341858 (103h10m1.590948949s) [noise=true , address=104.36.82.4:4804 , piece_id=SMCQKI5GIWKEHYLNSVUGHWQNBHP3S4C5RBB4E3K5CFMXE54Q4WAQ, seg_pos={0 0}, upload=82455]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g): protocol: expected piece hash; context canceled; EOF
Jul 23 06:00:19.742959346 (103h10m1.59156642s) [noise=true , address=77.90.31.16:4734 , piece_id=NFCPPB6P25YMO5UZRM4CQKMNRV5Q2FXMSR6M4VSTEERBMVNLYJJA, seg_pos={0 0}, upload=82455]: Done uploading piece. err: <nil>
Jul 23 06:00:19.743002232 (103h10m1.591609306s) [seg_pos={0 0}, upload=82455]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 23 06:00:19.743286680 (103h10m1.591893764s) [upload=82455]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 23 06:00:19.743293769 (103h10m1.591900841s) [upload=82455]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 23 06:00:19.815641707 (103h10m1.664248798s) [upload=82455]: Done uploading segments...
Jul 23 06:00:19.815666887 (103h10m1.664273954s) [upload=82455]: Upload finished. err: <nil>
Jul 23 06:01:08.401614022 (103h10m50.250221093s) [upload=82463]: Starting upload
Jul 23 06:01:08.401709823 (103h10m50.250316908s) [upload=82463]: Done starting upload
Jul 23 06:01:08.402138436 (103h10m50.250745505s) [upload=82463]: Uploading segments...
Jul 23 06:01:08.402189044 (103h10m50.250796137s) [upload=82463]: Got next segment. Inline: false
Jul 23 06:01:08.402231416 (103h10m50.250838487s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 23 06:01:08.402237731 (103h10m50.250844804s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 23 06:01:08.450301042 (103h10m50.298908146s) [seg_pos={0 0}, upload=82463]: Begin upload segment...
Jul 23 06:01:08.450364720 (103h10m50.298971798s) [seg_pos={0 0}, upload=82463]: Done begin upload segment.
Jul 23 06:01:08.450425015 (103h10m50.2990321s) [noise=true , address=45.91.116.5:4299 , piece_id=K3URA4ZO4R2CAGZLTO6YJTZQOZ6YIOB7XOJ3AQAVGDFIMVU63BIA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.450431954 (103h10m50.299039031s) [noise=true , address=45.91.116.7:4010 , piece_id=WVMOM2PPTVXY7NYI5UH25PFW6RODIUJWN73YB6FW6UOAMNY4C4SA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.450900413 (103h10m50.299507502s) [noise=true , address=77.90.31.17:4772 , piece_id=63V72ATK3KI3ADRC4N5HTRIXSEB3DRM7N73MVZNDPZJ7EVO3RZJA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.450963993 (103h10m50.299571083s) [noise=true , address=45.91.116.8:4048 , piece_id=AROG4HVUX7R6MHG3DT3N2FIQJBBEVVEVKQHSAGZCWMISWTFSQMAQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.451375091 (103h10m50.299982178s) [noise=true , address=45.91.116.13:4155 , piece_id=KMDEG3QMJNHRF7TETZAFVN3TOZWTQNW7IRTCVI3WLNE7NJAAAPCA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.451400410 (103h10m50.300007509s) [noise=true , address=77.90.31.12:4592 , piece_id=NAWAKPN4LUPCFZIXDIEPN2GKO3CSGX35LZNHI4QDCGYBYAMWBGOA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.451837258 (103h10m50.300444352s) [noise=true , address=77.90.31.17:4771 , piece_id=VCHLVD4PCT6SGJNCFA4LSEDIXSRMX27JH7VLUXDBM2PMIGZ54CNA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.452344092 (103h10m50.300951186s) [noise=true , address=45.91.116.4:4937 , piece_id=DZILXASXCQNDQGTAIYQIY4LCXHENKF6LLWHME5ILDWL3IZOSWMCQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.452773889 (103h10m50.301380982s) [noise=true , address=45.91.116.11:4119 , piece_id=LOMOQU77BVHAPGZ23ZY6CG63FZFQX2IG5QRY2P46WKSGB5AF5O3Q, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.453168858 (103h10m50.301775945s) [noise=true , address=45.91.116.5:4298 , piece_id=AIURH3V65NZD3QI3A5YGBJ222ANC4LTHEYFWTOFIC6SPVH3O4CKA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.453559736 (103h10m50.302166813s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQJDPJRZQEJHTED4UJBNSJ2CGAL6FPTLLIIRFTOVBUK65VSPLGA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.460854171 (103h10m50.309461265s) [noise=true , address=77.90.31.14:4661 , piece_id=MA3CRNAREYTT6ZDO7Q7QRSAOVQQ3EM3CHZTWYQYE6QOWNMYQBBQQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.462111720 (103h10m50.310718803s) [noise=true , address=45.91.116.12:4225 , piece_id=MFM7FUANQ2WU4GK46NPKKKFDUXSVVQ3MGK7GM3QBR7GMOD563PTQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.462819102 (103h10m50.311426189s) [noise=true , address=104.36.82.13:4186 , piece_id=O5BVVJBIWINWK2GXWWJDETED6T3Z462AAUQWCG6LGC7PVVFSHZBA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.463600977 (103h10m50.312208063s) [noise=true , address=45.91.116.7:4011 , piece_id=GKVDRTZ2TY2ESEKU2FKIJND4LXDMITNWQORE4Z47OQ454UZMM7BA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.464356397 (103h10m50.312963485s) [noise=true , address=77.90.31.13:4626 , piece_id=D63E7JGO5RRTEJGXVUMNEC3UYXHS45HBOB7S67VPCWLHJXNH7OSQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.587088382 (103h10m50.435695454s) [upload=82463]: Next returned nil segment
Jul 23 06:01:08.587107670 (103h10m50.435714757s) [upload=82463]: Waiting for error group managing segments...
Jul 23 06:01:08.609523184 (103h10m50.458130271s) [noise=true , address=77.90.31.14:4661 , piece_id=MA3CRNAREYTT6ZDO7Q7QRSAOVQQ3EM3CHZTWYQYE6QOWNMYQBBQQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.612539671 (103h10m50.46114674s) [noise=true , address=45.91.116.5:4299 , piece_id=K3URA4ZO4R2CAGZLTO6YJTZQOZ6YIOB7XOJ3AQAVGDFIMVU63BIA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.617758941 (103h10m50.466366025s) [noise=true , address=45.91.116.13:4155 , piece_id=KMDEG3QMJNHRF7TETZAFVN3TOZWTQNW7IRTCVI3WLNE7NJAAAPCA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.618513849 (103h10m50.467120922s) [noise=true , address=45.91.116.8:4048 , piece_id=AROG4HVUX7R6MHG3DT3N2FIQJBBEVVEVKQHSAGZCWMISWTFSQMAQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.624184177 (103h10m50.47279126s) [noise=true , address=45.91.116.7:4010 , piece_id=WVMOM2PPTVXY7NYI5UH25PFW6RODIUJWN73YB6FW6UOAMNY4C4SA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.625290831 (103h10m50.473897916s) [noise=true , address=45.91.116.7:4011 , piece_id=GKVDRTZ2TY2ESEKU2FKIJND4LXDMITNWQORE4Z47OQ454UZMM7BA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.628897649 (103h10m50.477504746s) [noise=true , address=45.91.116.5:4298 , piece_id=AIURH3V65NZD3QI3A5YGBJ222ANC4LTHEYFWTOFIC6SPVH3O4CKA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.629542622 (103h10m50.478149697s) [noise=true , address=45.91.116.12:4225 , piece_id=MFM7FUANQ2WU4GK46NPKKKFDUXSVVQ3MGK7GM3QBR7GMOD563PTQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.630003986 (103h10m50.478611073s) [noise=true , address=77.90.31.17:4772 , piece_id=63V72ATK3KI3ADRC4N5HTRIXSEB3DRM7N73MVZNDPZJ7EVO3RZJA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.630441959 (103h10m50.479049047s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQJDPJRZQEJHTED4UJBNSJ2CGAL6FPTLLIIRFTOVBUK65VSPLGA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.630941221 (103h10m50.479548305s) [noise=true , address=77.90.31.12:4592 , piece_id=NAWAKPN4LUPCFZIXDIEPN2GKO3CSGX35LZNHI4QDCGYBYAMWBGOA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.631378970 (103h10m50.47998605s) [noise=true , address=77.90.31.13:4626 , piece_id=D63E7JGO5RRTEJGXVUMNEC3UYXHS45HBOB7S67VPCWLHJXNH7OSQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.631840658 (103h10m50.480447733s) [noise=true , address=77.90.31.17:4771 , piece_id=VCHLVD4PCT6SGJNCFA4LSEDIXSRMX27JH7VLUXDBM2PMIGZ54CNA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.631850523 (103h10m50.480457598s) [seg_pos={0 0}, upload=82463]: Segment reached optimal threshold of 13 pieces.
Jul 23 06:01:08.631982956 (103h10m50.480590031s) [noise=true , address=104.36.82.13:4186 , piece_id=O5BVVJBIWINWK2GXWWJDETED6T3Z462AAUQWCG6LGC7PVVFSHZBA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF
Jul 23 06:01:08.632087803 (103h10m50.480694892s) [noise=true , address=45.91.116.4:4937 , piece_id=DZILXASXCQNDQGTAIYQIY4LCXHENKF6LLWHME5ILDWL3IZOSWMCQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF
Jul 23 06:01:08.633101069 (103h10m50.481708157s) [noise=true , address=45.91.116.11:4119 , piece_id=LOMOQU77BVHAPGZ23ZY6CG63FZFQX2IG5QRY2P46WKSGB5AF5O3Q, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.633132335 (103h10m50.481739428s) [seg_pos={0 0}, upload=82463]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 23 06:01:08.633221956 (103h10m50.481829049s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 23 06:01:08.633272252 (103h10m50.481879341s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 23 06:01:08.710605841 (103h10m50.559212933s) [upload=82463]: Done uploading segments...
Jul 23 06:01:08.710632085 (103h10m50.559239157s) [upload=82463]: Upload finished. err: <nil>
Duration: 876.770634ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....] duration:
609.477826ms
FAIL (2025-07-23 06:05:40 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
s) [upload=82448]: Got next segment. Inline: false
Jul 23 05:59:58.732247463 (103h9m40.580854536s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 23 05:59:58.732253833 (103h9m40.580860918s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 23 05:59:58.787057645 (103h9m40.635664731s) [seg_pos={0 0}, upload=82448]: Begin upload segment...
Jul 23 05:59:58.787102519 (103h9m40.635709584s) [seg_pos={0 0}, upload=82448]: Done begin upload segment.
Jul 23 05:59:58.787136403 (103h9m40.635743482s) [noise=true , address=77.90.31.12:4622 , piece_id=EZJAG75JRZTYILS7545KYAI37YOUTHHIOOSH26KIWMUQ4BCG556A, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787169354 (103h9m40.635776438s) [noise=true , address=79.127.205.235:4680 , piece_id=T3V7PMZXJOHWT77MQ52PUMFNAIAX4DLMTIIVL6XNVTNA565O5WCQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787544550 (103h9m40.636151681s) [noise=true , address=104.36.82.10:4105 , piece_id=GY72JPHTZOGEVFDG3FM5W5ZSYOPSY3BZXEME2Y37VVR2LR37SIUA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787598825 (103h9m40.636205915s) [noise=true , address=79.127.205.246:4280 , piece_id=NPBN75MFQYLCG4XKRJPLHJU5JCYBNQYIFQ7MKKB3JKXLT4TKF7AA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.787946084 (103h9m40.636553167s) [noise=true , address=77.90.31.17:4799 , piece_id=RUTSDNZEXKJ5Y7WQBIVTMFUIJRYLL6DDNNJEXZAL5S7CIATWYHBQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788041731 (103h9m40.636648823s) [noise=true , address=77.90.31.11:4569 , piece_id=MTNLU22JHOYH6VSRMWLKQGSCDI5GIADPIXBVVSC2RIJLTHPMXTOA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788383218 (103h9m40.636990305s) [noise=true , address=104.36.82.42:4818 , piece_id=A6U7TDLTAZHE5IILHHCZKIHCQINHCJSWTBYNXJV7GLCWTIBNB2VA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788633554 (103h9m40.637240644s) [noise=true , address=79.127.219.51:4202 , piece_id=LJZ5A5HAHG4CGXT5J2WNSMXD7HCYSCBUEYGAGOES3NWIVV34BXEQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.788735479 (103h9m40.637342568s) [noise=true , address=79.127.219.47:4040 , piece_id=BKWXBAVFXQ2RKG27WGPEAWG5UTHBVODW2BB4EGKHV6SN4EFI7KIQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789072444 (103h9m40.637679533s) [noise=true , address=77.90.31.15:4706 , piece_id=C3JOUPYB77PA2EHPSNPMDETERE724PBALBCSZ6MJCTZ6YRPUC2UA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789122202 (103h9m40.637729291s) [noise=true , address=104.36.82.9:4035 , piece_id=74BSWBLVWYQXPUI2PGV5NIJDSE6MSFKWK65TCVV6CNZKVQBQVNEA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789393961 (103h9m40.638001047s) [noise=true , address=45.91.116.10:4103 , piece_id=UGRDLKKFJRNCCDNDQTAY2LLOIS7CGI6KLNDZ7KJDOUNYWFNUAYQQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789591397 (103h9m40.638198479s) [noise=true , address=104.36.82.5:4870 , piece_id=AYTLB2OL7OAI5EO2ZE6LXIQLCGSGDXKDKT6V5HXIPAREK656MDPQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789716087 (103h9m40.638323176s) [noise=true , address=79.127.205.233:4645 , piece_id=6YAS5E7QYEZAJXB6HUY3R4PLHC4Z7EYTMSFWNFWD5AP7HKXOKTUQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.789937996 (103h9m40.638545078s) [noise=true , address=104.36.82.53:4121 , piece_id=GJLXLW2A5RZUF4MBUP5AZAWFKTWGBDADIIWVIUBXPQPPGPKYZ4YA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.790288745 (103h9m40.638895835s) [noise=true , address=77.90.31.15:4708 , piece_id=67LX4NZJUKLYHRTSW3RKHHXZRKMGWJ2KV32FDA2KE3DY364CJ7OA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.790730467 (103h9m40.639337554s) [noise=true , address=45.91.116.14:4206 , piece_id=OVX63SL5OGJTT7HWDRIUV4BDVEKN2MD4ORUUK5M4JIEWX4J6VHFA, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.791174576 (103h9m40.639781656s) [noise=true , address=45.91.116.13:4176 , piece_id=GOWXIBOORGZWFIBLMG46HQ4J632B3E6I3MVXNLTWPP2TY3IF2JUQ, seg_pos={0 0}, upload=82448]: Uploading piece...
Jul 23 05:59:58.898860128 (103h9m40.747467214s) [upload=82448]: Next returned nil segment
Jul 23 05:59:58.898881813 (103h9m40.747488906s) [upload=82448]: Waiting for error group managing segments...
Jul 23 05:59:58.905078785 (103h9m40.753685854s) [noise=true , address=45.91.116.14:4206 , piece_id=OVX63SL5OGJTT7HWDRIUV4BDVEKN2MD4ORUUK5M4JIEWX4J6VHFA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.906441982 (103h9m40.755049063s) [noise=true , address=77.90.31.15:4708 , piece_id=67LX4NZJUKLYHRTSW3RKHHXZRKMGWJ2KV32FDA2KE3DY364CJ7OA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.910115252 (103h9m40.758722337s) [noise=true , address=77.90.31.15:4706 , piece_id=C3JOUPYB77PA2EHPSNPMDETERE724PBALBCSZ6MJCTZ6YRPUC2UA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.910652695 (103h9m40.759259762s) [noise=true , address=104.36.82.9:4035 , piece_id=74BSWBLVWYQXPUI2PGV5NIJDSE6MSFKWK65TCVV6CNZKVQBQVNEA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.910672036 (103h9m40.759279117s) [noise=true , address=77.90.31.11:4569 , piece_id=MTNLU22JHOYH6VSRMWLKQGSCDI5GIADPIXBVVSC2RIJLTHPMXTOA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.911483080 (103h9m40.760090146s) [noise=true , address=45.91.116.13:4176 , piece_id=GOWXIBOORGZWFIBLMG46HQ4J632B3E6I3MVXNLTWPP2TY3IF2JUQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.911951970 (103h9m40.760559052s) [noise=true , address=45.91.116.10:4103 , piece_id=UGRDLKKFJRNCCDNDQTAY2LLOIS7CGI6KLNDZ7KJDOUNYWFNUAYQQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.912044937 (103h9m40.760652006s) [noise=true , address=77.90.31.12:4622 , piece_id=EZJAG75JRZTYILS7545KYAI37YOUTHHIOOSH26KIWMUQ4BCG556A, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.912649865 (103h9m40.761256931s) [noise=true , address=77.90.31.17:4799 , piece_id=RUTSDNZEXKJ5Y7WQBIVTMFUIJRYLL6DDNNJEXZAL5S7CIATWYHBQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.913487977 (103h9m40.76209505s) [noise=true , address=79.127.219.47:4040 , piece_id=BKWXBAVFXQ2RKG27WGPEAWG5UTHBVODW2BB4EGKHV6SN4EFI7KIQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.914486860 (103h9m40.763093927s) [noise=true , address=104.36.82.53:4121 , piece_id=GJLXLW2A5RZUF4MBUP5AZAWFKTWGBDADIIWVIUBXPQPPGPKYZ4YA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.918080770 (103h9m40.766687852s) [noise=true , address=104.36.82.5:4870 , piece_id=AYTLB2OL7OAI5EO2ZE6LXIQLCGSGDXKDKT6V5HXIPAREK656MDPQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.924012392 (103h9m40.772619461s) [noise=true , address=79.127.219.51:4202 , piece_id=LJZ5A5HAHG4CGXT5J2WNSMXD7HCYSCBUEYGAGOES3NWIVV34BXEQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.932892483 (103h9m40.78149955s) [noise=true , address=104.36.82.42:4818 , piece_id=A6U7TDLTAZHE5IILHHCZKIHCQINHCJSWTBYNXJV7GLCWTIBNB2VA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: <nil>
Jul 23 05:59:58.932913099 (103h9m40.781520167s) [seg_pos={0 0}, upload=82448]: Segment reached optimal threshold of 14 pieces.
Jul 23 05:59:58.933104957 (103h9m40.781712037s) [noise=true , address=104.36.82.10:4105 , piece_id=GY72JPHTZOGEVFDG3FM5W5ZSYOPSY3BZXEME2Y37VVR2LR37SIUA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GhDrKMcDEg2RxamqKtiHphpgNdvtZYdHXrjzGsNJFjBFYeEen): protocol: expected piece hash; context canceled; EOF
Jul 23 05:59:58.933505265 (103h9m40.782112345s) [noise=true , address=79.127.205.233:4645 , piece_id=6YAS5E7QYEZAJXB6HUY3R4PLHC4Z7EYTMSFWNFWD5AP7HKXOKTUQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12NR52pweTqZNNMTyc1EHSbifDKB4awrnnb4megtZHwANSrR18T): protocol: read tcp 159.223.143.39:46566->79.127.205.233:4645: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 23 05:59:58.933568113 (103h9m40.782175194s) [noise=true , address=79.127.205.235:4680 , piece_id=T3V7PMZXJOHWT77MQ52PUMFNAIAX4DLMTIIVL6XNVTNA565O5WCQ, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GGvQQLoBcTniZST8Hz9h8eCLWbU4DrXn66DGy33iGLXaFgJLj): protocol: read tcp 159.223.143.39:45394->79.127.205.235:4680: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 23 05:59:58.933618915 (103h9m40.782225986s) [noise=true , address=79.127.205.246:4280 , piece_id=NPBN75MFQYLCG4XKRJPLHJU5JCYBNQYIFQ7MKKB3JKXLT4TKF7AA, seg_pos={0 0}, upload=82448]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rZDBvhrfNrYQB6h9MGhx1tNtQ7AH9ehUF4SfNMCJRpv4SKrDX): protocol: read tcp 159.223.143.39:45458->79.127.205.246:4280: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jul 23 05:59:58.933631191 (103h9m40.78223826s) [seg_pos={0 0}, upload=82448]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jul 23 05:59:58.933756265 (103h9m40.782363345s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 23 05:59:58.933762566 (103h9m40.782369645s) [upload=82448]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 23 05:59:59.025408726 (103h9m40.874015808s) [upload=82448]: Done uploading segments...
Jul 23 05:59:59.025427547 (103h9m40.874034627s) [upload=82448]: Upload finished. err: <nil>
Duration: 609.749491ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....] duration:
892.607058ms
FAIL (2025-07-23 06:05:08 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
23 06:00:19.742341858 (103h10m1.590948949s) [noise=true , address=104.36.82.4:4804 , piece_id=SMCQKI5GIWKEHYLNSVUGHWQNBHP3S4C5RBB4E3K5CFMXE54Q4WAQ, seg_pos={0 0}, upload=82455]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g): protocol: expected piece hash; context canceled; EOF
Jul 23 06:00:19.742959346 (103h10m1.59156642s) [noise=true , address=77.90.31.16:4734 , piece_id=NFCPPB6P25YMO5UZRM4CQKMNRV5Q2FXMSR6M4VSTEERBMVNLYJJA, seg_pos={0 0}, upload=82455]: Done uploading piece. err: <nil>
Jul 23 06:00:19.743002232 (103h10m1.591609306s) [seg_pos={0 0}, upload=82455]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 23 06:00:19.743286680 (103h10m1.591893764s) [upload=82455]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 23 06:00:19.743293769 (103h10m1.591900841s) [upload=82455]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 23 06:00:19.815641707 (103h10m1.664248798s) [upload=82455]: Done uploading segments...
Jul 23 06:00:19.815666887 (103h10m1.664273954s) [upload=82455]: Upload finished. err: <nil>
Jul 23 06:01:08.401614022 (103h10m50.250221093s) [upload=82463]: Starting upload
Jul 23 06:01:08.401709823 (103h10m50.250316908s) [upload=82463]: Done starting upload
Jul 23 06:01:08.402138436 (103h10m50.250745505s) [upload=82463]: Uploading segments...
Jul 23 06:01:08.402189044 (103h10m50.250796137s) [upload=82463]: Got next segment. Inline: false
Jul 23 06:01:08.402231416 (103h10m50.250838487s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 23 06:01:08.402237731 (103h10m50.250844804s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 23 06:01:08.450301042 (103h10m50.298908146s) [seg_pos={0 0}, upload=82463]: Begin upload segment...
Jul 23 06:01:08.450364720 (103h10m50.298971798s) [seg_pos={0 0}, upload=82463]: Done begin upload segment.
Jul 23 06:01:08.450425015 (103h10m50.2990321s) [noise=true , address=45.91.116.5:4299 , piece_id=K3URA4ZO4R2CAGZLTO6YJTZQOZ6YIOB7XOJ3AQAVGDFIMVU63BIA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.450431954 (103h10m50.299039031s) [noise=true , address=45.91.116.7:4010 , piece_id=WVMOM2PPTVXY7NYI5UH25PFW6RODIUJWN73YB6FW6UOAMNY4C4SA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.450900413 (103h10m50.299507502s) [noise=true , address=77.90.31.17:4772 , piece_id=63V72ATK3KI3ADRC4N5HTRIXSEB3DRM7N73MVZNDPZJ7EVO3RZJA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.450963993 (103h10m50.299571083s) [noise=true , address=45.91.116.8:4048 , piece_id=AROG4HVUX7R6MHG3DT3N2FIQJBBEVVEVKQHSAGZCWMISWTFSQMAQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.451375091 (103h10m50.299982178s) [noise=true , address=45.91.116.13:4155 , piece_id=KMDEG3QMJNHRF7TETZAFVN3TOZWTQNW7IRTCVI3WLNE7NJAAAPCA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.451400410 (103h10m50.300007509s) [noise=true , address=77.90.31.12:4592 , piece_id=NAWAKPN4LUPCFZIXDIEPN2GKO3CSGX35LZNHI4QDCGYBYAMWBGOA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.451837258 (103h10m50.300444352s) [noise=true , address=77.90.31.17:4771 , piece_id=VCHLVD4PCT6SGJNCFA4LSEDIXSRMX27JH7VLUXDBM2PMIGZ54CNA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.452344092 (103h10m50.300951186s) [noise=true , address=45.91.116.4:4937 , piece_id=DZILXASXCQNDQGTAIYQIY4LCXHENKF6LLWHME5ILDWL3IZOSWMCQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.452773889 (103h10m50.301380982s) [noise=true , address=45.91.116.11:4119 , piece_id=LOMOQU77BVHAPGZ23ZY6CG63FZFQX2IG5QRY2P46WKSGB5AF5O3Q, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.453168858 (103h10m50.301775945s) [noise=true , address=45.91.116.5:4298 , piece_id=AIURH3V65NZD3QI3A5YGBJ222ANC4LTHEYFWTOFIC6SPVH3O4CKA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.453559736 (103h10m50.302166813s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQJDPJRZQEJHTED4UJBNSJ2CGAL6FPTLLIIRFTOVBUK65VSPLGA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.460854171 (103h10m50.309461265s) [noise=true , address=77.90.31.14:4661 , piece_id=MA3CRNAREYTT6ZDO7Q7QRSAOVQQ3EM3CHZTWYQYE6QOWNMYQBBQQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.462111720 (103h10m50.310718803s) [noise=true , address=45.91.116.12:4225 , piece_id=MFM7FUANQ2WU4GK46NPKKKFDUXSVVQ3MGK7GM3QBR7GMOD563PTQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.462819102 (103h10m50.311426189s) [noise=true , address=104.36.82.13:4186 , piece_id=O5BVVJBIWINWK2GXWWJDETED6T3Z462AAUQWCG6LGC7PVVFSHZBA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.463600977 (103h10m50.312208063s) [noise=true , address=45.91.116.7:4011 , piece_id=GKVDRTZ2TY2ESEKU2FKIJND4LXDMITNWQORE4Z47OQ454UZMM7BA, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.464356397 (103h10m50.312963485s) [noise=true , address=77.90.31.13:4626 , piece_id=D63E7JGO5RRTEJGXVUMNEC3UYXHS45HBOB7S67VPCWLHJXNH7OSQ, seg_pos={0 0}, upload=82463]: Uploading piece...
Jul 23 06:01:08.587088382 (103h10m50.435695454s) [upload=82463]: Next returned nil segment
Jul 23 06:01:08.587107670 (103h10m50.435714757s) [upload=82463]: Waiting for error group managing segments...
Jul 23 06:01:08.609523184 (103h10m50.458130271s) [noise=true , address=77.90.31.14:4661 , piece_id=MA3CRNAREYTT6ZDO7Q7QRSAOVQQ3EM3CHZTWYQYE6QOWNMYQBBQQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.612539671 (103h10m50.46114674s) [noise=true , address=45.91.116.5:4299 , piece_id=K3URA4ZO4R2CAGZLTO6YJTZQOZ6YIOB7XOJ3AQAVGDFIMVU63BIA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.617758941 (103h10m50.466366025s) [noise=true , address=45.91.116.13:4155 , piece_id=KMDEG3QMJNHRF7TETZAFVN3TOZWTQNW7IRTCVI3WLNE7NJAAAPCA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.618513849 (103h10m50.467120922s) [noise=true , address=45.91.116.8:4048 , piece_id=AROG4HVUX7R6MHG3DT3N2FIQJBBEVVEVKQHSAGZCWMISWTFSQMAQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.624184177 (103h10m50.47279126s) [noise=true , address=45.91.116.7:4010 , piece_id=WVMOM2PPTVXY7NYI5UH25PFW6RODIUJWN73YB6FW6UOAMNY4C4SA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.625290831 (103h10m50.473897916s) [noise=true , address=45.91.116.7:4011 , piece_id=GKVDRTZ2TY2ESEKU2FKIJND4LXDMITNWQORE4Z47OQ454UZMM7BA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.628897649 (103h10m50.477504746s) [noise=true , address=45.91.116.5:4298 , piece_id=AIURH3V65NZD3QI3A5YGBJ222ANC4LTHEYFWTOFIC6SPVH3O4CKA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.629542622 (103h10m50.478149697s) [noise=true , address=45.91.116.12:4225 , piece_id=MFM7FUANQ2WU4GK46NPKKKFDUXSVVQ3MGK7GM3QBR7GMOD563PTQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.630003986 (103h10m50.478611073s) [noise=true , address=77.90.31.17:4772 , piece_id=63V72ATK3KI3ADRC4N5HTRIXSEB3DRM7N73MVZNDPZJ7EVO3RZJA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.630441959 (103h10m50.479049047s) [noise=true , address=104.36.82.12:4151 , piece_id=DPQJDPJRZQEJHTED4UJBNSJ2CGAL6FPTLLIIRFTOVBUK65VSPLGA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.630941221 (103h10m50.479548305s) [noise=true , address=77.90.31.12:4592 , piece_id=NAWAKPN4LUPCFZIXDIEPN2GKO3CSGX35LZNHI4QDCGYBYAMWBGOA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.631378970 (103h10m50.47998605s) [noise=true , address=77.90.31.13:4626 , piece_id=D63E7JGO5RRTEJGXVUMNEC3UYXHS45HBOB7S67VPCWLHJXNH7OSQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.631840658 (103h10m50.480447733s) [noise=true , address=77.90.31.17:4771 , piece_id=VCHLVD4PCT6SGJNCFA4LSEDIXSRMX27JH7VLUXDBM2PMIGZ54CNA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.631850523 (103h10m50.480457598s) [seg_pos={0 0}, upload=82463]: Segment reached optimal threshold of 13 pieces.
Jul 23 06:01:08.631982956 (103h10m50.480590031s) [noise=true , address=104.36.82.13:4186 , piece_id=O5BVVJBIWINWK2GXWWJDETED6T3Z462AAUQWCG6LGC7PVVFSHZBA, seg_pos={0 0}, upload=82463]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF
Jul 23 06:01:08.632087803 (103h10m50.480694892s) [noise=true , address=45.91.116.4:4937 , piece_id=DZILXASXCQNDQGTAIYQIY4LCXHENKF6LLWHME5ILDWL3IZOSWMCQ, seg_pos={0 0}, upload=82463]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF
Jul 23 06:01:08.633101069 (103h10m50.481708157s) [noise=true , address=45.91.116.11:4119 , piece_id=LOMOQU77BVHAPGZ23ZY6CG63FZFQX2IG5QRY2P46WKSGB5AF5O3Q, seg_pos={0 0}, upload=82463]: Done uploading piece. err: <nil>
Jul 23 06:01:08.633132335 (103h10m50.481739428s) [seg_pos={0 0}, upload=82463]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil>
Jul 23 06:01:08.633221956 (103h10m50.481829049s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 23 06:01:08.633272252 (103h10m50.481879341s) [upload=82463]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 23 06:01:08.710605841 (103h10m50.559212933s) [upload=82463]: Done uploading segments...
Jul 23 06:01:08.710632085 (103h10m50.559239157s) [upload=82463]: Upload finished. err: <nil>
Duration: 892.644887ms
core
uplink
us-select-1
|