Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......] duration:
1.137456922s
FAIL (2025-08-08 03:02:18 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/4M")
======LOGS======
ad=17842]: Upload finished. err: <nil>
Aug 8 02:56:55.672801893 (17h17m30.187904459s) [upload=17854]: Starting upload
Aug 8 02:56:55.672849099 (17h17m30.187951667s) [upload=17854]: Done starting upload
Aug 8 02:56:55.672934546 (17h17m30.188037129s) [upload=17854]: Uploading segments...
Aug 8 02:56:55.673142624 (17h17m30.1882452s) [upload=17854]: Got next segment. Inline: false
Aug 8 02:56:55.673180963 (17h17m30.18828354s) [upload=17854]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 8 02:56:55.673185766 (17h17m30.188288335s) [upload=17854]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 8 02:56:55.817001710 (17h17m30.332104286s) [seg_pos={0 0}, upload=17854]: Begin upload segment...
Aug 8 02:56:55.817049073 (17h17m30.332151643s) [seg_pos={0 0}, upload=17854]: Done begin upload segment.
Aug 8 02:56:55.817272321 (17h17m30.3323749s) [noise=true , address=104.36.82.16:4310 , piece_id=25F4YPQTH2QEZ6DOLMDF2344TYDJL5JT2GQSMKY3VTUXO6ZX2ZYQ, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.817433916 (17h17m30.332536483s) [noise=true , address=104.36.82.5:4867 , piece_id=S3UA3ZI73DLUUNLDBS5RRMCMUGWDSI377AADY5D2F6APH4WFJRAQ, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.818127795 (17h17m30.333230364s) [noise=true , address=104.36.82.35:4599 , piece_id=Y4ZJOIICH7EKN34WRDBQWERFNVCRGCGPFTF5VYY5K45EXU5JVD6A, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.818952037 (17h17m30.334054608s) [noise=true , address=104.36.82.44:4875 , piece_id=NH5TIT5ST6MEGRB7YYJZM7I2HFBK4R2JR6LA55V3RXQ67CXITYXQ, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.819328753 (17h17m30.334431321s) [noise=true , address=104.36.82.56:4250 , piece_id=4SUTKVTMMTMNWW7B4O4NLVFIX4UYCVTCR6UWOVLAJUC7JHZBHNRA, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.819847891 (17h17m30.334950453s) [noise=true , address=104.36.82.60:4344 , piece_id=WZDEDH2CLCFPCIXGS36MXGM5SRZXLT754AAOOPIBOBJYYOI26IWQ, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.820277897 (17h17m30.33538046s) [noise=true , address=79.127.205.233:4650 , piece_id=BEVZPPYUQGQSBI334FN7I2ARNAIXDMIDXTHMQKYIKWTD2GWMIS7A, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.820670734 (17h17m30.335773297s) [noise=true , address=104.36.82.11:4137 , piece_id=IRDQQF52Z7GTWXITT3AD7YNMYQNRH73IZTXQYQ4LRM23ICQ24WDQ, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.821073724 (17h17m30.336176286s) [noise=true , address=45.91.116.7:4036 , piece_id=7UNVBZXGVPYGIWYE22HHF5DOZ3DJ2RQE5GA2S5YKP62FUTQ7YSGQ, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.821477247 (17h17m30.336579809s) [noise=true , address=104.36.82.55:4200 , piece_id=KRBC2DGBZ5BM6ZCPLJU4FARY2GXCVJKBPB266TK6P6DPSNSRA2EA, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.821956124 (17h17m30.33705869s) [noise=true , address=45.91.116.14:4200 , piece_id=RREGX2U6HK3KO66CXOIDIXAIJ2GXUD7O2HFUDKEKSANKP4LCEFFQ, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.822627361 (17h17m30.337729925s) [noise=true , address=104.36.82.23:4454 , piece_id=GNPYBDITM4PG56VEOBWS4YRTJ3M4CVXEKJ26SDKOOJEBSSSZ7CAA, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.823598424 (17h17m30.338700993s) [noise=true , address=104.36.82.46:4505 , piece_id=CHM2WFHCBUFYAJOFOX2XZ6P3KIBIIIJI3AYJOPVIATA2XQBV2KPA, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.824397787 (17h17m30.339500357s) [noise=true , address=104.36.82.62:4434 , piece_id=2EEQCBQECZHV5GYRD3U3T6MD5JK5WMXTSMY6DY4XJI5U4KAYRQ4Q, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.825235804 (17h17m30.340338373s) [noise=true , address=104.36.82.37:4673 , piece_id=WPUSW3NZVF6T3WWEVD3GMIV5S5EQBCJL5AQBYUDAVJXIOQMHRTUA, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.825350186 (17h17m30.340452757s) [noise=true , address=104.36.82.58:4282 , piece_id=KPAIL5BRZTRZOYWN3KN5QZSXM2JIO2A74UADAFFNI4Q6DR3SN4WA, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.825771286 (17h17m30.340873855s) [noise=true , address=104.36.82.44:4890 , piece_id=RIOGDT6QZQVDQPGC2LMA7D7X47KWOFAQSWTR3X73UVIY2QVY7RVQ, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.827131293 (17h17m30.342233862s) [noise=true , address=104.36.82.59:4303 , piece_id=YMQOCXNYW362BEHWCT4Y6FPBGZBNA2UAZLFG3TGIFJGTZVT32H7Q, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.932537806 (17h17m30.447640388s) [upload=17854]: Next returned nil segment
Aug 8 02:56:55.932554853 (17h17m30.447657422s) [upload=17854]: Waiting for error group managing segments...
Aug 8 02:56:55.937123715 (17h17m30.452226285s) [noise=true , address=45.91.116.7:4036 , piece_id=7UNVBZXGVPYGIWYE22HHF5DOZ3DJ2RQE5GA2S5YKP62FUTQ7YSGQ, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.940599216 (17h17m30.455701786s) [noise=true , address=104.36.82.37:4673 , piece_id=WPUSW3NZVF6T3WWEVD3GMIV5S5EQBCJL5AQBYUDAVJXIOQMHRTUA, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.941043576 (17h17m30.45614614s) [noise=true , address=104.36.82.55:4200 , piece_id=KRBC2DGBZ5BM6ZCPLJU4FARY2GXCVJKBPB266TK6P6DPSNSRA2EA, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.941482170 (17h17m30.456584748s) [noise=true , address=104.36.82.58:4282 , piece_id=KPAIL5BRZTRZOYWN3KN5QZSXM2JIO2A74UADAFFNI4Q6DR3SN4WA, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.942417232 (17h17m30.457519794s) [noise=true , address=104.36.82.16:4310 , piece_id=25F4YPQTH2QEZ6DOLMDF2344TYDJL5JT2GQSMKY3VTUXO6ZX2ZYQ, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.942947508 (17h17m30.458050078s) [noise=true , address=104.36.82.44:4890 , piece_id=RIOGDT6QZQVDQPGC2LMA7D7X47KWOFAQSWTR3X73UVIY2QVY7RVQ, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.943407533 (17h17m30.458510106s) [noise=true , address=104.36.82.62:4434 , piece_id=2EEQCBQECZHV5GYRD3U3T6MD5JK5WMXTSMY6DY4XJI5U4KAYRQ4Q, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.943536993 (17h17m30.45863957s) [noise=true , address=104.36.82.56:4250 , piece_id=4SUTKVTMMTMNWW7B4O4NLVFIX4UYCVTCR6UWOVLAJUC7JHZBHNRA, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.943985401 (17h17m30.459087978s) [noise=true , address=104.36.82.59:4303 , piece_id=YMQOCXNYW362BEHWCT4Y6FPBGZBNA2UAZLFG3TGIFJGTZVT32H7Q, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.944013966 (17h17m30.459116535s) [noise=true , address=104.36.82.5:4867 , piece_id=S3UA3ZI73DLUUNLDBS5RRMCMUGWDSI377AADY5D2F6APH4WFJRAQ, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.944504349 (17h17m30.459606918s) [noise=true , address=104.36.82.44:4875 , piece_id=NH5TIT5ST6MEGRB7YYJZM7I2HFBK4R2JR6LA55V3RXQ67CXITYXQ, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.944950961 (17h17m30.460053533s) [noise=true , address=104.36.82.35:4599 , piece_id=Y4ZJOIICH7EKN34WRDBQWERFNVCRGCGPFTF5VYY5K45EXU5JVD6A, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.945006095 (17h17m30.460108657s) [noise=true , address=104.36.82.46:4505 , piece_id=CHM2WFHCBUFYAJOFOX2XZ6P3KIBIIIJI3AYJOPVIATA2XQBV2KPA, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.945375636 (17h17m30.460478202s) [noise=true , address=104.36.82.23:4454 , piece_id=GNPYBDITM4PG56VEOBWS4YRTJ3M4CVXEKJ26SDKOOJEBSSSZ7CAA, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.945395444 (17h17m30.460498023s) [seg_pos={0 0}, upload=17854]: Segment reached optimal threshold of 14 pieces.
Aug 8 02:56:55.945509537 (17h17m30.4606121s) [noise=true , address=104.36.82.11:4137 , piece_id=IRDQQF52Z7GTWXITT3AD7YNMYQNRH73IZTXQYQ4LRM23ICQ24WDQ, seg_pos={0 0}, upload=17854]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12tkbodtFrNCuJbvZk8YnpSdJfVsKasNuKhg2wGD8rDkNPrHZ21): protocol: expected piece hash; context canceled; EOF
Aug 8 02:56:55.945551105 (17h17m30.460653674s) [noise=true , address=45.91.116.14:4200 , piece_id=RREGX2U6HK3KO66CXOIDIXAIJ2GXUD7O2HFUDKEKSANKP4LCEFFQ, seg_pos={0 0}, upload=17854]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fiQJkHYDv2J4V82doFX3dg43hLpgq3c34Eng3zwqFkF5znyx): protocol: expected piece hash; context canceled; EOF
Aug 8 02:56:55.945584503 (17h17m30.460687066s) [noise=true , address=104.36.82.60:4344 , piece_id=WZDEDH2CLCFPCIXGS36MXGM5SRZXLT754AAOOPIBOBJYYOI26IWQ, seg_pos={0 0}, upload=17854]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127WWD5hr1ZBkRLCHz3jb7kpqqnxPS6VuZ1MyQDmitFbhTSbJNK): protocol: expected piece hash; context canceled; EOF
Aug 8 02:56:55.945654567 (17h17m30.460757143s) [noise=true , address=79.127.205.233:4650 , piece_id=BEVZPPYUQGQSBI334FN7I2ARNAIXDMIDXTHMQKYIKWTD2GWMIS7A, seg_pos={0 0}, upload=17854]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hA5tXuKjGsjJCU9XLeNBXkkahm3FXqLWM83Fk3Hu1TH6rMgoh): protocol: context canceled; context canceled
Aug 8 02:56:55.945671410 (17h17m30.460773979s) [seg_pos={0 0}, upload=17854]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 8 02:56:55.945733549 (17h17m30.460836125s) [upload=17854]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 8 02:56:55.945737891 (17h17m30.460840467s) [upload=17854]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 8 02:56:56.198500182 (17h17m30.713602758s) [upload=17854]: Done uploading segments...
Aug 8 02:56:56.198516890 (17h17m30.713619466s) [upload=17854]: Upload finished. err: <nil>
Duration: 1.137501176s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........] duration:
267.220118ms
FAIL (2025-08-08 03:01:24 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== m43.691381585s) [upload=17645]: Done uploading segments... Aug 8 02:45:09.176296212 (17h5m43.691398786s) [upload=17645]: Upload finished. err: <nil> Aug 8 02:46:04.444774241 (17h6m38.959876804s) [upload=17660]: Starting upload Aug 8 02:46:04.444862810 (17h6m38.959965425s) [upload=17660]: Done starting upload Aug 8 02:46:04.444937571 (17h6m38.96004014s) [upload=17660]: Uploading segments... Aug 8 02:46:04.445020035 (17h6m38.960122601s) [upload=17660]: Got next segment. Inline: true Aug 8 02:46:04.445028641 (17h6m38.960131209s) [upload=17660]: Waiting for error group managing segments... Aug 8 02:46:04.445085559 (17h6m38.960188125s) [upload=17660]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:46:04.445090655 (17h6m38.960193232s) [upload=17660]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:46:04.445094218 (17h6m38.960196793s) [upload=17660]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:46:04.519397453 (17h6m39.034500036s) [upload=17660]: Done uploading segments... Aug 8 02:46:04.519419431 (17h6m39.034522009s) [upload=17660]: Upload finished. err: <nil> Aug 8 02:46:29.848775596 (17h7m4.363878185s) [upload=17669]: Starting upload Aug 8 02:46:29.848845713 (17h7m4.363948289s) [upload=17669]: Done starting upload Aug 8 02:46:29.848924385 (17h7m4.364026954s) [upload=17669]: Uploading segments... Aug 8 02:46:29.848961594 (17h7m4.364064177s) [upload=17669]: Got next segment. Inline: true Aug 8 02:46:29.848968008 (17h7m4.364070587s) [upload=17669]: Waiting for error group managing segments... Aug 8 02:46:29.849009835 (17h7m4.364112411s) [upload=17669]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:46:29.849014457 (17h7m4.364117034s) [upload=17669]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:46:29.849018027 (17h7m4.364120603s) [upload=17669]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:46:29.914967479 (17h7m4.430070058s) [upload=17669]: Done uploading segments... Aug 8 02:46:29.914984083 (17h7m4.430086659s) [upload=17669]: Upload finished. err: <nil> Aug 8 02:47:07.009158285 (17h7m41.524260866s) [upload=17683]: Starting upload Aug 8 02:47:07.009236490 (17h7m41.524339073s) [upload=17683]: Done starting upload Aug 8 02:47:07.009381821 (17h7m41.5244844s) [upload=17683]: Uploading segments... Aug 8 02:47:07.009460862 (17h7m41.524563448s) [upload=17683]: Got next segment. Inline: true Aug 8 02:47:07.009467487 (17h7m41.524570067s) [upload=17683]: Waiting for error group managing segments... Aug 8 02:47:07.009557493 (17h7m41.524660055s) [upload=17683]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:47:07.009562936 (17h7m41.524665513s) [upload=17683]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:47:07.009566255 (17h7m41.524668832s) [upload=17683]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:47:07.108408364 (17h7m41.623510942s) [upload=17683]: Done uploading segments... Aug 8 02:47:07.108426523 (17h7m41.623529101s) [upload=17683]: Upload finished. err: <nil> Aug 8 02:47:43.954966497 (17h8m18.470069065s) [upload=17694]: Starting upload Aug 8 02:47:43.955070525 (17h8m18.470173105s) [upload=17694]: Done starting upload Aug 8 02:47:43.955164363 (17h8m18.470266931s) [upload=17694]: Uploading segments... Aug 8 02:47:43.955233625 (17h8m18.470336207s) [upload=17694]: Got next segment. Inline: true Aug 8 02:47:43.955238449 (17h8m18.470341026s) [upload=17694]: Waiting for error group managing segments... Aug 8 02:47:43.955292283 (17h8m18.470394859s) [upload=17694]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:47:43.955296757 (17h8m18.470399334s) [upload=17694]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:47:43.955299998 (17h8m18.470402574s) [upload=17694]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:47:44.124192924 (17h8m18.639295505s) [upload=17694]: Done uploading segments... Aug 8 02:47:44.124209329 (17h8m18.639311892s) [upload=17694]: Upload finished. err: <nil> Aug 8 02:48:43.731612500 (17h9m18.246715079s) [upload=17714]: Starting upload Aug 8 02:48:43.731667994 (17h9m18.246770571s) [upload=17714]: Done starting upload Aug 8 02:48:43.731765653 (17h9m18.246868236s) [upload=17714]: Uploading segments... Aug 8 02:48:43.731812427 (17h9m18.246915014s) [upload=17714]: Got next segment. Inline: true Aug 8 02:48:43.731833486 (17h9m18.24693607s) [upload=17714]: Waiting for error group managing segments... Aug 8 02:48:43.731883643 (17h9m18.246986222s) [upload=17714]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:48:43.731888532 (17h9m18.246991109s) [upload=17714]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:48:43.731891882 (17h9m18.246994458s) [upload=17714]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:48:44.128878769 (17h9m18.643981354s) [upload=17714]: Done uploading segments... Aug 8 02:48:44.128904173 (17h9m18.644006818s) [upload=17714]: Upload finished. err: <nil> Aug 8 02:49:38.867826030 (17h10m13.382928597s) [upload=17730]: Starting upload Aug 8 02:49:38.867910640 (17h10m13.38301321s) [upload=17730]: Done starting upload Aug 8 02:49:38.868107732 (17h10m13.3832103s) [upload=17730]: Uploading segments... Aug 8 02:49:38.868198737 (17h10m13.383301322s) [upload=17730]: Got next segment. Inline: true Aug 8 02:49:38.868206330 (17h10m13.38330891s) [upload=17730]: Waiting for error group managing segments... Aug 8 02:49:38.868306560 (17h10m13.383409144s) [upload=17730]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:49:38.868323606 (17h10m13.383426205s) [upload=17730]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:49:38.868327207 (17h10m13.383429792s) [upload=17730]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:49:38.988012432 (17h10m13.503115008s) [upload=17730]: Done uploading segments... Aug 8 02:49:38.988031058 (17h10m13.503133625s) [upload=17730]: Upload finished. err: <nil> Aug 8 02:50:07.278685760 (17h10m41.793788343s) [upload=17739]: Starting upload Aug 8 02:50:07.278829106 (17h10m41.793931683s) [upload=17739]: Done starting upload Aug 8 02:50:07.278912448 (17h10m41.79401501s) [upload=17739]: Uploading segments... Aug 8 02:50:07.278959874 (17h10m41.79406246s) [upload=17739]: Got next segment. Inline: true Aug 8 02:50:07.278965894 (17h10m41.79406848s) [upload=17739]: Waiting for error group managing segments... Aug 8 02:50:07.279051579 (17h10m41.794154156s) [upload=17739]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:50:07.279058068 (17h10m41.794160653s) [upload=17739]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:50:07.279062000 (17h10m41.794164578s) [upload=17739]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:50:07.373452172 (17h10m41.88855475s) [upload=17739]: Done uploading segments... Aug 8 02:50:07.373481935 (17h10m41.888584519s) [upload=17739]: Upload finished. err: <nil> Aug 8 02:50:39.619819607 (17h11m14.134922187s) [upload=17749]: Starting upload Aug 8 02:50:39.619901137 (17h11m14.135003715s) [upload=17749]: Done starting upload Aug 8 02:50:39.620009105 (17h11m14.135111684s) [upload=17749]: Uploading segments... Aug 8 02:50:39.620061236 (17h11m14.135163825s) [upload=17749]: Got next segment. Inline: true Aug 8 02:50:39.620066774 (17h11m14.135169357s) [upload=17749]: Waiting for error group managing segments... Aug 8 02:50:39.620129403 (17h11m14.135231991s) [upload=17749]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:50:39.620134679 (17h11m14.135237257s) [upload=17749]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:50:39.620138422 (17h11m14.135241002s) [upload=17749]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:50:39.783502378 (17h11m14.298604959s) [upload=17749]: Done uploading segments... Aug 8 02:50:39.783519169 (17h11m14.298621746s) [upload=17749]: Upload finished. err: <nil> Aug 8 02:51:53.488163266 (17h12m28.003265854s) [upload=17769]: Starting upload Aug 8 02:51:53.488231606 (17h12m28.003334191s) [upload=17769]: Done starting upload Aug 8 02:51:53.488304051 (17h12m28.00340663s) [upload=17769]: Uploading segments... Aug 8 02:51:53.488381549 (17h12m28.003484136s) [upload=17769]: Got next segment. Inline: true Aug 8 02:51:53.488387602 (17h12m28.003490183s) [upload=17769]: Waiting for error group managing segments... Aug 8 02:51:53.488438849 (17h12m28.003541436s) [upload=17769]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:51:53.488445257 (17h12m28.003547838s) [upload=17769]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:51:53.488448976 (17h12m28.003551557s) [upload=17769]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:51:53.555883594 (17h12m28.070986169s) [upload=17769]: Done uploading segments... Aug 8 02:51:53.555899316 (17h12m28.071001892s) [upload=17769]: Upload finished. err: <nil> Aug 8 02:52:49.974130340 (17h13m24.489232916s) [upload=17782]: Starting upload Aug 8 02:52:49.974182339 (17h13m24.489284907s) [upload=17782]: Done starting upload Aug 8 02:52:49.974266182 (17h13m24.48936876s) [upload=17782]: Uploading segments... Aug 8 02:52:49.974303819 (17h13m24.489406465s) [upload=17782]: Got next segment. Inline: true Aug 8 02:52:49.974308907 (17h13m24.489411483s) [upload=17782]: Waiting for error group managing segments... Aug 8 02:52:49.974352082 (17h13m24.489454659s) [upload=17782]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:52:49.974356786 (17h13m24.489459362s) [upload=17782]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:52:49.974360363 (17h13m24.489462939s) [upload=17782]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:52:49.983269871 (17h13m24.498372455s) [upload=17782]: Done uploading segments... Aug 8 02:52:49.983289597 (17h13m24.498392175s) [upload=17782]: Upload finished. err: metaclient: Too Many Requests
Duration: 267.245065ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....] duration:
520.333817ms
FAIL (2025-08-08 03:01:12 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/4M")
======LOGS======
ad=17842]: Upload finished. err: <nil>
Aug 8 02:56:55.672801893 (17h17m30.187904459s) [upload=17854]: Starting upload
Aug 8 02:56:55.672849099 (17h17m30.187951667s) [upload=17854]: Done starting upload
Aug 8 02:56:55.672934546 (17h17m30.188037129s) [upload=17854]: Uploading segments...
Aug 8 02:56:55.673142624 (17h17m30.1882452s) [upload=17854]: Got next segment. Inline: false
Aug 8 02:56:55.673180963 (17h17m30.18828354s) [upload=17854]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 8 02:56:55.673185766 (17h17m30.188288335s) [upload=17854]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 8 02:56:55.817001710 (17h17m30.332104286s) [seg_pos={0 0}, upload=17854]: Begin upload segment...
Aug 8 02:56:55.817049073 (17h17m30.332151643s) [seg_pos={0 0}, upload=17854]: Done begin upload segment.
Aug 8 02:56:55.817272321 (17h17m30.3323749s) [noise=true , address=104.36.82.16:4310 , piece_id=25F4YPQTH2QEZ6DOLMDF2344TYDJL5JT2GQSMKY3VTUXO6ZX2ZYQ, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.817433916 (17h17m30.332536483s) [noise=true , address=104.36.82.5:4867 , piece_id=S3UA3ZI73DLUUNLDBS5RRMCMUGWDSI377AADY5D2F6APH4WFJRAQ, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.818127795 (17h17m30.333230364s) [noise=true , address=104.36.82.35:4599 , piece_id=Y4ZJOIICH7EKN34WRDBQWERFNVCRGCGPFTF5VYY5K45EXU5JVD6A, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.818952037 (17h17m30.334054608s) [noise=true , address=104.36.82.44:4875 , piece_id=NH5TIT5ST6MEGRB7YYJZM7I2HFBK4R2JR6LA55V3RXQ67CXITYXQ, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.819328753 (17h17m30.334431321s) [noise=true , address=104.36.82.56:4250 , piece_id=4SUTKVTMMTMNWW7B4O4NLVFIX4UYCVTCR6UWOVLAJUC7JHZBHNRA, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.819847891 (17h17m30.334950453s) [noise=true , address=104.36.82.60:4344 , piece_id=WZDEDH2CLCFPCIXGS36MXGM5SRZXLT754AAOOPIBOBJYYOI26IWQ, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.820277897 (17h17m30.33538046s) [noise=true , address=79.127.205.233:4650 , piece_id=BEVZPPYUQGQSBI334FN7I2ARNAIXDMIDXTHMQKYIKWTD2GWMIS7A, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.820670734 (17h17m30.335773297s) [noise=true , address=104.36.82.11:4137 , piece_id=IRDQQF52Z7GTWXITT3AD7YNMYQNRH73IZTXQYQ4LRM23ICQ24WDQ, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.821073724 (17h17m30.336176286s) [noise=true , address=45.91.116.7:4036 , piece_id=7UNVBZXGVPYGIWYE22HHF5DOZ3DJ2RQE5GA2S5YKP62FUTQ7YSGQ, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.821477247 (17h17m30.336579809s) [noise=true , address=104.36.82.55:4200 , piece_id=KRBC2DGBZ5BM6ZCPLJU4FARY2GXCVJKBPB266TK6P6DPSNSRA2EA, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.821956124 (17h17m30.33705869s) [noise=true , address=45.91.116.14:4200 , piece_id=RREGX2U6HK3KO66CXOIDIXAIJ2GXUD7O2HFUDKEKSANKP4LCEFFQ, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.822627361 (17h17m30.337729925s) [noise=true , address=104.36.82.23:4454 , piece_id=GNPYBDITM4PG56VEOBWS4YRTJ3M4CVXEKJ26SDKOOJEBSSSZ7CAA, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.823598424 (17h17m30.338700993s) [noise=true , address=104.36.82.46:4505 , piece_id=CHM2WFHCBUFYAJOFOX2XZ6P3KIBIIIJI3AYJOPVIATA2XQBV2KPA, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.824397787 (17h17m30.339500357s) [noise=true , address=104.36.82.62:4434 , piece_id=2EEQCBQECZHV5GYRD3U3T6MD5JK5WMXTSMY6DY4XJI5U4KAYRQ4Q, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.825235804 (17h17m30.340338373s) [noise=true , address=104.36.82.37:4673 , piece_id=WPUSW3NZVF6T3WWEVD3GMIV5S5EQBCJL5AQBYUDAVJXIOQMHRTUA, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.825350186 (17h17m30.340452757s) [noise=true , address=104.36.82.58:4282 , piece_id=KPAIL5BRZTRZOYWN3KN5QZSXM2JIO2A74UADAFFNI4Q6DR3SN4WA, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.825771286 (17h17m30.340873855s) [noise=true , address=104.36.82.44:4890 , piece_id=RIOGDT6QZQVDQPGC2LMA7D7X47KWOFAQSWTR3X73UVIY2QVY7RVQ, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.827131293 (17h17m30.342233862s) [noise=true , address=104.36.82.59:4303 , piece_id=YMQOCXNYW362BEHWCT4Y6FPBGZBNA2UAZLFG3TGIFJGTZVT32H7Q, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.932537806 (17h17m30.447640388s) [upload=17854]: Next returned nil segment
Aug 8 02:56:55.932554853 (17h17m30.447657422s) [upload=17854]: Waiting for error group managing segments...
Aug 8 02:56:55.937123715 (17h17m30.452226285s) [noise=true , address=45.91.116.7:4036 , piece_id=7UNVBZXGVPYGIWYE22HHF5DOZ3DJ2RQE5GA2S5YKP62FUTQ7YSGQ, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.940599216 (17h17m30.455701786s) [noise=true , address=104.36.82.37:4673 , piece_id=WPUSW3NZVF6T3WWEVD3GMIV5S5EQBCJL5AQBYUDAVJXIOQMHRTUA, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.941043576 (17h17m30.45614614s) [noise=true , address=104.36.82.55:4200 , piece_id=KRBC2DGBZ5BM6ZCPLJU4FARY2GXCVJKBPB266TK6P6DPSNSRA2EA, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.941482170 (17h17m30.456584748s) [noise=true , address=104.36.82.58:4282 , piece_id=KPAIL5BRZTRZOYWN3KN5QZSXM2JIO2A74UADAFFNI4Q6DR3SN4WA, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.942417232 (17h17m30.457519794s) [noise=true , address=104.36.82.16:4310 , piece_id=25F4YPQTH2QEZ6DOLMDF2344TYDJL5JT2GQSMKY3VTUXO6ZX2ZYQ, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.942947508 (17h17m30.458050078s) [noise=true , address=104.36.82.44:4890 , piece_id=RIOGDT6QZQVDQPGC2LMA7D7X47KWOFAQSWTR3X73UVIY2QVY7RVQ, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.943407533 (17h17m30.458510106s) [noise=true , address=104.36.82.62:4434 , piece_id=2EEQCBQECZHV5GYRD3U3T6MD5JK5WMXTSMY6DY4XJI5U4KAYRQ4Q, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.943536993 (17h17m30.45863957s) [noise=true , address=104.36.82.56:4250 , piece_id=4SUTKVTMMTMNWW7B4O4NLVFIX4UYCVTCR6UWOVLAJUC7JHZBHNRA, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.943985401 (17h17m30.459087978s) [noise=true , address=104.36.82.59:4303 , piece_id=YMQOCXNYW362BEHWCT4Y6FPBGZBNA2UAZLFG3TGIFJGTZVT32H7Q, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.944013966 (17h17m30.459116535s) [noise=true , address=104.36.82.5:4867 , piece_id=S3UA3ZI73DLUUNLDBS5RRMCMUGWDSI377AADY5D2F6APH4WFJRAQ, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.944504349 (17h17m30.459606918s) [noise=true , address=104.36.82.44:4875 , piece_id=NH5TIT5ST6MEGRB7YYJZM7I2HFBK4R2JR6LA55V3RXQ67CXITYXQ, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.944950961 (17h17m30.460053533s) [noise=true , address=104.36.82.35:4599 , piece_id=Y4ZJOIICH7EKN34WRDBQWERFNVCRGCGPFTF5VYY5K45EXU5JVD6A, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.945006095 (17h17m30.460108657s) [noise=true , address=104.36.82.46:4505 , piece_id=CHM2WFHCBUFYAJOFOX2XZ6P3KIBIIIJI3AYJOPVIATA2XQBV2KPA, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.945375636 (17h17m30.460478202s) [noise=true , address=104.36.82.23:4454 , piece_id=GNPYBDITM4PG56VEOBWS4YRTJ3M4CVXEKJ26SDKOOJEBSSSZ7CAA, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.945395444 (17h17m30.460498023s) [seg_pos={0 0}, upload=17854]: Segment reached optimal threshold of 14 pieces.
Aug 8 02:56:55.945509537 (17h17m30.4606121s) [noise=true , address=104.36.82.11:4137 , piece_id=IRDQQF52Z7GTWXITT3AD7YNMYQNRH73IZTXQYQ4LRM23ICQ24WDQ, seg_pos={0 0}, upload=17854]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12tkbodtFrNCuJbvZk8YnpSdJfVsKasNuKhg2wGD8rDkNPrHZ21): protocol: expected piece hash; context canceled; EOF
Aug 8 02:56:55.945551105 (17h17m30.460653674s) [noise=true , address=45.91.116.14:4200 , piece_id=RREGX2U6HK3KO66CXOIDIXAIJ2GXUD7O2HFUDKEKSANKP4LCEFFQ, seg_pos={0 0}, upload=17854]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fiQJkHYDv2J4V82doFX3dg43hLpgq3c34Eng3zwqFkF5znyx): protocol: expected piece hash; context canceled; EOF
Aug 8 02:56:55.945584503 (17h17m30.460687066s) [noise=true , address=104.36.82.60:4344 , piece_id=WZDEDH2CLCFPCIXGS36MXGM5SRZXLT754AAOOPIBOBJYYOI26IWQ, seg_pos={0 0}, upload=17854]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127WWD5hr1ZBkRLCHz3jb7kpqqnxPS6VuZ1MyQDmitFbhTSbJNK): protocol: expected piece hash; context canceled; EOF
Aug 8 02:56:55.945654567 (17h17m30.460757143s) [noise=true , address=79.127.205.233:4650 , piece_id=BEVZPPYUQGQSBI334FN7I2ARNAIXDMIDXTHMQKYIKWTD2GWMIS7A, seg_pos={0 0}, upload=17854]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hA5tXuKjGsjJCU9XLeNBXkkahm3FXqLWM83Fk3Hu1TH6rMgoh): protocol: context canceled; context canceled
Aug 8 02:56:55.945671410 (17h17m30.460773979s) [seg_pos={0 0}, upload=17854]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 8 02:56:55.945733549 (17h17m30.460836125s) [upload=17854]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 8 02:56:55.945737891 (17h17m30.460840467s) [upload=17854]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 8 02:56:56.198500182 (17h17m30.713602758s) [upload=17854]: Done uploading segments...
Aug 8 02:56:56.198516890 (17h17m30.713619466s) [upload=17854]: Upload finished. err: <nil>
Duration: 520.366923ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
273.57949ms
FAIL (2025-08-08 03:00:20 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== m43.691381585s) [upload=17645]: Done uploading segments... Aug 8 02:45:09.176296212 (17h5m43.691398786s) [upload=17645]: Upload finished. err: <nil> Aug 8 02:46:04.444774241 (17h6m38.959876804s) [upload=17660]: Starting upload Aug 8 02:46:04.444862810 (17h6m38.959965425s) [upload=17660]: Done starting upload Aug 8 02:46:04.444937571 (17h6m38.96004014s) [upload=17660]: Uploading segments... Aug 8 02:46:04.445020035 (17h6m38.960122601s) [upload=17660]: Got next segment. Inline: true Aug 8 02:46:04.445028641 (17h6m38.960131209s) [upload=17660]: Waiting for error group managing segments... Aug 8 02:46:04.445085559 (17h6m38.960188125s) [upload=17660]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:46:04.445090655 (17h6m38.960193232s) [upload=17660]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:46:04.445094218 (17h6m38.960196793s) [upload=17660]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:46:04.519397453 (17h6m39.034500036s) [upload=17660]: Done uploading segments... Aug 8 02:46:04.519419431 (17h6m39.034522009s) [upload=17660]: Upload finished. err: <nil> Aug 8 02:46:29.848775596 (17h7m4.363878185s) [upload=17669]: Starting upload Aug 8 02:46:29.848845713 (17h7m4.363948289s) [upload=17669]: Done starting upload Aug 8 02:46:29.848924385 (17h7m4.364026954s) [upload=17669]: Uploading segments... Aug 8 02:46:29.848961594 (17h7m4.364064177s) [upload=17669]: Got next segment. Inline: true Aug 8 02:46:29.848968008 (17h7m4.364070587s) [upload=17669]: Waiting for error group managing segments... Aug 8 02:46:29.849009835 (17h7m4.364112411s) [upload=17669]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:46:29.849014457 (17h7m4.364117034s) [upload=17669]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:46:29.849018027 (17h7m4.364120603s) [upload=17669]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:46:29.914967479 (17h7m4.430070058s) [upload=17669]: Done uploading segments... Aug 8 02:46:29.914984083 (17h7m4.430086659s) [upload=17669]: Upload finished. err: <nil> Aug 8 02:47:07.009158285 (17h7m41.524260866s) [upload=17683]: Starting upload Aug 8 02:47:07.009236490 (17h7m41.524339073s) [upload=17683]: Done starting upload Aug 8 02:47:07.009381821 (17h7m41.5244844s) [upload=17683]: Uploading segments... Aug 8 02:47:07.009460862 (17h7m41.524563448s) [upload=17683]: Got next segment. Inline: true Aug 8 02:47:07.009467487 (17h7m41.524570067s) [upload=17683]: Waiting for error group managing segments... Aug 8 02:47:07.009557493 (17h7m41.524660055s) [upload=17683]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:47:07.009562936 (17h7m41.524665513s) [upload=17683]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:47:07.009566255 (17h7m41.524668832s) [upload=17683]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:47:07.108408364 (17h7m41.623510942s) [upload=17683]: Done uploading segments... Aug 8 02:47:07.108426523 (17h7m41.623529101s) [upload=17683]: Upload finished. err: <nil> Aug 8 02:47:43.954966497 (17h8m18.470069065s) [upload=17694]: Starting upload Aug 8 02:47:43.955070525 (17h8m18.470173105s) [upload=17694]: Done starting upload Aug 8 02:47:43.955164363 (17h8m18.470266931s) [upload=17694]: Uploading segments... Aug 8 02:47:43.955233625 (17h8m18.470336207s) [upload=17694]: Got next segment. Inline: true Aug 8 02:47:43.955238449 (17h8m18.470341026s) [upload=17694]: Waiting for error group managing segments... Aug 8 02:47:43.955292283 (17h8m18.470394859s) [upload=17694]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:47:43.955296757 (17h8m18.470399334s) [upload=17694]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:47:43.955299998 (17h8m18.470402574s) [upload=17694]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:47:44.124192924 (17h8m18.639295505s) [upload=17694]: Done uploading segments... Aug 8 02:47:44.124209329 (17h8m18.639311892s) [upload=17694]: Upload finished. err: <nil> Aug 8 02:48:43.731612500 (17h9m18.246715079s) [upload=17714]: Starting upload Aug 8 02:48:43.731667994 (17h9m18.246770571s) [upload=17714]: Done starting upload Aug 8 02:48:43.731765653 (17h9m18.246868236s) [upload=17714]: Uploading segments... Aug 8 02:48:43.731812427 (17h9m18.246915014s) [upload=17714]: Got next segment. Inline: true Aug 8 02:48:43.731833486 (17h9m18.24693607s) [upload=17714]: Waiting for error group managing segments... Aug 8 02:48:43.731883643 (17h9m18.246986222s) [upload=17714]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:48:43.731888532 (17h9m18.246991109s) [upload=17714]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:48:43.731891882 (17h9m18.246994458s) [upload=17714]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:48:44.128878769 (17h9m18.643981354s) [upload=17714]: Done uploading segments... Aug 8 02:48:44.128904173 (17h9m18.644006818s) [upload=17714]: Upload finished. err: <nil> Aug 8 02:49:38.867826030 (17h10m13.382928597s) [upload=17730]: Starting upload Aug 8 02:49:38.867910640 (17h10m13.38301321s) [upload=17730]: Done starting upload Aug 8 02:49:38.868107732 (17h10m13.3832103s) [upload=17730]: Uploading segments... Aug 8 02:49:38.868198737 (17h10m13.383301322s) [upload=17730]: Got next segment. Inline: true Aug 8 02:49:38.868206330 (17h10m13.38330891s) [upload=17730]: Waiting for error group managing segments... Aug 8 02:49:38.868306560 (17h10m13.383409144s) [upload=17730]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:49:38.868323606 (17h10m13.383426205s) [upload=17730]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:49:38.868327207 (17h10m13.383429792s) [upload=17730]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:49:38.988012432 (17h10m13.503115008s) [upload=17730]: Done uploading segments... Aug 8 02:49:38.988031058 (17h10m13.503133625s) [upload=17730]: Upload finished. err: <nil> Aug 8 02:50:07.278685760 (17h10m41.793788343s) [upload=17739]: Starting upload Aug 8 02:50:07.278829106 (17h10m41.793931683s) [upload=17739]: Done starting upload Aug 8 02:50:07.278912448 (17h10m41.79401501s) [upload=17739]: Uploading segments... Aug 8 02:50:07.278959874 (17h10m41.79406246s) [upload=17739]: Got next segment. Inline: true Aug 8 02:50:07.278965894 (17h10m41.79406848s) [upload=17739]: Waiting for error group managing segments... Aug 8 02:50:07.279051579 (17h10m41.794154156s) [upload=17739]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:50:07.279058068 (17h10m41.794160653s) [upload=17739]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:50:07.279062000 (17h10m41.794164578s) [upload=17739]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:50:07.373452172 (17h10m41.88855475s) [upload=17739]: Done uploading segments... Aug 8 02:50:07.373481935 (17h10m41.888584519s) [upload=17739]: Upload finished. err: <nil> Aug 8 02:50:39.619819607 (17h11m14.134922187s) [upload=17749]: Starting upload Aug 8 02:50:39.619901137 (17h11m14.135003715s) [upload=17749]: Done starting upload Aug 8 02:50:39.620009105 (17h11m14.135111684s) [upload=17749]: Uploading segments... Aug 8 02:50:39.620061236 (17h11m14.135163825s) [upload=17749]: Got next segment. Inline: true Aug 8 02:50:39.620066774 (17h11m14.135169357s) [upload=17749]: Waiting for error group managing segments... Aug 8 02:50:39.620129403 (17h11m14.135231991s) [upload=17749]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:50:39.620134679 (17h11m14.135237257s) [upload=17749]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:50:39.620138422 (17h11m14.135241002s) [upload=17749]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:50:39.783502378 (17h11m14.298604959s) [upload=17749]: Done uploading segments... Aug 8 02:50:39.783519169 (17h11m14.298621746s) [upload=17749]: Upload finished. err: <nil> Aug 8 02:51:53.488163266 (17h12m28.003265854s) [upload=17769]: Starting upload Aug 8 02:51:53.488231606 (17h12m28.003334191s) [upload=17769]: Done starting upload Aug 8 02:51:53.488304051 (17h12m28.00340663s) [upload=17769]: Uploading segments... Aug 8 02:51:53.488381549 (17h12m28.003484136s) [upload=17769]: Got next segment. Inline: true Aug 8 02:51:53.488387602 (17h12m28.003490183s) [upload=17769]: Waiting for error group managing segments... Aug 8 02:51:53.488438849 (17h12m28.003541436s) [upload=17769]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:51:53.488445257 (17h12m28.003547838s) [upload=17769]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:51:53.488448976 (17h12m28.003551557s) [upload=17769]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:51:53.555883594 (17h12m28.070986169s) [upload=17769]: Done uploading segments... Aug 8 02:51:53.555899316 (17h12m28.071001892s) [upload=17769]: Upload finished. err: <nil> Aug 8 02:52:49.974130340 (17h13m24.489232916s) [upload=17782]: Starting upload Aug 8 02:52:49.974182339 (17h13m24.489284907s) [upload=17782]: Done starting upload Aug 8 02:52:49.974266182 (17h13m24.48936876s) [upload=17782]: Uploading segments... Aug 8 02:52:49.974303819 (17h13m24.489406465s) [upload=17782]: Got next segment. Inline: true Aug 8 02:52:49.974308907 (17h13m24.489411483s) [upload=17782]: Waiting for error group managing segments... Aug 8 02:52:49.974352082 (17h13m24.489454659s) [upload=17782]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:52:49.974356786 (17h13m24.489459362s) [upload=17782]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:52:49.974360363 (17h13m24.489462939s) [upload=17782]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:52:49.983269871 (17h13m24.498372455s) [upload=17782]: Done uploading segments... Aug 8 02:52:49.983289597 (17h13m24.498392175s) [upload=17782]: Upload finished. err: metaclient: Too Many Requests
Duration: 273.614482ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....] duration:
790.87686ms
FAIL (2025-08-08 03:00:10 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/4M")
======LOGS======
ad=17842]: Upload finished. err: <nil>
Aug 8 02:56:55.672801893 (17h17m30.187904459s) [upload=17854]: Starting upload
Aug 8 02:56:55.672849099 (17h17m30.187951667s) [upload=17854]: Done starting upload
Aug 8 02:56:55.672934546 (17h17m30.188037129s) [upload=17854]: Uploading segments...
Aug 8 02:56:55.673142624 (17h17m30.1882452s) [upload=17854]: Got next segment. Inline: false
Aug 8 02:56:55.673180963 (17h17m30.18828354s) [upload=17854]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 8 02:56:55.673185766 (17h17m30.188288335s) [upload=17854]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 8 02:56:55.817001710 (17h17m30.332104286s) [seg_pos={0 0}, upload=17854]: Begin upload segment...
Aug 8 02:56:55.817049073 (17h17m30.332151643s) [seg_pos={0 0}, upload=17854]: Done begin upload segment.
Aug 8 02:56:55.817272321 (17h17m30.3323749s) [noise=true , address=104.36.82.16:4310 , piece_id=25F4YPQTH2QEZ6DOLMDF2344TYDJL5JT2GQSMKY3VTUXO6ZX2ZYQ, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.817433916 (17h17m30.332536483s) [noise=true , address=104.36.82.5:4867 , piece_id=S3UA3ZI73DLUUNLDBS5RRMCMUGWDSI377AADY5D2F6APH4WFJRAQ, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.818127795 (17h17m30.333230364s) [noise=true , address=104.36.82.35:4599 , piece_id=Y4ZJOIICH7EKN34WRDBQWERFNVCRGCGPFTF5VYY5K45EXU5JVD6A, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.818952037 (17h17m30.334054608s) [noise=true , address=104.36.82.44:4875 , piece_id=NH5TIT5ST6MEGRB7YYJZM7I2HFBK4R2JR6LA55V3RXQ67CXITYXQ, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.819328753 (17h17m30.334431321s) [noise=true , address=104.36.82.56:4250 , piece_id=4SUTKVTMMTMNWW7B4O4NLVFIX4UYCVTCR6UWOVLAJUC7JHZBHNRA, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.819847891 (17h17m30.334950453s) [noise=true , address=104.36.82.60:4344 , piece_id=WZDEDH2CLCFPCIXGS36MXGM5SRZXLT754AAOOPIBOBJYYOI26IWQ, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.820277897 (17h17m30.33538046s) [noise=true , address=79.127.205.233:4650 , piece_id=BEVZPPYUQGQSBI334FN7I2ARNAIXDMIDXTHMQKYIKWTD2GWMIS7A, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.820670734 (17h17m30.335773297s) [noise=true , address=104.36.82.11:4137 , piece_id=IRDQQF52Z7GTWXITT3AD7YNMYQNRH73IZTXQYQ4LRM23ICQ24WDQ, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.821073724 (17h17m30.336176286s) [noise=true , address=45.91.116.7:4036 , piece_id=7UNVBZXGVPYGIWYE22HHF5DOZ3DJ2RQE5GA2S5YKP62FUTQ7YSGQ, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.821477247 (17h17m30.336579809s) [noise=true , address=104.36.82.55:4200 , piece_id=KRBC2DGBZ5BM6ZCPLJU4FARY2GXCVJKBPB266TK6P6DPSNSRA2EA, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.821956124 (17h17m30.33705869s) [noise=true , address=45.91.116.14:4200 , piece_id=RREGX2U6HK3KO66CXOIDIXAIJ2GXUD7O2HFUDKEKSANKP4LCEFFQ, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.822627361 (17h17m30.337729925s) [noise=true , address=104.36.82.23:4454 , piece_id=GNPYBDITM4PG56VEOBWS4YRTJ3M4CVXEKJ26SDKOOJEBSSSZ7CAA, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.823598424 (17h17m30.338700993s) [noise=true , address=104.36.82.46:4505 , piece_id=CHM2WFHCBUFYAJOFOX2XZ6P3KIBIIIJI3AYJOPVIATA2XQBV2KPA, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.824397787 (17h17m30.339500357s) [noise=true , address=104.36.82.62:4434 , piece_id=2EEQCBQECZHV5GYRD3U3T6MD5JK5WMXTSMY6DY4XJI5U4KAYRQ4Q, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.825235804 (17h17m30.340338373s) [noise=true , address=104.36.82.37:4673 , piece_id=WPUSW3NZVF6T3WWEVD3GMIV5S5EQBCJL5AQBYUDAVJXIOQMHRTUA, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.825350186 (17h17m30.340452757s) [noise=true , address=104.36.82.58:4282 , piece_id=KPAIL5BRZTRZOYWN3KN5QZSXM2JIO2A74UADAFFNI4Q6DR3SN4WA, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.825771286 (17h17m30.340873855s) [noise=true , address=104.36.82.44:4890 , piece_id=RIOGDT6QZQVDQPGC2LMA7D7X47KWOFAQSWTR3X73UVIY2QVY7RVQ, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.827131293 (17h17m30.342233862s) [noise=true , address=104.36.82.59:4303 , piece_id=YMQOCXNYW362BEHWCT4Y6FPBGZBNA2UAZLFG3TGIFJGTZVT32H7Q, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.932537806 (17h17m30.447640388s) [upload=17854]: Next returned nil segment
Aug 8 02:56:55.932554853 (17h17m30.447657422s) [upload=17854]: Waiting for error group managing segments...
Aug 8 02:56:55.937123715 (17h17m30.452226285s) [noise=true , address=45.91.116.7:4036 , piece_id=7UNVBZXGVPYGIWYE22HHF5DOZ3DJ2RQE5GA2S5YKP62FUTQ7YSGQ, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.940599216 (17h17m30.455701786s) [noise=true , address=104.36.82.37:4673 , piece_id=WPUSW3NZVF6T3WWEVD3GMIV5S5EQBCJL5AQBYUDAVJXIOQMHRTUA, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.941043576 (17h17m30.45614614s) [noise=true , address=104.36.82.55:4200 , piece_id=KRBC2DGBZ5BM6ZCPLJU4FARY2GXCVJKBPB266TK6P6DPSNSRA2EA, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.941482170 (17h17m30.456584748s) [noise=true , address=104.36.82.58:4282 , piece_id=KPAIL5BRZTRZOYWN3KN5QZSXM2JIO2A74UADAFFNI4Q6DR3SN4WA, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.942417232 (17h17m30.457519794s) [noise=true , address=104.36.82.16:4310 , piece_id=25F4YPQTH2QEZ6DOLMDF2344TYDJL5JT2GQSMKY3VTUXO6ZX2ZYQ, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.942947508 (17h17m30.458050078s) [noise=true , address=104.36.82.44:4890 , piece_id=RIOGDT6QZQVDQPGC2LMA7D7X47KWOFAQSWTR3X73UVIY2QVY7RVQ, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.943407533 (17h17m30.458510106s) [noise=true , address=104.36.82.62:4434 , piece_id=2EEQCBQECZHV5GYRD3U3T6MD5JK5WMXTSMY6DY4XJI5U4KAYRQ4Q, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.943536993 (17h17m30.45863957s) [noise=true , address=104.36.82.56:4250 , piece_id=4SUTKVTMMTMNWW7B4O4NLVFIX4UYCVTCR6UWOVLAJUC7JHZBHNRA, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.943985401 (17h17m30.459087978s) [noise=true , address=104.36.82.59:4303 , piece_id=YMQOCXNYW362BEHWCT4Y6FPBGZBNA2UAZLFG3TGIFJGTZVT32H7Q, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.944013966 (17h17m30.459116535s) [noise=true , address=104.36.82.5:4867 , piece_id=S3UA3ZI73DLUUNLDBS5RRMCMUGWDSI377AADY5D2F6APH4WFJRAQ, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.944504349 (17h17m30.459606918s) [noise=true , address=104.36.82.44:4875 , piece_id=NH5TIT5ST6MEGRB7YYJZM7I2HFBK4R2JR6LA55V3RXQ67CXITYXQ, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.944950961 (17h17m30.460053533s) [noise=true , address=104.36.82.35:4599 , piece_id=Y4ZJOIICH7EKN34WRDBQWERFNVCRGCGPFTF5VYY5K45EXU5JVD6A, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.945006095 (17h17m30.460108657s) [noise=true , address=104.36.82.46:4505 , piece_id=CHM2WFHCBUFYAJOFOX2XZ6P3KIBIIIJI3AYJOPVIATA2XQBV2KPA, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.945375636 (17h17m30.460478202s) [noise=true , address=104.36.82.23:4454 , piece_id=GNPYBDITM4PG56VEOBWS4YRTJ3M4CVXEKJ26SDKOOJEBSSSZ7CAA, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.945395444 (17h17m30.460498023s) [seg_pos={0 0}, upload=17854]: Segment reached optimal threshold of 14 pieces.
Aug 8 02:56:55.945509537 (17h17m30.4606121s) [noise=true , address=104.36.82.11:4137 , piece_id=IRDQQF52Z7GTWXITT3AD7YNMYQNRH73IZTXQYQ4LRM23ICQ24WDQ, seg_pos={0 0}, upload=17854]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12tkbodtFrNCuJbvZk8YnpSdJfVsKasNuKhg2wGD8rDkNPrHZ21): protocol: expected piece hash; context canceled; EOF
Aug 8 02:56:55.945551105 (17h17m30.460653674s) [noise=true , address=45.91.116.14:4200 , piece_id=RREGX2U6HK3KO66CXOIDIXAIJ2GXUD7O2HFUDKEKSANKP4LCEFFQ, seg_pos={0 0}, upload=17854]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fiQJkHYDv2J4V82doFX3dg43hLpgq3c34Eng3zwqFkF5znyx): protocol: expected piece hash; context canceled; EOF
Aug 8 02:56:55.945584503 (17h17m30.460687066s) [noise=true , address=104.36.82.60:4344 , piece_id=WZDEDH2CLCFPCIXGS36MXGM5SRZXLT754AAOOPIBOBJYYOI26IWQ, seg_pos={0 0}, upload=17854]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127WWD5hr1ZBkRLCHz3jb7kpqqnxPS6VuZ1MyQDmitFbhTSbJNK): protocol: expected piece hash; context canceled; EOF
Aug 8 02:56:55.945654567 (17h17m30.460757143s) [noise=true , address=79.127.205.233:4650 , piece_id=BEVZPPYUQGQSBI334FN7I2ARNAIXDMIDXTHMQKYIKWTD2GWMIS7A, seg_pos={0 0}, upload=17854]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hA5tXuKjGsjJCU9XLeNBXkkahm3FXqLWM83Fk3Hu1TH6rMgoh): protocol: context canceled; context canceled
Aug 8 02:56:55.945671410 (17h17m30.460773979s) [seg_pos={0 0}, upload=17854]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 8 02:56:55.945733549 (17h17m30.460836125s) [upload=17854]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 8 02:56:55.945737891 (17h17m30.460840467s) [upload=17854]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 8 02:56:56.198500182 (17h17m30.713602758s) [upload=17854]: Done uploading segments...
Aug 8 02:56:56.198516890 (17h17m30.713619466s) [upload=17854]: Upload finished. err: <nil>
Duration: 791.121055ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...] duration:
573.524427ms
FAIL (2025-08-08 02:59:44 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/4M")
======LOGS======
ad=17842]: Upload finished. err: <nil>
Aug 8 02:56:55.672801893 (17h17m30.187904459s) [upload=17854]: Starting upload
Aug 8 02:56:55.672849099 (17h17m30.187951667s) [upload=17854]: Done starting upload
Aug 8 02:56:55.672934546 (17h17m30.188037129s) [upload=17854]: Uploading segments...
Aug 8 02:56:55.673142624 (17h17m30.1882452s) [upload=17854]: Got next segment. Inline: false
Aug 8 02:56:55.673180963 (17h17m30.18828354s) [upload=17854]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 8 02:56:55.673185766 (17h17m30.188288335s) [upload=17854]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 8 02:56:55.817001710 (17h17m30.332104286s) [seg_pos={0 0}, upload=17854]: Begin upload segment...
Aug 8 02:56:55.817049073 (17h17m30.332151643s) [seg_pos={0 0}, upload=17854]: Done begin upload segment.
Aug 8 02:56:55.817272321 (17h17m30.3323749s) [noise=true , address=104.36.82.16:4310 , piece_id=25F4YPQTH2QEZ6DOLMDF2344TYDJL5JT2GQSMKY3VTUXO6ZX2ZYQ, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.817433916 (17h17m30.332536483s) [noise=true , address=104.36.82.5:4867 , piece_id=S3UA3ZI73DLUUNLDBS5RRMCMUGWDSI377AADY5D2F6APH4WFJRAQ, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.818127795 (17h17m30.333230364s) [noise=true , address=104.36.82.35:4599 , piece_id=Y4ZJOIICH7EKN34WRDBQWERFNVCRGCGPFTF5VYY5K45EXU5JVD6A, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.818952037 (17h17m30.334054608s) [noise=true , address=104.36.82.44:4875 , piece_id=NH5TIT5ST6MEGRB7YYJZM7I2HFBK4R2JR6LA55V3RXQ67CXITYXQ, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.819328753 (17h17m30.334431321s) [noise=true , address=104.36.82.56:4250 , piece_id=4SUTKVTMMTMNWW7B4O4NLVFIX4UYCVTCR6UWOVLAJUC7JHZBHNRA, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.819847891 (17h17m30.334950453s) [noise=true , address=104.36.82.60:4344 , piece_id=WZDEDH2CLCFPCIXGS36MXGM5SRZXLT754AAOOPIBOBJYYOI26IWQ, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.820277897 (17h17m30.33538046s) [noise=true , address=79.127.205.233:4650 , piece_id=BEVZPPYUQGQSBI334FN7I2ARNAIXDMIDXTHMQKYIKWTD2GWMIS7A, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.820670734 (17h17m30.335773297s) [noise=true , address=104.36.82.11:4137 , piece_id=IRDQQF52Z7GTWXITT3AD7YNMYQNRH73IZTXQYQ4LRM23ICQ24WDQ, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.821073724 (17h17m30.336176286s) [noise=true , address=45.91.116.7:4036 , piece_id=7UNVBZXGVPYGIWYE22HHF5DOZ3DJ2RQE5GA2S5YKP62FUTQ7YSGQ, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.821477247 (17h17m30.336579809s) [noise=true , address=104.36.82.55:4200 , piece_id=KRBC2DGBZ5BM6ZCPLJU4FARY2GXCVJKBPB266TK6P6DPSNSRA2EA, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.821956124 (17h17m30.33705869s) [noise=true , address=45.91.116.14:4200 , piece_id=RREGX2U6HK3KO66CXOIDIXAIJ2GXUD7O2HFUDKEKSANKP4LCEFFQ, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.822627361 (17h17m30.337729925s) [noise=true , address=104.36.82.23:4454 , piece_id=GNPYBDITM4PG56VEOBWS4YRTJ3M4CVXEKJ26SDKOOJEBSSSZ7CAA, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.823598424 (17h17m30.338700993s) [noise=true , address=104.36.82.46:4505 , piece_id=CHM2WFHCBUFYAJOFOX2XZ6P3KIBIIIJI3AYJOPVIATA2XQBV2KPA, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.824397787 (17h17m30.339500357s) [noise=true , address=104.36.82.62:4434 , piece_id=2EEQCBQECZHV5GYRD3U3T6MD5JK5WMXTSMY6DY4XJI5U4KAYRQ4Q, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.825235804 (17h17m30.340338373s) [noise=true , address=104.36.82.37:4673 , piece_id=WPUSW3NZVF6T3WWEVD3GMIV5S5EQBCJL5AQBYUDAVJXIOQMHRTUA, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.825350186 (17h17m30.340452757s) [noise=true , address=104.36.82.58:4282 , piece_id=KPAIL5BRZTRZOYWN3KN5QZSXM2JIO2A74UADAFFNI4Q6DR3SN4WA, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.825771286 (17h17m30.340873855s) [noise=true , address=104.36.82.44:4890 , piece_id=RIOGDT6QZQVDQPGC2LMA7D7X47KWOFAQSWTR3X73UVIY2QVY7RVQ, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.827131293 (17h17m30.342233862s) [noise=true , address=104.36.82.59:4303 , piece_id=YMQOCXNYW362BEHWCT4Y6FPBGZBNA2UAZLFG3TGIFJGTZVT32H7Q, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.932537806 (17h17m30.447640388s) [upload=17854]: Next returned nil segment
Aug 8 02:56:55.932554853 (17h17m30.447657422s) [upload=17854]: Waiting for error group managing segments...
Aug 8 02:56:55.937123715 (17h17m30.452226285s) [noise=true , address=45.91.116.7:4036 , piece_id=7UNVBZXGVPYGIWYE22HHF5DOZ3DJ2RQE5GA2S5YKP62FUTQ7YSGQ, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.940599216 (17h17m30.455701786s) [noise=true , address=104.36.82.37:4673 , piece_id=WPUSW3NZVF6T3WWEVD3GMIV5S5EQBCJL5AQBYUDAVJXIOQMHRTUA, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.941043576 (17h17m30.45614614s) [noise=true , address=104.36.82.55:4200 , piece_id=KRBC2DGBZ5BM6ZCPLJU4FARY2GXCVJKBPB266TK6P6DPSNSRA2EA, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.941482170 (17h17m30.456584748s) [noise=true , address=104.36.82.58:4282 , piece_id=KPAIL5BRZTRZOYWN3KN5QZSXM2JIO2A74UADAFFNI4Q6DR3SN4WA, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.942417232 (17h17m30.457519794s) [noise=true , address=104.36.82.16:4310 , piece_id=25F4YPQTH2QEZ6DOLMDF2344TYDJL5JT2GQSMKY3VTUXO6ZX2ZYQ, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.942947508 (17h17m30.458050078s) [noise=true , address=104.36.82.44:4890 , piece_id=RIOGDT6QZQVDQPGC2LMA7D7X47KWOFAQSWTR3X73UVIY2QVY7RVQ, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.943407533 (17h17m30.458510106s) [noise=true , address=104.36.82.62:4434 , piece_id=2EEQCBQECZHV5GYRD3U3T6MD5JK5WMXTSMY6DY4XJI5U4KAYRQ4Q, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.943536993 (17h17m30.45863957s) [noise=true , address=104.36.82.56:4250 , piece_id=4SUTKVTMMTMNWW7B4O4NLVFIX4UYCVTCR6UWOVLAJUC7JHZBHNRA, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.943985401 (17h17m30.459087978s) [noise=true , address=104.36.82.59:4303 , piece_id=YMQOCXNYW362BEHWCT4Y6FPBGZBNA2UAZLFG3TGIFJGTZVT32H7Q, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.944013966 (17h17m30.459116535s) [noise=true , address=104.36.82.5:4867 , piece_id=S3UA3ZI73DLUUNLDBS5RRMCMUGWDSI377AADY5D2F6APH4WFJRAQ, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.944504349 (17h17m30.459606918s) [noise=true , address=104.36.82.44:4875 , piece_id=NH5TIT5ST6MEGRB7YYJZM7I2HFBK4R2JR6LA55V3RXQ67CXITYXQ, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.944950961 (17h17m30.460053533s) [noise=true , address=104.36.82.35:4599 , piece_id=Y4ZJOIICH7EKN34WRDBQWERFNVCRGCGPFTF5VYY5K45EXU5JVD6A, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.945006095 (17h17m30.460108657s) [noise=true , address=104.36.82.46:4505 , piece_id=CHM2WFHCBUFYAJOFOX2XZ6P3KIBIIIJI3AYJOPVIATA2XQBV2KPA, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.945375636 (17h17m30.460478202s) [noise=true , address=104.36.82.23:4454 , piece_id=GNPYBDITM4PG56VEOBWS4YRTJ3M4CVXEKJ26SDKOOJEBSSSZ7CAA, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.945395444 (17h17m30.460498023s) [seg_pos={0 0}, upload=17854]: Segment reached optimal threshold of 14 pieces.
Aug 8 02:56:55.945509537 (17h17m30.4606121s) [noise=true , address=104.36.82.11:4137 , piece_id=IRDQQF52Z7GTWXITT3AD7YNMYQNRH73IZTXQYQ4LRM23ICQ24WDQ, seg_pos={0 0}, upload=17854]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12tkbodtFrNCuJbvZk8YnpSdJfVsKasNuKhg2wGD8rDkNPrHZ21): protocol: expected piece hash; context canceled; EOF
Aug 8 02:56:55.945551105 (17h17m30.460653674s) [noise=true , address=45.91.116.14:4200 , piece_id=RREGX2U6HK3KO66CXOIDIXAIJ2GXUD7O2HFUDKEKSANKP4LCEFFQ, seg_pos={0 0}, upload=17854]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fiQJkHYDv2J4V82doFX3dg43hLpgq3c34Eng3zwqFkF5znyx): protocol: expected piece hash; context canceled; EOF
Aug 8 02:56:55.945584503 (17h17m30.460687066s) [noise=true , address=104.36.82.60:4344 , piece_id=WZDEDH2CLCFPCIXGS36MXGM5SRZXLT754AAOOPIBOBJYYOI26IWQ, seg_pos={0 0}, upload=17854]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127WWD5hr1ZBkRLCHz3jb7kpqqnxPS6VuZ1MyQDmitFbhTSbJNK): protocol: expected piece hash; context canceled; EOF
Aug 8 02:56:55.945654567 (17h17m30.460757143s) [noise=true , address=79.127.205.233:4650 , piece_id=BEVZPPYUQGQSBI334FN7I2ARNAIXDMIDXTHMQKYIKWTD2GWMIS7A, seg_pos={0 0}, upload=17854]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hA5tXuKjGsjJCU9XLeNBXkkahm3FXqLWM83Fk3Hu1TH6rMgoh): protocol: context canceled; context canceled
Aug 8 02:56:55.945671410 (17h17m30.460773979s) [seg_pos={0 0}, upload=17854]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 8 02:56:55.945733549 (17h17m30.460836125s) [upload=17854]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 8 02:56:55.945737891 (17h17m30.460840467s) [upload=17854]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 8 02:56:56.198500182 (17h17m30.713602758s) [upload=17854]: Done uploading segments...
Aug 8 02:56:56.198516890 (17h17m30.713619466s) [upload=17854]: Upload finished. err: <nil>
Duration: 573.552498ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........] duration:
236.855118ms
FAIL (2025-08-08 02:59:09 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== m43.691381585s) [upload=17645]: Done uploading segments... Aug 8 02:45:09.176296212 (17h5m43.691398786s) [upload=17645]: Upload finished. err: <nil> Aug 8 02:46:04.444774241 (17h6m38.959876804s) [upload=17660]: Starting upload Aug 8 02:46:04.444862810 (17h6m38.959965425s) [upload=17660]: Done starting upload Aug 8 02:46:04.444937571 (17h6m38.96004014s) [upload=17660]: Uploading segments... Aug 8 02:46:04.445020035 (17h6m38.960122601s) [upload=17660]: Got next segment. Inline: true Aug 8 02:46:04.445028641 (17h6m38.960131209s) [upload=17660]: Waiting for error group managing segments... Aug 8 02:46:04.445085559 (17h6m38.960188125s) [upload=17660]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:46:04.445090655 (17h6m38.960193232s) [upload=17660]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:46:04.445094218 (17h6m38.960196793s) [upload=17660]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:46:04.519397453 (17h6m39.034500036s) [upload=17660]: Done uploading segments... Aug 8 02:46:04.519419431 (17h6m39.034522009s) [upload=17660]: Upload finished. err: <nil> Aug 8 02:46:29.848775596 (17h7m4.363878185s) [upload=17669]: Starting upload Aug 8 02:46:29.848845713 (17h7m4.363948289s) [upload=17669]: Done starting upload Aug 8 02:46:29.848924385 (17h7m4.364026954s) [upload=17669]: Uploading segments... Aug 8 02:46:29.848961594 (17h7m4.364064177s) [upload=17669]: Got next segment. Inline: true Aug 8 02:46:29.848968008 (17h7m4.364070587s) [upload=17669]: Waiting for error group managing segments... Aug 8 02:46:29.849009835 (17h7m4.364112411s) [upload=17669]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:46:29.849014457 (17h7m4.364117034s) [upload=17669]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:46:29.849018027 (17h7m4.364120603s) [upload=17669]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:46:29.914967479 (17h7m4.430070058s) [upload=17669]: Done uploading segments... Aug 8 02:46:29.914984083 (17h7m4.430086659s) [upload=17669]: Upload finished. err: <nil> Aug 8 02:47:07.009158285 (17h7m41.524260866s) [upload=17683]: Starting upload Aug 8 02:47:07.009236490 (17h7m41.524339073s) [upload=17683]: Done starting upload Aug 8 02:47:07.009381821 (17h7m41.5244844s) [upload=17683]: Uploading segments... Aug 8 02:47:07.009460862 (17h7m41.524563448s) [upload=17683]: Got next segment. Inline: true Aug 8 02:47:07.009467487 (17h7m41.524570067s) [upload=17683]: Waiting for error group managing segments... Aug 8 02:47:07.009557493 (17h7m41.524660055s) [upload=17683]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:47:07.009562936 (17h7m41.524665513s) [upload=17683]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:47:07.009566255 (17h7m41.524668832s) [upload=17683]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:47:07.108408364 (17h7m41.623510942s) [upload=17683]: Done uploading segments... Aug 8 02:47:07.108426523 (17h7m41.623529101s) [upload=17683]: Upload finished. err: <nil> Aug 8 02:47:43.954966497 (17h8m18.470069065s) [upload=17694]: Starting upload Aug 8 02:47:43.955070525 (17h8m18.470173105s) [upload=17694]: Done starting upload Aug 8 02:47:43.955164363 (17h8m18.470266931s) [upload=17694]: Uploading segments... Aug 8 02:47:43.955233625 (17h8m18.470336207s) [upload=17694]: Got next segment. Inline: true Aug 8 02:47:43.955238449 (17h8m18.470341026s) [upload=17694]: Waiting for error group managing segments... Aug 8 02:47:43.955292283 (17h8m18.470394859s) [upload=17694]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:47:43.955296757 (17h8m18.470399334s) [upload=17694]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:47:43.955299998 (17h8m18.470402574s) [upload=17694]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:47:44.124192924 (17h8m18.639295505s) [upload=17694]: Done uploading segments... Aug 8 02:47:44.124209329 (17h8m18.639311892s) [upload=17694]: Upload finished. err: <nil> Aug 8 02:48:43.731612500 (17h9m18.246715079s) [upload=17714]: Starting upload Aug 8 02:48:43.731667994 (17h9m18.246770571s) [upload=17714]: Done starting upload Aug 8 02:48:43.731765653 (17h9m18.246868236s) [upload=17714]: Uploading segments... Aug 8 02:48:43.731812427 (17h9m18.246915014s) [upload=17714]: Got next segment. Inline: true Aug 8 02:48:43.731833486 (17h9m18.24693607s) [upload=17714]: Waiting for error group managing segments... Aug 8 02:48:43.731883643 (17h9m18.246986222s) [upload=17714]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:48:43.731888532 (17h9m18.246991109s) [upload=17714]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:48:43.731891882 (17h9m18.246994458s) [upload=17714]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:48:44.128878769 (17h9m18.643981354s) [upload=17714]: Done uploading segments... Aug 8 02:48:44.128904173 (17h9m18.644006818s) [upload=17714]: Upload finished. err: <nil> Aug 8 02:49:38.867826030 (17h10m13.382928597s) [upload=17730]: Starting upload Aug 8 02:49:38.867910640 (17h10m13.38301321s) [upload=17730]: Done starting upload Aug 8 02:49:38.868107732 (17h10m13.3832103s) [upload=17730]: Uploading segments... Aug 8 02:49:38.868198737 (17h10m13.383301322s) [upload=17730]: Got next segment. Inline: true Aug 8 02:49:38.868206330 (17h10m13.38330891s) [upload=17730]: Waiting for error group managing segments... Aug 8 02:49:38.868306560 (17h10m13.383409144s) [upload=17730]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:49:38.868323606 (17h10m13.383426205s) [upload=17730]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:49:38.868327207 (17h10m13.383429792s) [upload=17730]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:49:38.988012432 (17h10m13.503115008s) [upload=17730]: Done uploading segments... Aug 8 02:49:38.988031058 (17h10m13.503133625s) [upload=17730]: Upload finished. err: <nil> Aug 8 02:50:07.278685760 (17h10m41.793788343s) [upload=17739]: Starting upload Aug 8 02:50:07.278829106 (17h10m41.793931683s) [upload=17739]: Done starting upload Aug 8 02:50:07.278912448 (17h10m41.79401501s) [upload=17739]: Uploading segments... Aug 8 02:50:07.278959874 (17h10m41.79406246s) [upload=17739]: Got next segment. Inline: true Aug 8 02:50:07.278965894 (17h10m41.79406848s) [upload=17739]: Waiting for error group managing segments... Aug 8 02:50:07.279051579 (17h10m41.794154156s) [upload=17739]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:50:07.279058068 (17h10m41.794160653s) [upload=17739]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:50:07.279062000 (17h10m41.794164578s) [upload=17739]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:50:07.373452172 (17h10m41.88855475s) [upload=17739]: Done uploading segments... Aug 8 02:50:07.373481935 (17h10m41.888584519s) [upload=17739]: Upload finished. err: <nil> Aug 8 02:50:39.619819607 (17h11m14.134922187s) [upload=17749]: Starting upload Aug 8 02:50:39.619901137 (17h11m14.135003715s) [upload=17749]: Done starting upload Aug 8 02:50:39.620009105 (17h11m14.135111684s) [upload=17749]: Uploading segments... Aug 8 02:50:39.620061236 (17h11m14.135163825s) [upload=17749]: Got next segment. Inline: true Aug 8 02:50:39.620066774 (17h11m14.135169357s) [upload=17749]: Waiting for error group managing segments... Aug 8 02:50:39.620129403 (17h11m14.135231991s) [upload=17749]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:50:39.620134679 (17h11m14.135237257s) [upload=17749]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:50:39.620138422 (17h11m14.135241002s) [upload=17749]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:50:39.783502378 (17h11m14.298604959s) [upload=17749]: Done uploading segments... Aug 8 02:50:39.783519169 (17h11m14.298621746s) [upload=17749]: Upload finished. err: <nil> Aug 8 02:51:53.488163266 (17h12m28.003265854s) [upload=17769]: Starting upload Aug 8 02:51:53.488231606 (17h12m28.003334191s) [upload=17769]: Done starting upload Aug 8 02:51:53.488304051 (17h12m28.00340663s) [upload=17769]: Uploading segments... Aug 8 02:51:53.488381549 (17h12m28.003484136s) [upload=17769]: Got next segment. Inline: true Aug 8 02:51:53.488387602 (17h12m28.003490183s) [upload=17769]: Waiting for error group managing segments... Aug 8 02:51:53.488438849 (17h12m28.003541436s) [upload=17769]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:51:53.488445257 (17h12m28.003547838s) [upload=17769]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:51:53.488448976 (17h12m28.003551557s) [upload=17769]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:51:53.555883594 (17h12m28.070986169s) [upload=17769]: Done uploading segments... Aug 8 02:51:53.555899316 (17h12m28.071001892s) [upload=17769]: Upload finished. err: <nil> Aug 8 02:52:49.974130340 (17h13m24.489232916s) [upload=17782]: Starting upload Aug 8 02:52:49.974182339 (17h13m24.489284907s) [upload=17782]: Done starting upload Aug 8 02:52:49.974266182 (17h13m24.48936876s) [upload=17782]: Uploading segments... Aug 8 02:52:49.974303819 (17h13m24.489406465s) [upload=17782]: Got next segment. Inline: true Aug 8 02:52:49.974308907 (17h13m24.489411483s) [upload=17782]: Waiting for error group managing segments... Aug 8 02:52:49.974352082 (17h13m24.489454659s) [upload=17782]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:52:49.974356786 (17h13m24.489459362s) [upload=17782]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:52:49.974360363 (17h13m24.489462939s) [upload=17782]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:52:49.983269871 (17h13m24.498372455s) [upload=17782]: Done uploading segments... Aug 8 02:52:49.983289597 (17h13m24.498392175s) [upload=17782]: Upload finished. err: metaclient: Too Many Requests
Duration: 236.883069ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..] duration:
460.058283ms
FAIL (2025-08-08 02:58:52 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/4M")
======LOGS======
ad=17842]: Upload finished. err: <nil>
Aug 8 02:56:55.672801893 (17h17m30.187904459s) [upload=17854]: Starting upload
Aug 8 02:56:55.672849099 (17h17m30.187951667s) [upload=17854]: Done starting upload
Aug 8 02:56:55.672934546 (17h17m30.188037129s) [upload=17854]: Uploading segments...
Aug 8 02:56:55.673142624 (17h17m30.1882452s) [upload=17854]: Got next segment. Inline: false
Aug 8 02:56:55.673180963 (17h17m30.18828354s) [upload=17854]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 8 02:56:55.673185766 (17h17m30.188288335s) [upload=17854]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 8 02:56:55.817001710 (17h17m30.332104286s) [seg_pos={0 0}, upload=17854]: Begin upload segment...
Aug 8 02:56:55.817049073 (17h17m30.332151643s) [seg_pos={0 0}, upload=17854]: Done begin upload segment.
Aug 8 02:56:55.817272321 (17h17m30.3323749s) [noise=true , address=104.36.82.16:4310 , piece_id=25F4YPQTH2QEZ6DOLMDF2344TYDJL5JT2GQSMKY3VTUXO6ZX2ZYQ, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.817433916 (17h17m30.332536483s) [noise=true , address=104.36.82.5:4867 , piece_id=S3UA3ZI73DLUUNLDBS5RRMCMUGWDSI377AADY5D2F6APH4WFJRAQ, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.818127795 (17h17m30.333230364s) [noise=true , address=104.36.82.35:4599 , piece_id=Y4ZJOIICH7EKN34WRDBQWERFNVCRGCGPFTF5VYY5K45EXU5JVD6A, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.818952037 (17h17m30.334054608s) [noise=true , address=104.36.82.44:4875 , piece_id=NH5TIT5ST6MEGRB7YYJZM7I2HFBK4R2JR6LA55V3RXQ67CXITYXQ, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.819328753 (17h17m30.334431321s) [noise=true , address=104.36.82.56:4250 , piece_id=4SUTKVTMMTMNWW7B4O4NLVFIX4UYCVTCR6UWOVLAJUC7JHZBHNRA, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.819847891 (17h17m30.334950453s) [noise=true , address=104.36.82.60:4344 , piece_id=WZDEDH2CLCFPCIXGS36MXGM5SRZXLT754AAOOPIBOBJYYOI26IWQ, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.820277897 (17h17m30.33538046s) [noise=true , address=79.127.205.233:4650 , piece_id=BEVZPPYUQGQSBI334FN7I2ARNAIXDMIDXTHMQKYIKWTD2GWMIS7A, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.820670734 (17h17m30.335773297s) [noise=true , address=104.36.82.11:4137 , piece_id=IRDQQF52Z7GTWXITT3AD7YNMYQNRH73IZTXQYQ4LRM23ICQ24WDQ, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.821073724 (17h17m30.336176286s) [noise=true , address=45.91.116.7:4036 , piece_id=7UNVBZXGVPYGIWYE22HHF5DOZ3DJ2RQE5GA2S5YKP62FUTQ7YSGQ, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.821477247 (17h17m30.336579809s) [noise=true , address=104.36.82.55:4200 , piece_id=KRBC2DGBZ5BM6ZCPLJU4FARY2GXCVJKBPB266TK6P6DPSNSRA2EA, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.821956124 (17h17m30.33705869s) [noise=true , address=45.91.116.14:4200 , piece_id=RREGX2U6HK3KO66CXOIDIXAIJ2GXUD7O2HFUDKEKSANKP4LCEFFQ, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.822627361 (17h17m30.337729925s) [noise=true , address=104.36.82.23:4454 , piece_id=GNPYBDITM4PG56VEOBWS4YRTJ3M4CVXEKJ26SDKOOJEBSSSZ7CAA, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.823598424 (17h17m30.338700993s) [noise=true , address=104.36.82.46:4505 , piece_id=CHM2WFHCBUFYAJOFOX2XZ6P3KIBIIIJI3AYJOPVIATA2XQBV2KPA, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.824397787 (17h17m30.339500357s) [noise=true , address=104.36.82.62:4434 , piece_id=2EEQCBQECZHV5GYRD3U3T6MD5JK5WMXTSMY6DY4XJI5U4KAYRQ4Q, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.825235804 (17h17m30.340338373s) [noise=true , address=104.36.82.37:4673 , piece_id=WPUSW3NZVF6T3WWEVD3GMIV5S5EQBCJL5AQBYUDAVJXIOQMHRTUA, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.825350186 (17h17m30.340452757s) [noise=true , address=104.36.82.58:4282 , piece_id=KPAIL5BRZTRZOYWN3KN5QZSXM2JIO2A74UADAFFNI4Q6DR3SN4WA, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.825771286 (17h17m30.340873855s) [noise=true , address=104.36.82.44:4890 , piece_id=RIOGDT6QZQVDQPGC2LMA7D7X47KWOFAQSWTR3X73UVIY2QVY7RVQ, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.827131293 (17h17m30.342233862s) [noise=true , address=104.36.82.59:4303 , piece_id=YMQOCXNYW362BEHWCT4Y6FPBGZBNA2UAZLFG3TGIFJGTZVT32H7Q, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.932537806 (17h17m30.447640388s) [upload=17854]: Next returned nil segment
Aug 8 02:56:55.932554853 (17h17m30.447657422s) [upload=17854]: Waiting for error group managing segments...
Aug 8 02:56:55.937123715 (17h17m30.452226285s) [noise=true , address=45.91.116.7:4036 , piece_id=7UNVBZXGVPYGIWYE22HHF5DOZ3DJ2RQE5GA2S5YKP62FUTQ7YSGQ, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.940599216 (17h17m30.455701786s) [noise=true , address=104.36.82.37:4673 , piece_id=WPUSW3NZVF6T3WWEVD3GMIV5S5EQBCJL5AQBYUDAVJXIOQMHRTUA, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.941043576 (17h17m30.45614614s) [noise=true , address=104.36.82.55:4200 , piece_id=KRBC2DGBZ5BM6ZCPLJU4FARY2GXCVJKBPB266TK6P6DPSNSRA2EA, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.941482170 (17h17m30.456584748s) [noise=true , address=104.36.82.58:4282 , piece_id=KPAIL5BRZTRZOYWN3KN5QZSXM2JIO2A74UADAFFNI4Q6DR3SN4WA, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.942417232 (17h17m30.457519794s) [noise=true , address=104.36.82.16:4310 , piece_id=25F4YPQTH2QEZ6DOLMDF2344TYDJL5JT2GQSMKY3VTUXO6ZX2ZYQ, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.942947508 (17h17m30.458050078s) [noise=true , address=104.36.82.44:4890 , piece_id=RIOGDT6QZQVDQPGC2LMA7D7X47KWOFAQSWTR3X73UVIY2QVY7RVQ, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.943407533 (17h17m30.458510106s) [noise=true , address=104.36.82.62:4434 , piece_id=2EEQCBQECZHV5GYRD3U3T6MD5JK5WMXTSMY6DY4XJI5U4KAYRQ4Q, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.943536993 (17h17m30.45863957s) [noise=true , address=104.36.82.56:4250 , piece_id=4SUTKVTMMTMNWW7B4O4NLVFIX4UYCVTCR6UWOVLAJUC7JHZBHNRA, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.943985401 (17h17m30.459087978s) [noise=true , address=104.36.82.59:4303 , piece_id=YMQOCXNYW362BEHWCT4Y6FPBGZBNA2UAZLFG3TGIFJGTZVT32H7Q, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.944013966 (17h17m30.459116535s) [noise=true , address=104.36.82.5:4867 , piece_id=S3UA3ZI73DLUUNLDBS5RRMCMUGWDSI377AADY5D2F6APH4WFJRAQ, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.944504349 (17h17m30.459606918s) [noise=true , address=104.36.82.44:4875 , piece_id=NH5TIT5ST6MEGRB7YYJZM7I2HFBK4R2JR6LA55V3RXQ67CXITYXQ, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.944950961 (17h17m30.460053533s) [noise=true , address=104.36.82.35:4599 , piece_id=Y4ZJOIICH7EKN34WRDBQWERFNVCRGCGPFTF5VYY5K45EXU5JVD6A, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.945006095 (17h17m30.460108657s) [noise=true , address=104.36.82.46:4505 , piece_id=CHM2WFHCBUFYAJOFOX2XZ6P3KIBIIIJI3AYJOPVIATA2XQBV2KPA, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.945375636 (17h17m30.460478202s) [noise=true , address=104.36.82.23:4454 , piece_id=GNPYBDITM4PG56VEOBWS4YRTJ3M4CVXEKJ26SDKOOJEBSSSZ7CAA, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.945395444 (17h17m30.460498023s) [seg_pos={0 0}, upload=17854]: Segment reached optimal threshold of 14 pieces.
Aug 8 02:56:55.945509537 (17h17m30.4606121s) [noise=true , address=104.36.82.11:4137 , piece_id=IRDQQF52Z7GTWXITT3AD7YNMYQNRH73IZTXQYQ4LRM23ICQ24WDQ, seg_pos={0 0}, upload=17854]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12tkbodtFrNCuJbvZk8YnpSdJfVsKasNuKhg2wGD8rDkNPrHZ21): protocol: expected piece hash; context canceled; EOF
Aug 8 02:56:55.945551105 (17h17m30.460653674s) [noise=true , address=45.91.116.14:4200 , piece_id=RREGX2U6HK3KO66CXOIDIXAIJ2GXUD7O2HFUDKEKSANKP4LCEFFQ, seg_pos={0 0}, upload=17854]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fiQJkHYDv2J4V82doFX3dg43hLpgq3c34Eng3zwqFkF5znyx): protocol: expected piece hash; context canceled; EOF
Aug 8 02:56:55.945584503 (17h17m30.460687066s) [noise=true , address=104.36.82.60:4344 , piece_id=WZDEDH2CLCFPCIXGS36MXGM5SRZXLT754AAOOPIBOBJYYOI26IWQ, seg_pos={0 0}, upload=17854]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127WWD5hr1ZBkRLCHz3jb7kpqqnxPS6VuZ1MyQDmitFbhTSbJNK): protocol: expected piece hash; context canceled; EOF
Aug 8 02:56:55.945654567 (17h17m30.460757143s) [noise=true , address=79.127.205.233:4650 , piece_id=BEVZPPYUQGQSBI334FN7I2ARNAIXDMIDXTHMQKYIKWTD2GWMIS7A, seg_pos={0 0}, upload=17854]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hA5tXuKjGsjJCU9XLeNBXkkahm3FXqLWM83Fk3Hu1TH6rMgoh): protocol: context canceled; context canceled
Aug 8 02:56:55.945671410 (17h17m30.460773979s) [seg_pos={0 0}, upload=17854]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 8 02:56:55.945733549 (17h17m30.460836125s) [upload=17854]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 8 02:56:55.945737891 (17h17m30.460840467s) [upload=17854]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 8 02:56:56.198500182 (17h17m30.713602758s) [upload=17854]: Done uploading segments...
Aug 8 02:56:56.198516890 (17h17m30.713619466s) [upload=17854]: Upload finished. err: <nil>
Duration: 460.087492ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......] duration:
230.768541ms
FAIL (2025-08-08 02:58:28 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== m43.691381585s) [upload=17645]: Done uploading segments... Aug 8 02:45:09.176296212 (17h5m43.691398786s) [upload=17645]: Upload finished. err: <nil> Aug 8 02:46:04.444774241 (17h6m38.959876804s) [upload=17660]: Starting upload Aug 8 02:46:04.444862810 (17h6m38.959965425s) [upload=17660]: Done starting upload Aug 8 02:46:04.444937571 (17h6m38.96004014s) [upload=17660]: Uploading segments... Aug 8 02:46:04.445020035 (17h6m38.960122601s) [upload=17660]: Got next segment. Inline: true Aug 8 02:46:04.445028641 (17h6m38.960131209s) [upload=17660]: Waiting for error group managing segments... Aug 8 02:46:04.445085559 (17h6m38.960188125s) [upload=17660]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:46:04.445090655 (17h6m38.960193232s) [upload=17660]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:46:04.445094218 (17h6m38.960196793s) [upload=17660]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:46:04.519397453 (17h6m39.034500036s) [upload=17660]: Done uploading segments... Aug 8 02:46:04.519419431 (17h6m39.034522009s) [upload=17660]: Upload finished. err: <nil> Aug 8 02:46:29.848775596 (17h7m4.363878185s) [upload=17669]: Starting upload Aug 8 02:46:29.848845713 (17h7m4.363948289s) [upload=17669]: Done starting upload Aug 8 02:46:29.848924385 (17h7m4.364026954s) [upload=17669]: Uploading segments... Aug 8 02:46:29.848961594 (17h7m4.364064177s) [upload=17669]: Got next segment. Inline: true Aug 8 02:46:29.848968008 (17h7m4.364070587s) [upload=17669]: Waiting for error group managing segments... Aug 8 02:46:29.849009835 (17h7m4.364112411s) [upload=17669]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:46:29.849014457 (17h7m4.364117034s) [upload=17669]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:46:29.849018027 (17h7m4.364120603s) [upload=17669]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:46:29.914967479 (17h7m4.430070058s) [upload=17669]: Done uploading segments... Aug 8 02:46:29.914984083 (17h7m4.430086659s) [upload=17669]: Upload finished. err: <nil> Aug 8 02:47:07.009158285 (17h7m41.524260866s) [upload=17683]: Starting upload Aug 8 02:47:07.009236490 (17h7m41.524339073s) [upload=17683]: Done starting upload Aug 8 02:47:07.009381821 (17h7m41.5244844s) [upload=17683]: Uploading segments... Aug 8 02:47:07.009460862 (17h7m41.524563448s) [upload=17683]: Got next segment. Inline: true Aug 8 02:47:07.009467487 (17h7m41.524570067s) [upload=17683]: Waiting for error group managing segments... Aug 8 02:47:07.009557493 (17h7m41.524660055s) [upload=17683]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:47:07.009562936 (17h7m41.524665513s) [upload=17683]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:47:07.009566255 (17h7m41.524668832s) [upload=17683]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:47:07.108408364 (17h7m41.623510942s) [upload=17683]: Done uploading segments... Aug 8 02:47:07.108426523 (17h7m41.623529101s) [upload=17683]: Upload finished. err: <nil> Aug 8 02:47:43.954966497 (17h8m18.470069065s) [upload=17694]: Starting upload Aug 8 02:47:43.955070525 (17h8m18.470173105s) [upload=17694]: Done starting upload Aug 8 02:47:43.955164363 (17h8m18.470266931s) [upload=17694]: Uploading segments... Aug 8 02:47:43.955233625 (17h8m18.470336207s) [upload=17694]: Got next segment. Inline: true Aug 8 02:47:43.955238449 (17h8m18.470341026s) [upload=17694]: Waiting for error group managing segments... Aug 8 02:47:43.955292283 (17h8m18.470394859s) [upload=17694]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:47:43.955296757 (17h8m18.470399334s) [upload=17694]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:47:43.955299998 (17h8m18.470402574s) [upload=17694]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:47:44.124192924 (17h8m18.639295505s) [upload=17694]: Done uploading segments... Aug 8 02:47:44.124209329 (17h8m18.639311892s) [upload=17694]: Upload finished. err: <nil> Aug 8 02:48:43.731612500 (17h9m18.246715079s) [upload=17714]: Starting upload Aug 8 02:48:43.731667994 (17h9m18.246770571s) [upload=17714]: Done starting upload Aug 8 02:48:43.731765653 (17h9m18.246868236s) [upload=17714]: Uploading segments... Aug 8 02:48:43.731812427 (17h9m18.246915014s) [upload=17714]: Got next segment. Inline: true Aug 8 02:48:43.731833486 (17h9m18.24693607s) [upload=17714]: Waiting for error group managing segments... Aug 8 02:48:43.731883643 (17h9m18.246986222s) [upload=17714]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:48:43.731888532 (17h9m18.246991109s) [upload=17714]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:48:43.731891882 (17h9m18.246994458s) [upload=17714]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:48:44.128878769 (17h9m18.643981354s) [upload=17714]: Done uploading segments... Aug 8 02:48:44.128904173 (17h9m18.644006818s) [upload=17714]: Upload finished. err: <nil> Aug 8 02:49:38.867826030 (17h10m13.382928597s) [upload=17730]: Starting upload Aug 8 02:49:38.867910640 (17h10m13.38301321s) [upload=17730]: Done starting upload Aug 8 02:49:38.868107732 (17h10m13.3832103s) [upload=17730]: Uploading segments... Aug 8 02:49:38.868198737 (17h10m13.383301322s) [upload=17730]: Got next segment. Inline: true Aug 8 02:49:38.868206330 (17h10m13.38330891s) [upload=17730]: Waiting for error group managing segments... Aug 8 02:49:38.868306560 (17h10m13.383409144s) [upload=17730]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:49:38.868323606 (17h10m13.383426205s) [upload=17730]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:49:38.868327207 (17h10m13.383429792s) [upload=17730]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:49:38.988012432 (17h10m13.503115008s) [upload=17730]: Done uploading segments... Aug 8 02:49:38.988031058 (17h10m13.503133625s) [upload=17730]: Upload finished. err: <nil> Aug 8 02:50:07.278685760 (17h10m41.793788343s) [upload=17739]: Starting upload Aug 8 02:50:07.278829106 (17h10m41.793931683s) [upload=17739]: Done starting upload Aug 8 02:50:07.278912448 (17h10m41.79401501s) [upload=17739]: Uploading segments... Aug 8 02:50:07.278959874 (17h10m41.79406246s) [upload=17739]: Got next segment. Inline: true Aug 8 02:50:07.278965894 (17h10m41.79406848s) [upload=17739]: Waiting for error group managing segments... Aug 8 02:50:07.279051579 (17h10m41.794154156s) [upload=17739]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:50:07.279058068 (17h10m41.794160653s) [upload=17739]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:50:07.279062000 (17h10m41.794164578s) [upload=17739]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:50:07.373452172 (17h10m41.88855475s) [upload=17739]: Done uploading segments... Aug 8 02:50:07.373481935 (17h10m41.888584519s) [upload=17739]: Upload finished. err: <nil> Aug 8 02:50:39.619819607 (17h11m14.134922187s) [upload=17749]: Starting upload Aug 8 02:50:39.619901137 (17h11m14.135003715s) [upload=17749]: Done starting upload Aug 8 02:50:39.620009105 (17h11m14.135111684s) [upload=17749]: Uploading segments... Aug 8 02:50:39.620061236 (17h11m14.135163825s) [upload=17749]: Got next segment. Inline: true Aug 8 02:50:39.620066774 (17h11m14.135169357s) [upload=17749]: Waiting for error group managing segments... Aug 8 02:50:39.620129403 (17h11m14.135231991s) [upload=17749]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:50:39.620134679 (17h11m14.135237257s) [upload=17749]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:50:39.620138422 (17h11m14.135241002s) [upload=17749]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:50:39.783502378 (17h11m14.298604959s) [upload=17749]: Done uploading segments... Aug 8 02:50:39.783519169 (17h11m14.298621746s) [upload=17749]: Upload finished. err: <nil> Aug 8 02:51:53.488163266 (17h12m28.003265854s) [upload=17769]: Starting upload Aug 8 02:51:53.488231606 (17h12m28.003334191s) [upload=17769]: Done starting upload Aug 8 02:51:53.488304051 (17h12m28.00340663s) [upload=17769]: Uploading segments... Aug 8 02:51:53.488381549 (17h12m28.003484136s) [upload=17769]: Got next segment. Inline: true Aug 8 02:51:53.488387602 (17h12m28.003490183s) [upload=17769]: Waiting for error group managing segments... Aug 8 02:51:53.488438849 (17h12m28.003541436s) [upload=17769]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:51:53.488445257 (17h12m28.003547838s) [upload=17769]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:51:53.488448976 (17h12m28.003551557s) [upload=17769]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:51:53.555883594 (17h12m28.070986169s) [upload=17769]: Done uploading segments... Aug 8 02:51:53.555899316 (17h12m28.071001892s) [upload=17769]: Upload finished. err: <nil> Aug 8 02:52:49.974130340 (17h13m24.489232916s) [upload=17782]: Starting upload Aug 8 02:52:49.974182339 (17h13m24.489284907s) [upload=17782]: Done starting upload Aug 8 02:52:49.974266182 (17h13m24.48936876s) [upload=17782]: Uploading segments... Aug 8 02:52:49.974303819 (17h13m24.489406465s) [upload=17782]: Got next segment. Inline: true Aug 8 02:52:49.974308907 (17h13m24.489411483s) [upload=17782]: Waiting for error group managing segments... Aug 8 02:52:49.974352082 (17h13m24.489454659s) [upload=17782]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:52:49.974356786 (17h13m24.489459362s) [upload=17782]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:52:49.974360363 (17h13m24.489462939s) [upload=17782]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:52:49.983269871 (17h13m24.498372455s) [upload=17782]: Done uploading segments... Aug 8 02:52:49.983289597 (17h13m24.498392175s) [upload=17782]: Upload finished. err: metaclient: Too Many Requests
Duration: 230.809631ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.] duration:
502.381124ms
FAIL (2025-08-08 02:57:43 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/4M")
======LOGS======
ad=17842]: Upload finished. err: <nil>
Aug 8 02:56:55.672801893 (17h17m30.187904459s) [upload=17854]: Starting upload
Aug 8 02:56:55.672849099 (17h17m30.187951667s) [upload=17854]: Done starting upload
Aug 8 02:56:55.672934546 (17h17m30.188037129s) [upload=17854]: Uploading segments...
Aug 8 02:56:55.673142624 (17h17m30.1882452s) [upload=17854]: Got next segment. Inline: false
Aug 8 02:56:55.673180963 (17h17m30.18828354s) [upload=17854]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 8 02:56:55.673185766 (17h17m30.188288335s) [upload=17854]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 8 02:56:55.817001710 (17h17m30.332104286s) [seg_pos={0 0}, upload=17854]: Begin upload segment...
Aug 8 02:56:55.817049073 (17h17m30.332151643s) [seg_pos={0 0}, upload=17854]: Done begin upload segment.
Aug 8 02:56:55.817272321 (17h17m30.3323749s) [noise=true , address=104.36.82.16:4310 , piece_id=25F4YPQTH2QEZ6DOLMDF2344TYDJL5JT2GQSMKY3VTUXO6ZX2ZYQ, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.817433916 (17h17m30.332536483s) [noise=true , address=104.36.82.5:4867 , piece_id=S3UA3ZI73DLUUNLDBS5RRMCMUGWDSI377AADY5D2F6APH4WFJRAQ, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.818127795 (17h17m30.333230364s) [noise=true , address=104.36.82.35:4599 , piece_id=Y4ZJOIICH7EKN34WRDBQWERFNVCRGCGPFTF5VYY5K45EXU5JVD6A, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.818952037 (17h17m30.334054608s) [noise=true , address=104.36.82.44:4875 , piece_id=NH5TIT5ST6MEGRB7YYJZM7I2HFBK4R2JR6LA55V3RXQ67CXITYXQ, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.819328753 (17h17m30.334431321s) [noise=true , address=104.36.82.56:4250 , piece_id=4SUTKVTMMTMNWW7B4O4NLVFIX4UYCVTCR6UWOVLAJUC7JHZBHNRA, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.819847891 (17h17m30.334950453s) [noise=true , address=104.36.82.60:4344 , piece_id=WZDEDH2CLCFPCIXGS36MXGM5SRZXLT754AAOOPIBOBJYYOI26IWQ, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.820277897 (17h17m30.33538046s) [noise=true , address=79.127.205.233:4650 , piece_id=BEVZPPYUQGQSBI334FN7I2ARNAIXDMIDXTHMQKYIKWTD2GWMIS7A, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.820670734 (17h17m30.335773297s) [noise=true , address=104.36.82.11:4137 , piece_id=IRDQQF52Z7GTWXITT3AD7YNMYQNRH73IZTXQYQ4LRM23ICQ24WDQ, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.821073724 (17h17m30.336176286s) [noise=true , address=45.91.116.7:4036 , piece_id=7UNVBZXGVPYGIWYE22HHF5DOZ3DJ2RQE5GA2S5YKP62FUTQ7YSGQ, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.821477247 (17h17m30.336579809s) [noise=true , address=104.36.82.55:4200 , piece_id=KRBC2DGBZ5BM6ZCPLJU4FARY2GXCVJKBPB266TK6P6DPSNSRA2EA, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.821956124 (17h17m30.33705869s) [noise=true , address=45.91.116.14:4200 , piece_id=RREGX2U6HK3KO66CXOIDIXAIJ2GXUD7O2HFUDKEKSANKP4LCEFFQ, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.822627361 (17h17m30.337729925s) [noise=true , address=104.36.82.23:4454 , piece_id=GNPYBDITM4PG56VEOBWS4YRTJ3M4CVXEKJ26SDKOOJEBSSSZ7CAA, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.823598424 (17h17m30.338700993s) [noise=true , address=104.36.82.46:4505 , piece_id=CHM2WFHCBUFYAJOFOX2XZ6P3KIBIIIJI3AYJOPVIATA2XQBV2KPA, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.824397787 (17h17m30.339500357s) [noise=true , address=104.36.82.62:4434 , piece_id=2EEQCBQECZHV5GYRD3U3T6MD5JK5WMXTSMY6DY4XJI5U4KAYRQ4Q, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.825235804 (17h17m30.340338373s) [noise=true , address=104.36.82.37:4673 , piece_id=WPUSW3NZVF6T3WWEVD3GMIV5S5EQBCJL5AQBYUDAVJXIOQMHRTUA, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.825350186 (17h17m30.340452757s) [noise=true , address=104.36.82.58:4282 , piece_id=KPAIL5BRZTRZOYWN3KN5QZSXM2JIO2A74UADAFFNI4Q6DR3SN4WA, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.825771286 (17h17m30.340873855s) [noise=true , address=104.36.82.44:4890 , piece_id=RIOGDT6QZQVDQPGC2LMA7D7X47KWOFAQSWTR3X73UVIY2QVY7RVQ, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.827131293 (17h17m30.342233862s) [noise=true , address=104.36.82.59:4303 , piece_id=YMQOCXNYW362BEHWCT4Y6FPBGZBNA2UAZLFG3TGIFJGTZVT32H7Q, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.932537806 (17h17m30.447640388s) [upload=17854]: Next returned nil segment
Aug 8 02:56:55.932554853 (17h17m30.447657422s) [upload=17854]: Waiting for error group managing segments...
Aug 8 02:56:55.937123715 (17h17m30.452226285s) [noise=true , address=45.91.116.7:4036 , piece_id=7UNVBZXGVPYGIWYE22HHF5DOZ3DJ2RQE5GA2S5YKP62FUTQ7YSGQ, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.940599216 (17h17m30.455701786s) [noise=true , address=104.36.82.37:4673 , piece_id=WPUSW3NZVF6T3WWEVD3GMIV5S5EQBCJL5AQBYUDAVJXIOQMHRTUA, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.941043576 (17h17m30.45614614s) [noise=true , address=104.36.82.55:4200 , piece_id=KRBC2DGBZ5BM6ZCPLJU4FARY2GXCVJKBPB266TK6P6DPSNSRA2EA, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.941482170 (17h17m30.456584748s) [noise=true , address=104.36.82.58:4282 , piece_id=KPAIL5BRZTRZOYWN3KN5QZSXM2JIO2A74UADAFFNI4Q6DR3SN4WA, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.942417232 (17h17m30.457519794s) [noise=true , address=104.36.82.16:4310 , piece_id=25F4YPQTH2QEZ6DOLMDF2344TYDJL5JT2GQSMKY3VTUXO6ZX2ZYQ, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.942947508 (17h17m30.458050078s) [noise=true , address=104.36.82.44:4890 , piece_id=RIOGDT6QZQVDQPGC2LMA7D7X47KWOFAQSWTR3X73UVIY2QVY7RVQ, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.943407533 (17h17m30.458510106s) [noise=true , address=104.36.82.62:4434 , piece_id=2EEQCBQECZHV5GYRD3U3T6MD5JK5WMXTSMY6DY4XJI5U4KAYRQ4Q, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.943536993 (17h17m30.45863957s) [noise=true , address=104.36.82.56:4250 , piece_id=4SUTKVTMMTMNWW7B4O4NLVFIX4UYCVTCR6UWOVLAJUC7JHZBHNRA, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.943985401 (17h17m30.459087978s) [noise=true , address=104.36.82.59:4303 , piece_id=YMQOCXNYW362BEHWCT4Y6FPBGZBNA2UAZLFG3TGIFJGTZVT32H7Q, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.944013966 (17h17m30.459116535s) [noise=true , address=104.36.82.5:4867 , piece_id=S3UA3ZI73DLUUNLDBS5RRMCMUGWDSI377AADY5D2F6APH4WFJRAQ, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.944504349 (17h17m30.459606918s) [noise=true , address=104.36.82.44:4875 , piece_id=NH5TIT5ST6MEGRB7YYJZM7I2HFBK4R2JR6LA55V3RXQ67CXITYXQ, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.944950961 (17h17m30.460053533s) [noise=true , address=104.36.82.35:4599 , piece_id=Y4ZJOIICH7EKN34WRDBQWERFNVCRGCGPFTF5VYY5K45EXU5JVD6A, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.945006095 (17h17m30.460108657s) [noise=true , address=104.36.82.46:4505 , piece_id=CHM2WFHCBUFYAJOFOX2XZ6P3KIBIIIJI3AYJOPVIATA2XQBV2KPA, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.945375636 (17h17m30.460478202s) [noise=true , address=104.36.82.23:4454 , piece_id=GNPYBDITM4PG56VEOBWS4YRTJ3M4CVXEKJ26SDKOOJEBSSSZ7CAA, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.945395444 (17h17m30.460498023s) [seg_pos={0 0}, upload=17854]: Segment reached optimal threshold of 14 pieces.
Aug 8 02:56:55.945509537 (17h17m30.4606121s) [noise=true , address=104.36.82.11:4137 , piece_id=IRDQQF52Z7GTWXITT3AD7YNMYQNRH73IZTXQYQ4LRM23ICQ24WDQ, seg_pos={0 0}, upload=17854]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12tkbodtFrNCuJbvZk8YnpSdJfVsKasNuKhg2wGD8rDkNPrHZ21): protocol: expected piece hash; context canceled; EOF
Aug 8 02:56:55.945551105 (17h17m30.460653674s) [noise=true , address=45.91.116.14:4200 , piece_id=RREGX2U6HK3KO66CXOIDIXAIJ2GXUD7O2HFUDKEKSANKP4LCEFFQ, seg_pos={0 0}, upload=17854]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fiQJkHYDv2J4V82doFX3dg43hLpgq3c34Eng3zwqFkF5znyx): protocol: expected piece hash; context canceled; EOF
Aug 8 02:56:55.945584503 (17h17m30.460687066s) [noise=true , address=104.36.82.60:4344 , piece_id=WZDEDH2CLCFPCIXGS36MXGM5SRZXLT754AAOOPIBOBJYYOI26IWQ, seg_pos={0 0}, upload=17854]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127WWD5hr1ZBkRLCHz3jb7kpqqnxPS6VuZ1MyQDmitFbhTSbJNK): protocol: expected piece hash; context canceled; EOF
Aug 8 02:56:55.945654567 (17h17m30.460757143s) [noise=true , address=79.127.205.233:4650 , piece_id=BEVZPPYUQGQSBI334FN7I2ARNAIXDMIDXTHMQKYIKWTD2GWMIS7A, seg_pos={0 0}, upload=17854]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hA5tXuKjGsjJCU9XLeNBXkkahm3FXqLWM83Fk3Hu1TH6rMgoh): protocol: context canceled; context canceled
Aug 8 02:56:55.945671410 (17h17m30.460773979s) [seg_pos={0 0}, upload=17854]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 8 02:56:55.945733549 (17h17m30.460836125s) [upload=17854]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 8 02:56:55.945737891 (17h17m30.460840467s) [upload=17854]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 8 02:56:56.198500182 (17h17m30.713602758s) [upload=17854]: Done uploading segments...
Aug 8 02:56:56.198516890 (17h17m30.713619466s) [upload=17854]: Upload finished. err: <nil>
Duration: 502.407758ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......] duration:
635.379013ms
FAIL (2025-08-08 02:57:24 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== m43.691381585s) [upload=17645]: Done uploading segments... Aug 8 02:45:09.176296212 (17h5m43.691398786s) [upload=17645]: Upload finished. err: <nil> Aug 8 02:46:04.444774241 (17h6m38.959876804s) [upload=17660]: Starting upload Aug 8 02:46:04.444862810 (17h6m38.959965425s) [upload=17660]: Done starting upload Aug 8 02:46:04.444937571 (17h6m38.96004014s) [upload=17660]: Uploading segments... Aug 8 02:46:04.445020035 (17h6m38.960122601s) [upload=17660]: Got next segment. Inline: true Aug 8 02:46:04.445028641 (17h6m38.960131209s) [upload=17660]: Waiting for error group managing segments... Aug 8 02:46:04.445085559 (17h6m38.960188125s) [upload=17660]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:46:04.445090655 (17h6m38.960193232s) [upload=17660]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:46:04.445094218 (17h6m38.960196793s) [upload=17660]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:46:04.519397453 (17h6m39.034500036s) [upload=17660]: Done uploading segments... Aug 8 02:46:04.519419431 (17h6m39.034522009s) [upload=17660]: Upload finished. err: <nil> Aug 8 02:46:29.848775596 (17h7m4.363878185s) [upload=17669]: Starting upload Aug 8 02:46:29.848845713 (17h7m4.363948289s) [upload=17669]: Done starting upload Aug 8 02:46:29.848924385 (17h7m4.364026954s) [upload=17669]: Uploading segments... Aug 8 02:46:29.848961594 (17h7m4.364064177s) [upload=17669]: Got next segment. Inline: true Aug 8 02:46:29.848968008 (17h7m4.364070587s) [upload=17669]: Waiting for error group managing segments... Aug 8 02:46:29.849009835 (17h7m4.364112411s) [upload=17669]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:46:29.849014457 (17h7m4.364117034s) [upload=17669]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:46:29.849018027 (17h7m4.364120603s) [upload=17669]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:46:29.914967479 (17h7m4.430070058s) [upload=17669]: Done uploading segments... Aug 8 02:46:29.914984083 (17h7m4.430086659s) [upload=17669]: Upload finished. err: <nil> Aug 8 02:47:07.009158285 (17h7m41.524260866s) [upload=17683]: Starting upload Aug 8 02:47:07.009236490 (17h7m41.524339073s) [upload=17683]: Done starting upload Aug 8 02:47:07.009381821 (17h7m41.5244844s) [upload=17683]: Uploading segments... Aug 8 02:47:07.009460862 (17h7m41.524563448s) [upload=17683]: Got next segment. Inline: true Aug 8 02:47:07.009467487 (17h7m41.524570067s) [upload=17683]: Waiting for error group managing segments... Aug 8 02:47:07.009557493 (17h7m41.524660055s) [upload=17683]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:47:07.009562936 (17h7m41.524665513s) [upload=17683]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:47:07.009566255 (17h7m41.524668832s) [upload=17683]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:47:07.108408364 (17h7m41.623510942s) [upload=17683]: Done uploading segments... Aug 8 02:47:07.108426523 (17h7m41.623529101s) [upload=17683]: Upload finished. err: <nil> Aug 8 02:47:43.954966497 (17h8m18.470069065s) [upload=17694]: Starting upload Aug 8 02:47:43.955070525 (17h8m18.470173105s) [upload=17694]: Done starting upload Aug 8 02:47:43.955164363 (17h8m18.470266931s) [upload=17694]: Uploading segments... Aug 8 02:47:43.955233625 (17h8m18.470336207s) [upload=17694]: Got next segment. Inline: true Aug 8 02:47:43.955238449 (17h8m18.470341026s) [upload=17694]: Waiting for error group managing segments... Aug 8 02:47:43.955292283 (17h8m18.470394859s) [upload=17694]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:47:43.955296757 (17h8m18.470399334s) [upload=17694]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:47:43.955299998 (17h8m18.470402574s) [upload=17694]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:47:44.124192924 (17h8m18.639295505s) [upload=17694]: Done uploading segments... Aug 8 02:47:44.124209329 (17h8m18.639311892s) [upload=17694]: Upload finished. err: <nil> Aug 8 02:48:43.731612500 (17h9m18.246715079s) [upload=17714]: Starting upload Aug 8 02:48:43.731667994 (17h9m18.246770571s) [upload=17714]: Done starting upload Aug 8 02:48:43.731765653 (17h9m18.246868236s) [upload=17714]: Uploading segments... Aug 8 02:48:43.731812427 (17h9m18.246915014s) [upload=17714]: Got next segment. Inline: true Aug 8 02:48:43.731833486 (17h9m18.24693607s) [upload=17714]: Waiting for error group managing segments... Aug 8 02:48:43.731883643 (17h9m18.246986222s) [upload=17714]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:48:43.731888532 (17h9m18.246991109s) [upload=17714]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:48:43.731891882 (17h9m18.246994458s) [upload=17714]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:48:44.128878769 (17h9m18.643981354s) [upload=17714]: Done uploading segments... Aug 8 02:48:44.128904173 (17h9m18.644006818s) [upload=17714]: Upload finished. err: <nil> Aug 8 02:49:38.867826030 (17h10m13.382928597s) [upload=17730]: Starting upload Aug 8 02:49:38.867910640 (17h10m13.38301321s) [upload=17730]: Done starting upload Aug 8 02:49:38.868107732 (17h10m13.3832103s) [upload=17730]: Uploading segments... Aug 8 02:49:38.868198737 (17h10m13.383301322s) [upload=17730]: Got next segment. Inline: true Aug 8 02:49:38.868206330 (17h10m13.38330891s) [upload=17730]: Waiting for error group managing segments... Aug 8 02:49:38.868306560 (17h10m13.383409144s) [upload=17730]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:49:38.868323606 (17h10m13.383426205s) [upload=17730]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:49:38.868327207 (17h10m13.383429792s) [upload=17730]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:49:38.988012432 (17h10m13.503115008s) [upload=17730]: Done uploading segments... Aug 8 02:49:38.988031058 (17h10m13.503133625s) [upload=17730]: Upload finished. err: <nil> Aug 8 02:50:07.278685760 (17h10m41.793788343s) [upload=17739]: Starting upload Aug 8 02:50:07.278829106 (17h10m41.793931683s) [upload=17739]: Done starting upload Aug 8 02:50:07.278912448 (17h10m41.79401501s) [upload=17739]: Uploading segments... Aug 8 02:50:07.278959874 (17h10m41.79406246s) [upload=17739]: Got next segment. Inline: true Aug 8 02:50:07.278965894 (17h10m41.79406848s) [upload=17739]: Waiting for error group managing segments... Aug 8 02:50:07.279051579 (17h10m41.794154156s) [upload=17739]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:50:07.279058068 (17h10m41.794160653s) [upload=17739]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:50:07.279062000 (17h10m41.794164578s) [upload=17739]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:50:07.373452172 (17h10m41.88855475s) [upload=17739]: Done uploading segments... Aug 8 02:50:07.373481935 (17h10m41.888584519s) [upload=17739]: Upload finished. err: <nil> Aug 8 02:50:39.619819607 (17h11m14.134922187s) [upload=17749]: Starting upload Aug 8 02:50:39.619901137 (17h11m14.135003715s) [upload=17749]: Done starting upload Aug 8 02:50:39.620009105 (17h11m14.135111684s) [upload=17749]: Uploading segments... Aug 8 02:50:39.620061236 (17h11m14.135163825s) [upload=17749]: Got next segment. Inline: true Aug 8 02:50:39.620066774 (17h11m14.135169357s) [upload=17749]: Waiting for error group managing segments... Aug 8 02:50:39.620129403 (17h11m14.135231991s) [upload=17749]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:50:39.620134679 (17h11m14.135237257s) [upload=17749]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:50:39.620138422 (17h11m14.135241002s) [upload=17749]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:50:39.783502378 (17h11m14.298604959s) [upload=17749]: Done uploading segments... Aug 8 02:50:39.783519169 (17h11m14.298621746s) [upload=17749]: Upload finished. err: <nil> Aug 8 02:51:53.488163266 (17h12m28.003265854s) [upload=17769]: Starting upload Aug 8 02:51:53.488231606 (17h12m28.003334191s) [upload=17769]: Done starting upload Aug 8 02:51:53.488304051 (17h12m28.00340663s) [upload=17769]: Uploading segments... Aug 8 02:51:53.488381549 (17h12m28.003484136s) [upload=17769]: Got next segment. Inline: true Aug 8 02:51:53.488387602 (17h12m28.003490183s) [upload=17769]: Waiting for error group managing segments... Aug 8 02:51:53.488438849 (17h12m28.003541436s) [upload=17769]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:51:53.488445257 (17h12m28.003547838s) [upload=17769]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:51:53.488448976 (17h12m28.003551557s) [upload=17769]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:51:53.555883594 (17h12m28.070986169s) [upload=17769]: Done uploading segments... Aug 8 02:51:53.555899316 (17h12m28.071001892s) [upload=17769]: Upload finished. err: <nil> Aug 8 02:52:49.974130340 (17h13m24.489232916s) [upload=17782]: Starting upload Aug 8 02:52:49.974182339 (17h13m24.489284907s) [upload=17782]: Done starting upload Aug 8 02:52:49.974266182 (17h13m24.48936876s) [upload=17782]: Uploading segments... Aug 8 02:52:49.974303819 (17h13m24.489406465s) [upload=17782]: Got next segment. Inline: true Aug 8 02:52:49.974308907 (17h13m24.489411483s) [upload=17782]: Waiting for error group managing segments... Aug 8 02:52:49.974352082 (17h13m24.489454659s) [upload=17782]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:52:49.974356786 (17h13m24.489459362s) [upload=17782]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:52:49.974360363 (17h13m24.489462939s) [upload=17782]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:52:49.983269871 (17h13m24.498372455s) [upload=17782]: Done uploading segments... Aug 8 02:52:49.983289597 (17h13m24.498392175s) [upload=17782]: Upload finished. err: metaclient: Too Many Requests
Duration: 635.40579ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
704.812415ms
FAIL (2025-08-08 02:56:56 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/4M")
======LOGS======
ad=17842]: Upload finished. err: <nil>
Aug 8 02:56:55.672801893 (17h17m30.187904459s) [upload=17854]: Starting upload
Aug 8 02:56:55.672849099 (17h17m30.187951667s) [upload=17854]: Done starting upload
Aug 8 02:56:55.672934546 (17h17m30.188037129s) [upload=17854]: Uploading segments...
Aug 8 02:56:55.673142624 (17h17m30.1882452s) [upload=17854]: Got next segment. Inline: false
Aug 8 02:56:55.673180963 (17h17m30.18828354s) [upload=17854]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 8 02:56:55.673185766 (17h17m30.188288335s) [upload=17854]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 8 02:56:55.817001710 (17h17m30.332104286s) [seg_pos={0 0}, upload=17854]: Begin upload segment...
Aug 8 02:56:55.817049073 (17h17m30.332151643s) [seg_pos={0 0}, upload=17854]: Done begin upload segment.
Aug 8 02:56:55.817272321 (17h17m30.3323749s) [noise=true , address=104.36.82.16:4310 , piece_id=25F4YPQTH2QEZ6DOLMDF2344TYDJL5JT2GQSMKY3VTUXO6ZX2ZYQ, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.817433916 (17h17m30.332536483s) [noise=true , address=104.36.82.5:4867 , piece_id=S3UA3ZI73DLUUNLDBS5RRMCMUGWDSI377AADY5D2F6APH4WFJRAQ, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.818127795 (17h17m30.333230364s) [noise=true , address=104.36.82.35:4599 , piece_id=Y4ZJOIICH7EKN34WRDBQWERFNVCRGCGPFTF5VYY5K45EXU5JVD6A, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.818952037 (17h17m30.334054608s) [noise=true , address=104.36.82.44:4875 , piece_id=NH5TIT5ST6MEGRB7YYJZM7I2HFBK4R2JR6LA55V3RXQ67CXITYXQ, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.819328753 (17h17m30.334431321s) [noise=true , address=104.36.82.56:4250 , piece_id=4SUTKVTMMTMNWW7B4O4NLVFIX4UYCVTCR6UWOVLAJUC7JHZBHNRA, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.819847891 (17h17m30.334950453s) [noise=true , address=104.36.82.60:4344 , piece_id=WZDEDH2CLCFPCIXGS36MXGM5SRZXLT754AAOOPIBOBJYYOI26IWQ, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.820277897 (17h17m30.33538046s) [noise=true , address=79.127.205.233:4650 , piece_id=BEVZPPYUQGQSBI334FN7I2ARNAIXDMIDXTHMQKYIKWTD2GWMIS7A, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.820670734 (17h17m30.335773297s) [noise=true , address=104.36.82.11:4137 , piece_id=IRDQQF52Z7GTWXITT3AD7YNMYQNRH73IZTXQYQ4LRM23ICQ24WDQ, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.821073724 (17h17m30.336176286s) [noise=true , address=45.91.116.7:4036 , piece_id=7UNVBZXGVPYGIWYE22HHF5DOZ3DJ2RQE5GA2S5YKP62FUTQ7YSGQ, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.821477247 (17h17m30.336579809s) [noise=true , address=104.36.82.55:4200 , piece_id=KRBC2DGBZ5BM6ZCPLJU4FARY2GXCVJKBPB266TK6P6DPSNSRA2EA, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.821956124 (17h17m30.33705869s) [noise=true , address=45.91.116.14:4200 , piece_id=RREGX2U6HK3KO66CXOIDIXAIJ2GXUD7O2HFUDKEKSANKP4LCEFFQ, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.822627361 (17h17m30.337729925s) [noise=true , address=104.36.82.23:4454 , piece_id=GNPYBDITM4PG56VEOBWS4YRTJ3M4CVXEKJ26SDKOOJEBSSSZ7CAA, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.823598424 (17h17m30.338700993s) [noise=true , address=104.36.82.46:4505 , piece_id=CHM2WFHCBUFYAJOFOX2XZ6P3KIBIIIJI3AYJOPVIATA2XQBV2KPA, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.824397787 (17h17m30.339500357s) [noise=true , address=104.36.82.62:4434 , piece_id=2EEQCBQECZHV5GYRD3U3T6MD5JK5WMXTSMY6DY4XJI5U4KAYRQ4Q, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.825235804 (17h17m30.340338373s) [noise=true , address=104.36.82.37:4673 , piece_id=WPUSW3NZVF6T3WWEVD3GMIV5S5EQBCJL5AQBYUDAVJXIOQMHRTUA, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.825350186 (17h17m30.340452757s) [noise=true , address=104.36.82.58:4282 , piece_id=KPAIL5BRZTRZOYWN3KN5QZSXM2JIO2A74UADAFFNI4Q6DR3SN4WA, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.825771286 (17h17m30.340873855s) [noise=true , address=104.36.82.44:4890 , piece_id=RIOGDT6QZQVDQPGC2LMA7D7X47KWOFAQSWTR3X73UVIY2QVY7RVQ, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.827131293 (17h17m30.342233862s) [noise=true , address=104.36.82.59:4303 , piece_id=YMQOCXNYW362BEHWCT4Y6FPBGZBNA2UAZLFG3TGIFJGTZVT32H7Q, seg_pos={0 0}, upload=17854]: Uploading piece...
Aug 8 02:56:55.932537806 (17h17m30.447640388s) [upload=17854]: Next returned nil segment
Aug 8 02:56:55.932554853 (17h17m30.447657422s) [upload=17854]: Waiting for error group managing segments...
Aug 8 02:56:55.937123715 (17h17m30.452226285s) [noise=true , address=45.91.116.7:4036 , piece_id=7UNVBZXGVPYGIWYE22HHF5DOZ3DJ2RQE5GA2S5YKP62FUTQ7YSGQ, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.940599216 (17h17m30.455701786s) [noise=true , address=104.36.82.37:4673 , piece_id=WPUSW3NZVF6T3WWEVD3GMIV5S5EQBCJL5AQBYUDAVJXIOQMHRTUA, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.941043576 (17h17m30.45614614s) [noise=true , address=104.36.82.55:4200 , piece_id=KRBC2DGBZ5BM6ZCPLJU4FARY2GXCVJKBPB266TK6P6DPSNSRA2EA, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.941482170 (17h17m30.456584748s) [noise=true , address=104.36.82.58:4282 , piece_id=KPAIL5BRZTRZOYWN3KN5QZSXM2JIO2A74UADAFFNI4Q6DR3SN4WA, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.942417232 (17h17m30.457519794s) [noise=true , address=104.36.82.16:4310 , piece_id=25F4YPQTH2QEZ6DOLMDF2344TYDJL5JT2GQSMKY3VTUXO6ZX2ZYQ, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.942947508 (17h17m30.458050078s) [noise=true , address=104.36.82.44:4890 , piece_id=RIOGDT6QZQVDQPGC2LMA7D7X47KWOFAQSWTR3X73UVIY2QVY7RVQ, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.943407533 (17h17m30.458510106s) [noise=true , address=104.36.82.62:4434 , piece_id=2EEQCBQECZHV5GYRD3U3T6MD5JK5WMXTSMY6DY4XJI5U4KAYRQ4Q, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.943536993 (17h17m30.45863957s) [noise=true , address=104.36.82.56:4250 , piece_id=4SUTKVTMMTMNWW7B4O4NLVFIX4UYCVTCR6UWOVLAJUC7JHZBHNRA, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.943985401 (17h17m30.459087978s) [noise=true , address=104.36.82.59:4303 , piece_id=YMQOCXNYW362BEHWCT4Y6FPBGZBNA2UAZLFG3TGIFJGTZVT32H7Q, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.944013966 (17h17m30.459116535s) [noise=true , address=104.36.82.5:4867 , piece_id=S3UA3ZI73DLUUNLDBS5RRMCMUGWDSI377AADY5D2F6APH4WFJRAQ, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.944504349 (17h17m30.459606918s) [noise=true , address=104.36.82.44:4875 , piece_id=NH5TIT5ST6MEGRB7YYJZM7I2HFBK4R2JR6LA55V3RXQ67CXITYXQ, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.944950961 (17h17m30.460053533s) [noise=true , address=104.36.82.35:4599 , piece_id=Y4ZJOIICH7EKN34WRDBQWERFNVCRGCGPFTF5VYY5K45EXU5JVD6A, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.945006095 (17h17m30.460108657s) [noise=true , address=104.36.82.46:4505 , piece_id=CHM2WFHCBUFYAJOFOX2XZ6P3KIBIIIJI3AYJOPVIATA2XQBV2KPA, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.945375636 (17h17m30.460478202s) [noise=true , address=104.36.82.23:4454 , piece_id=GNPYBDITM4PG56VEOBWS4YRTJ3M4CVXEKJ26SDKOOJEBSSSZ7CAA, seg_pos={0 0}, upload=17854]: Done uploading piece. err: <nil>
Aug 8 02:56:55.945395444 (17h17m30.460498023s) [seg_pos={0 0}, upload=17854]: Segment reached optimal threshold of 14 pieces.
Aug 8 02:56:55.945509537 (17h17m30.4606121s) [noise=true , address=104.36.82.11:4137 , piece_id=IRDQQF52Z7GTWXITT3AD7YNMYQNRH73IZTXQYQ4LRM23ICQ24WDQ, seg_pos={0 0}, upload=17854]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12tkbodtFrNCuJbvZk8YnpSdJfVsKasNuKhg2wGD8rDkNPrHZ21): protocol: expected piece hash; context canceled; EOF
Aug 8 02:56:55.945551105 (17h17m30.460653674s) [noise=true , address=45.91.116.14:4200 , piece_id=RREGX2U6HK3KO66CXOIDIXAIJ2GXUD7O2HFUDKEKSANKP4LCEFFQ, seg_pos={0 0}, upload=17854]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fiQJkHYDv2J4V82doFX3dg43hLpgq3c34Eng3zwqFkF5znyx): protocol: expected piece hash; context canceled; EOF
Aug 8 02:56:55.945584503 (17h17m30.460687066s) [noise=true , address=104.36.82.60:4344 , piece_id=WZDEDH2CLCFPCIXGS36MXGM5SRZXLT754AAOOPIBOBJYYOI26IWQ, seg_pos={0 0}, upload=17854]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127WWD5hr1ZBkRLCHz3jb7kpqqnxPS6VuZ1MyQDmitFbhTSbJNK): protocol: expected piece hash; context canceled; EOF
Aug 8 02:56:55.945654567 (17h17m30.460757143s) [noise=true , address=79.127.205.233:4650 , piece_id=BEVZPPYUQGQSBI334FN7I2ARNAIXDMIDXTHMQKYIKWTD2GWMIS7A, seg_pos={0 0}, upload=17854]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hA5tXuKjGsjJCU9XLeNBXkkahm3FXqLWM83Fk3Hu1TH6rMgoh): protocol: context canceled; context canceled
Aug 8 02:56:55.945671410 (17h17m30.460773979s) [seg_pos={0 0}, upload=17854]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 8 02:56:55.945733549 (17h17m30.460836125s) [upload=17854]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 8 02:56:55.945737891 (17h17m30.460840467s) [upload=17854]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 8 02:56:56.198500182 (17h17m30.713602758s) [upload=17854]: Done uploading segments...
Aug 8 02:56:56.198516890 (17h17m30.713619466s) [upload=17854]: Upload finished. err: <nil>
Duration: 704.83513ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....] duration:
641.273658ms
FAIL (2025-08-08 02:56:51 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== m43.691381585s) [upload=17645]: Done uploading segments... Aug 8 02:45:09.176296212 (17h5m43.691398786s) [upload=17645]: Upload finished. err: <nil> Aug 8 02:46:04.444774241 (17h6m38.959876804s) [upload=17660]: Starting upload Aug 8 02:46:04.444862810 (17h6m38.959965425s) [upload=17660]: Done starting upload Aug 8 02:46:04.444937571 (17h6m38.96004014s) [upload=17660]: Uploading segments... Aug 8 02:46:04.445020035 (17h6m38.960122601s) [upload=17660]: Got next segment. Inline: true Aug 8 02:46:04.445028641 (17h6m38.960131209s) [upload=17660]: Waiting for error group managing segments... Aug 8 02:46:04.445085559 (17h6m38.960188125s) [upload=17660]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:46:04.445090655 (17h6m38.960193232s) [upload=17660]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:46:04.445094218 (17h6m38.960196793s) [upload=17660]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:46:04.519397453 (17h6m39.034500036s) [upload=17660]: Done uploading segments... Aug 8 02:46:04.519419431 (17h6m39.034522009s) [upload=17660]: Upload finished. err: <nil> Aug 8 02:46:29.848775596 (17h7m4.363878185s) [upload=17669]: Starting upload Aug 8 02:46:29.848845713 (17h7m4.363948289s) [upload=17669]: Done starting upload Aug 8 02:46:29.848924385 (17h7m4.364026954s) [upload=17669]: Uploading segments... Aug 8 02:46:29.848961594 (17h7m4.364064177s) [upload=17669]: Got next segment. Inline: true Aug 8 02:46:29.848968008 (17h7m4.364070587s) [upload=17669]: Waiting for error group managing segments... Aug 8 02:46:29.849009835 (17h7m4.364112411s) [upload=17669]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:46:29.849014457 (17h7m4.364117034s) [upload=17669]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:46:29.849018027 (17h7m4.364120603s) [upload=17669]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:46:29.914967479 (17h7m4.430070058s) [upload=17669]: Done uploading segments... Aug 8 02:46:29.914984083 (17h7m4.430086659s) [upload=17669]: Upload finished. err: <nil> Aug 8 02:47:07.009158285 (17h7m41.524260866s) [upload=17683]: Starting upload Aug 8 02:47:07.009236490 (17h7m41.524339073s) [upload=17683]: Done starting upload Aug 8 02:47:07.009381821 (17h7m41.5244844s) [upload=17683]: Uploading segments... Aug 8 02:47:07.009460862 (17h7m41.524563448s) [upload=17683]: Got next segment. Inline: true Aug 8 02:47:07.009467487 (17h7m41.524570067s) [upload=17683]: Waiting for error group managing segments... Aug 8 02:47:07.009557493 (17h7m41.524660055s) [upload=17683]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:47:07.009562936 (17h7m41.524665513s) [upload=17683]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:47:07.009566255 (17h7m41.524668832s) [upload=17683]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:47:07.108408364 (17h7m41.623510942s) [upload=17683]: Done uploading segments... Aug 8 02:47:07.108426523 (17h7m41.623529101s) [upload=17683]: Upload finished. err: <nil> Aug 8 02:47:43.954966497 (17h8m18.470069065s) [upload=17694]: Starting upload Aug 8 02:47:43.955070525 (17h8m18.470173105s) [upload=17694]: Done starting upload Aug 8 02:47:43.955164363 (17h8m18.470266931s) [upload=17694]: Uploading segments... Aug 8 02:47:43.955233625 (17h8m18.470336207s) [upload=17694]: Got next segment. Inline: true Aug 8 02:47:43.955238449 (17h8m18.470341026s) [upload=17694]: Waiting for error group managing segments... Aug 8 02:47:43.955292283 (17h8m18.470394859s) [upload=17694]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:47:43.955296757 (17h8m18.470399334s) [upload=17694]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:47:43.955299998 (17h8m18.470402574s) [upload=17694]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:47:44.124192924 (17h8m18.639295505s) [upload=17694]: Done uploading segments... Aug 8 02:47:44.124209329 (17h8m18.639311892s) [upload=17694]: Upload finished. err: <nil> Aug 8 02:48:43.731612500 (17h9m18.246715079s) [upload=17714]: Starting upload Aug 8 02:48:43.731667994 (17h9m18.246770571s) [upload=17714]: Done starting upload Aug 8 02:48:43.731765653 (17h9m18.246868236s) [upload=17714]: Uploading segments... Aug 8 02:48:43.731812427 (17h9m18.246915014s) [upload=17714]: Got next segment. Inline: true Aug 8 02:48:43.731833486 (17h9m18.24693607s) [upload=17714]: Waiting for error group managing segments... Aug 8 02:48:43.731883643 (17h9m18.246986222s) [upload=17714]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:48:43.731888532 (17h9m18.246991109s) [upload=17714]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:48:43.731891882 (17h9m18.246994458s) [upload=17714]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:48:44.128878769 (17h9m18.643981354s) [upload=17714]: Done uploading segments... Aug 8 02:48:44.128904173 (17h9m18.644006818s) [upload=17714]: Upload finished. err: <nil> Aug 8 02:49:38.867826030 (17h10m13.382928597s) [upload=17730]: Starting upload Aug 8 02:49:38.867910640 (17h10m13.38301321s) [upload=17730]: Done starting upload Aug 8 02:49:38.868107732 (17h10m13.3832103s) [upload=17730]: Uploading segments... Aug 8 02:49:38.868198737 (17h10m13.383301322s) [upload=17730]: Got next segment. Inline: true Aug 8 02:49:38.868206330 (17h10m13.38330891s) [upload=17730]: Waiting for error group managing segments... Aug 8 02:49:38.868306560 (17h10m13.383409144s) [upload=17730]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:49:38.868323606 (17h10m13.383426205s) [upload=17730]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:49:38.868327207 (17h10m13.383429792s) [upload=17730]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:49:38.988012432 (17h10m13.503115008s) [upload=17730]: Done uploading segments... Aug 8 02:49:38.988031058 (17h10m13.503133625s) [upload=17730]: Upload finished. err: <nil> Aug 8 02:50:07.278685760 (17h10m41.793788343s) [upload=17739]: Starting upload Aug 8 02:50:07.278829106 (17h10m41.793931683s) [upload=17739]: Done starting upload Aug 8 02:50:07.278912448 (17h10m41.79401501s) [upload=17739]: Uploading segments... Aug 8 02:50:07.278959874 (17h10m41.79406246s) [upload=17739]: Got next segment. Inline: true Aug 8 02:50:07.278965894 (17h10m41.79406848s) [upload=17739]: Waiting for error group managing segments... Aug 8 02:50:07.279051579 (17h10m41.794154156s) [upload=17739]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:50:07.279058068 (17h10m41.794160653s) [upload=17739]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:50:07.279062000 (17h10m41.794164578s) [upload=17739]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:50:07.373452172 (17h10m41.88855475s) [upload=17739]: Done uploading segments... Aug 8 02:50:07.373481935 (17h10m41.888584519s) [upload=17739]: Upload finished. err: <nil> Aug 8 02:50:39.619819607 (17h11m14.134922187s) [upload=17749]: Starting upload Aug 8 02:50:39.619901137 (17h11m14.135003715s) [upload=17749]: Done starting upload Aug 8 02:50:39.620009105 (17h11m14.135111684s) [upload=17749]: Uploading segments... Aug 8 02:50:39.620061236 (17h11m14.135163825s) [upload=17749]: Got next segment. Inline: true Aug 8 02:50:39.620066774 (17h11m14.135169357s) [upload=17749]: Waiting for error group managing segments... Aug 8 02:50:39.620129403 (17h11m14.135231991s) [upload=17749]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:50:39.620134679 (17h11m14.135237257s) [upload=17749]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:50:39.620138422 (17h11m14.135241002s) [upload=17749]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:50:39.783502378 (17h11m14.298604959s) [upload=17749]: Done uploading segments... Aug 8 02:50:39.783519169 (17h11m14.298621746s) [upload=17749]: Upload finished. err: <nil> Aug 8 02:51:53.488163266 (17h12m28.003265854s) [upload=17769]: Starting upload Aug 8 02:51:53.488231606 (17h12m28.003334191s) [upload=17769]: Done starting upload Aug 8 02:51:53.488304051 (17h12m28.00340663s) [upload=17769]: Uploading segments... Aug 8 02:51:53.488381549 (17h12m28.003484136s) [upload=17769]: Got next segment. Inline: true Aug 8 02:51:53.488387602 (17h12m28.003490183s) [upload=17769]: Waiting for error group managing segments... Aug 8 02:51:53.488438849 (17h12m28.003541436s) [upload=17769]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:51:53.488445257 (17h12m28.003547838s) [upload=17769]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:51:53.488448976 (17h12m28.003551557s) [upload=17769]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:51:53.555883594 (17h12m28.070986169s) [upload=17769]: Done uploading segments... Aug 8 02:51:53.555899316 (17h12m28.071001892s) [upload=17769]: Upload finished. err: <nil> Aug 8 02:52:49.974130340 (17h13m24.489232916s) [upload=17782]: Starting upload Aug 8 02:52:49.974182339 (17h13m24.489284907s) [upload=17782]: Done starting upload Aug 8 02:52:49.974266182 (17h13m24.48936876s) [upload=17782]: Uploading segments... Aug 8 02:52:49.974303819 (17h13m24.489406465s) [upload=17782]: Got next segment. Inline: true Aug 8 02:52:49.974308907 (17h13m24.489411483s) [upload=17782]: Waiting for error group managing segments... Aug 8 02:52:49.974352082 (17h13m24.489454659s) [upload=17782]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:52:49.974356786 (17h13m24.489459362s) [upload=17782]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:52:49.974360363 (17h13m24.489462939s) [upload=17782]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:52:49.983269871 (17h13m24.498372455s) [upload=17782]: Done uploading segments... Aug 8 02:52:49.983289597 (17h13m24.498392175s) [upload=17782]: Upload finished. err: metaclient: Too Many Requests
Duration: 641.300353ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....] duration:
196.989852ms
FAIL (2025-08-08 02:55:45 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== m43.691381585s) [upload=17645]: Done uploading segments... Aug 8 02:45:09.176296212 (17h5m43.691398786s) [upload=17645]: Upload finished. err: <nil> Aug 8 02:46:04.444774241 (17h6m38.959876804s) [upload=17660]: Starting upload Aug 8 02:46:04.444862810 (17h6m38.959965425s) [upload=17660]: Done starting upload Aug 8 02:46:04.444937571 (17h6m38.96004014s) [upload=17660]: Uploading segments... Aug 8 02:46:04.445020035 (17h6m38.960122601s) [upload=17660]: Got next segment. Inline: true Aug 8 02:46:04.445028641 (17h6m38.960131209s) [upload=17660]: Waiting for error group managing segments... Aug 8 02:46:04.445085559 (17h6m38.960188125s) [upload=17660]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:46:04.445090655 (17h6m38.960193232s) [upload=17660]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:46:04.445094218 (17h6m38.960196793s) [upload=17660]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:46:04.519397453 (17h6m39.034500036s) [upload=17660]: Done uploading segments... Aug 8 02:46:04.519419431 (17h6m39.034522009s) [upload=17660]: Upload finished. err: <nil> Aug 8 02:46:29.848775596 (17h7m4.363878185s) [upload=17669]: Starting upload Aug 8 02:46:29.848845713 (17h7m4.363948289s) [upload=17669]: Done starting upload Aug 8 02:46:29.848924385 (17h7m4.364026954s) [upload=17669]: Uploading segments... Aug 8 02:46:29.848961594 (17h7m4.364064177s) [upload=17669]: Got next segment. Inline: true Aug 8 02:46:29.848968008 (17h7m4.364070587s) [upload=17669]: Waiting for error group managing segments... Aug 8 02:46:29.849009835 (17h7m4.364112411s) [upload=17669]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:46:29.849014457 (17h7m4.364117034s) [upload=17669]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:46:29.849018027 (17h7m4.364120603s) [upload=17669]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:46:29.914967479 (17h7m4.430070058s) [upload=17669]: Done uploading segments... Aug 8 02:46:29.914984083 (17h7m4.430086659s) [upload=17669]: Upload finished. err: <nil> Aug 8 02:47:07.009158285 (17h7m41.524260866s) [upload=17683]: Starting upload Aug 8 02:47:07.009236490 (17h7m41.524339073s) [upload=17683]: Done starting upload Aug 8 02:47:07.009381821 (17h7m41.5244844s) [upload=17683]: Uploading segments... Aug 8 02:47:07.009460862 (17h7m41.524563448s) [upload=17683]: Got next segment. Inline: true Aug 8 02:47:07.009467487 (17h7m41.524570067s) [upload=17683]: Waiting for error group managing segments... Aug 8 02:47:07.009557493 (17h7m41.524660055s) [upload=17683]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:47:07.009562936 (17h7m41.524665513s) [upload=17683]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:47:07.009566255 (17h7m41.524668832s) [upload=17683]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:47:07.108408364 (17h7m41.623510942s) [upload=17683]: Done uploading segments... Aug 8 02:47:07.108426523 (17h7m41.623529101s) [upload=17683]: Upload finished. err: <nil> Aug 8 02:47:43.954966497 (17h8m18.470069065s) [upload=17694]: Starting upload Aug 8 02:47:43.955070525 (17h8m18.470173105s) [upload=17694]: Done starting upload Aug 8 02:47:43.955164363 (17h8m18.470266931s) [upload=17694]: Uploading segments... Aug 8 02:47:43.955233625 (17h8m18.470336207s) [upload=17694]: Got next segment. Inline: true Aug 8 02:47:43.955238449 (17h8m18.470341026s) [upload=17694]: Waiting for error group managing segments... Aug 8 02:47:43.955292283 (17h8m18.470394859s) [upload=17694]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:47:43.955296757 (17h8m18.470399334s) [upload=17694]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:47:43.955299998 (17h8m18.470402574s) [upload=17694]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:47:44.124192924 (17h8m18.639295505s) [upload=17694]: Done uploading segments... Aug 8 02:47:44.124209329 (17h8m18.639311892s) [upload=17694]: Upload finished. err: <nil> Aug 8 02:48:43.731612500 (17h9m18.246715079s) [upload=17714]: Starting upload Aug 8 02:48:43.731667994 (17h9m18.246770571s) [upload=17714]: Done starting upload Aug 8 02:48:43.731765653 (17h9m18.246868236s) [upload=17714]: Uploading segments... Aug 8 02:48:43.731812427 (17h9m18.246915014s) [upload=17714]: Got next segment. Inline: true Aug 8 02:48:43.731833486 (17h9m18.24693607s) [upload=17714]: Waiting for error group managing segments... Aug 8 02:48:43.731883643 (17h9m18.246986222s) [upload=17714]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:48:43.731888532 (17h9m18.246991109s) [upload=17714]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:48:43.731891882 (17h9m18.246994458s) [upload=17714]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:48:44.128878769 (17h9m18.643981354s) [upload=17714]: Done uploading segments... Aug 8 02:48:44.128904173 (17h9m18.644006818s) [upload=17714]: Upload finished. err: <nil> Aug 8 02:49:38.867826030 (17h10m13.382928597s) [upload=17730]: Starting upload Aug 8 02:49:38.867910640 (17h10m13.38301321s) [upload=17730]: Done starting upload Aug 8 02:49:38.868107732 (17h10m13.3832103s) [upload=17730]: Uploading segments... Aug 8 02:49:38.868198737 (17h10m13.383301322s) [upload=17730]: Got next segment. Inline: true Aug 8 02:49:38.868206330 (17h10m13.38330891s) [upload=17730]: Waiting for error group managing segments... Aug 8 02:49:38.868306560 (17h10m13.383409144s) [upload=17730]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:49:38.868323606 (17h10m13.383426205s) [upload=17730]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:49:38.868327207 (17h10m13.383429792s) [upload=17730]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:49:38.988012432 (17h10m13.503115008s) [upload=17730]: Done uploading segments... Aug 8 02:49:38.988031058 (17h10m13.503133625s) [upload=17730]: Upload finished. err: <nil> Aug 8 02:50:07.278685760 (17h10m41.793788343s) [upload=17739]: Starting upload Aug 8 02:50:07.278829106 (17h10m41.793931683s) [upload=17739]: Done starting upload Aug 8 02:50:07.278912448 (17h10m41.79401501s) [upload=17739]: Uploading segments... Aug 8 02:50:07.278959874 (17h10m41.79406246s) [upload=17739]: Got next segment. Inline: true Aug 8 02:50:07.278965894 (17h10m41.79406848s) [upload=17739]: Waiting for error group managing segments... Aug 8 02:50:07.279051579 (17h10m41.794154156s) [upload=17739]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:50:07.279058068 (17h10m41.794160653s) [upload=17739]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:50:07.279062000 (17h10m41.794164578s) [upload=17739]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:50:07.373452172 (17h10m41.88855475s) [upload=17739]: Done uploading segments... Aug 8 02:50:07.373481935 (17h10m41.888584519s) [upload=17739]: Upload finished. err: <nil> Aug 8 02:50:39.619819607 (17h11m14.134922187s) [upload=17749]: Starting upload Aug 8 02:50:39.619901137 (17h11m14.135003715s) [upload=17749]: Done starting upload Aug 8 02:50:39.620009105 (17h11m14.135111684s) [upload=17749]: Uploading segments... Aug 8 02:50:39.620061236 (17h11m14.135163825s) [upload=17749]: Got next segment. Inline: true Aug 8 02:50:39.620066774 (17h11m14.135169357s) [upload=17749]: Waiting for error group managing segments... Aug 8 02:50:39.620129403 (17h11m14.135231991s) [upload=17749]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:50:39.620134679 (17h11m14.135237257s) [upload=17749]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:50:39.620138422 (17h11m14.135241002s) [upload=17749]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:50:39.783502378 (17h11m14.298604959s) [upload=17749]: Done uploading segments... Aug 8 02:50:39.783519169 (17h11m14.298621746s) [upload=17749]: Upload finished. err: <nil> Aug 8 02:51:53.488163266 (17h12m28.003265854s) [upload=17769]: Starting upload Aug 8 02:51:53.488231606 (17h12m28.003334191s) [upload=17769]: Done starting upload Aug 8 02:51:53.488304051 (17h12m28.00340663s) [upload=17769]: Uploading segments... Aug 8 02:51:53.488381549 (17h12m28.003484136s) [upload=17769]: Got next segment. Inline: true Aug 8 02:51:53.488387602 (17h12m28.003490183s) [upload=17769]: Waiting for error group managing segments... Aug 8 02:51:53.488438849 (17h12m28.003541436s) [upload=17769]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:51:53.488445257 (17h12m28.003547838s) [upload=17769]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:51:53.488448976 (17h12m28.003551557s) [upload=17769]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:51:53.555883594 (17h12m28.070986169s) [upload=17769]: Done uploading segments... Aug 8 02:51:53.555899316 (17h12m28.071001892s) [upload=17769]: Upload finished. err: <nil> Aug 8 02:52:49.974130340 (17h13m24.489232916s) [upload=17782]: Starting upload Aug 8 02:52:49.974182339 (17h13m24.489284907s) [upload=17782]: Done starting upload Aug 8 02:52:49.974266182 (17h13m24.48936876s) [upload=17782]: Uploading segments... Aug 8 02:52:49.974303819 (17h13m24.489406465s) [upload=17782]: Got next segment. Inline: true Aug 8 02:52:49.974308907 (17h13m24.489411483s) [upload=17782]: Waiting for error group managing segments... Aug 8 02:52:49.974352082 (17h13m24.489454659s) [upload=17782]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:52:49.974356786 (17h13m24.489459362s) [upload=17782]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:52:49.974360363 (17h13m24.489462939s) [upload=17782]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:52:49.983269871 (17h13m24.498372455s) [upload=17782]: Done uploading segments... Aug 8 02:52:49.983289597 (17h13m24.498392175s) [upload=17782]: Upload finished. err: metaclient: Too Many Requests
Duration: 197.035017ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...] duration:
213.695545ms
FAIL (2025-08-08 02:55:24 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== m43.691381585s) [upload=17645]: Done uploading segments... Aug 8 02:45:09.176296212 (17h5m43.691398786s) [upload=17645]: Upload finished. err: <nil> Aug 8 02:46:04.444774241 (17h6m38.959876804s) [upload=17660]: Starting upload Aug 8 02:46:04.444862810 (17h6m38.959965425s) [upload=17660]: Done starting upload Aug 8 02:46:04.444937571 (17h6m38.96004014s) [upload=17660]: Uploading segments... Aug 8 02:46:04.445020035 (17h6m38.960122601s) [upload=17660]: Got next segment. Inline: true Aug 8 02:46:04.445028641 (17h6m38.960131209s) [upload=17660]: Waiting for error group managing segments... Aug 8 02:46:04.445085559 (17h6m38.960188125s) [upload=17660]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:46:04.445090655 (17h6m38.960193232s) [upload=17660]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:46:04.445094218 (17h6m38.960196793s) [upload=17660]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:46:04.519397453 (17h6m39.034500036s) [upload=17660]: Done uploading segments... Aug 8 02:46:04.519419431 (17h6m39.034522009s) [upload=17660]: Upload finished. err: <nil> Aug 8 02:46:29.848775596 (17h7m4.363878185s) [upload=17669]: Starting upload Aug 8 02:46:29.848845713 (17h7m4.363948289s) [upload=17669]: Done starting upload Aug 8 02:46:29.848924385 (17h7m4.364026954s) [upload=17669]: Uploading segments... Aug 8 02:46:29.848961594 (17h7m4.364064177s) [upload=17669]: Got next segment. Inline: true Aug 8 02:46:29.848968008 (17h7m4.364070587s) [upload=17669]: Waiting for error group managing segments... Aug 8 02:46:29.849009835 (17h7m4.364112411s) [upload=17669]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:46:29.849014457 (17h7m4.364117034s) [upload=17669]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:46:29.849018027 (17h7m4.364120603s) [upload=17669]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:46:29.914967479 (17h7m4.430070058s) [upload=17669]: Done uploading segments... Aug 8 02:46:29.914984083 (17h7m4.430086659s) [upload=17669]: Upload finished. err: <nil> Aug 8 02:47:07.009158285 (17h7m41.524260866s) [upload=17683]: Starting upload Aug 8 02:47:07.009236490 (17h7m41.524339073s) [upload=17683]: Done starting upload Aug 8 02:47:07.009381821 (17h7m41.5244844s) [upload=17683]: Uploading segments... Aug 8 02:47:07.009460862 (17h7m41.524563448s) [upload=17683]: Got next segment. Inline: true Aug 8 02:47:07.009467487 (17h7m41.524570067s) [upload=17683]: Waiting for error group managing segments... Aug 8 02:47:07.009557493 (17h7m41.524660055s) [upload=17683]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:47:07.009562936 (17h7m41.524665513s) [upload=17683]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:47:07.009566255 (17h7m41.524668832s) [upload=17683]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:47:07.108408364 (17h7m41.623510942s) [upload=17683]: Done uploading segments... Aug 8 02:47:07.108426523 (17h7m41.623529101s) [upload=17683]: Upload finished. err: <nil> Aug 8 02:47:43.954966497 (17h8m18.470069065s) [upload=17694]: Starting upload Aug 8 02:47:43.955070525 (17h8m18.470173105s) [upload=17694]: Done starting upload Aug 8 02:47:43.955164363 (17h8m18.470266931s) [upload=17694]: Uploading segments... Aug 8 02:47:43.955233625 (17h8m18.470336207s) [upload=17694]: Got next segment. Inline: true Aug 8 02:47:43.955238449 (17h8m18.470341026s) [upload=17694]: Waiting for error group managing segments... Aug 8 02:47:43.955292283 (17h8m18.470394859s) [upload=17694]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:47:43.955296757 (17h8m18.470399334s) [upload=17694]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:47:43.955299998 (17h8m18.470402574s) [upload=17694]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:47:44.124192924 (17h8m18.639295505s) [upload=17694]: Done uploading segments... Aug 8 02:47:44.124209329 (17h8m18.639311892s) [upload=17694]: Upload finished. err: <nil> Aug 8 02:48:43.731612500 (17h9m18.246715079s) [upload=17714]: Starting upload Aug 8 02:48:43.731667994 (17h9m18.246770571s) [upload=17714]: Done starting upload Aug 8 02:48:43.731765653 (17h9m18.246868236s) [upload=17714]: Uploading segments... Aug 8 02:48:43.731812427 (17h9m18.246915014s) [upload=17714]: Got next segment. Inline: true Aug 8 02:48:43.731833486 (17h9m18.24693607s) [upload=17714]: Waiting for error group managing segments... Aug 8 02:48:43.731883643 (17h9m18.246986222s) [upload=17714]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:48:43.731888532 (17h9m18.246991109s) [upload=17714]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:48:43.731891882 (17h9m18.246994458s) [upload=17714]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:48:44.128878769 (17h9m18.643981354s) [upload=17714]: Done uploading segments... Aug 8 02:48:44.128904173 (17h9m18.644006818s) [upload=17714]: Upload finished. err: <nil> Aug 8 02:49:38.867826030 (17h10m13.382928597s) [upload=17730]: Starting upload Aug 8 02:49:38.867910640 (17h10m13.38301321s) [upload=17730]: Done starting upload Aug 8 02:49:38.868107732 (17h10m13.3832103s) [upload=17730]: Uploading segments... Aug 8 02:49:38.868198737 (17h10m13.383301322s) [upload=17730]: Got next segment. Inline: true Aug 8 02:49:38.868206330 (17h10m13.38330891s) [upload=17730]: Waiting for error group managing segments... Aug 8 02:49:38.868306560 (17h10m13.383409144s) [upload=17730]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:49:38.868323606 (17h10m13.383426205s) [upload=17730]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:49:38.868327207 (17h10m13.383429792s) [upload=17730]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:49:38.988012432 (17h10m13.503115008s) [upload=17730]: Done uploading segments... Aug 8 02:49:38.988031058 (17h10m13.503133625s) [upload=17730]: Upload finished. err: <nil> Aug 8 02:50:07.278685760 (17h10m41.793788343s) [upload=17739]: Starting upload Aug 8 02:50:07.278829106 (17h10m41.793931683s) [upload=17739]: Done starting upload Aug 8 02:50:07.278912448 (17h10m41.79401501s) [upload=17739]: Uploading segments... Aug 8 02:50:07.278959874 (17h10m41.79406246s) [upload=17739]: Got next segment. Inline: true Aug 8 02:50:07.278965894 (17h10m41.79406848s) [upload=17739]: Waiting for error group managing segments... Aug 8 02:50:07.279051579 (17h10m41.794154156s) [upload=17739]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:50:07.279058068 (17h10m41.794160653s) [upload=17739]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:50:07.279062000 (17h10m41.794164578s) [upload=17739]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:50:07.373452172 (17h10m41.88855475s) [upload=17739]: Done uploading segments... Aug 8 02:50:07.373481935 (17h10m41.888584519s) [upload=17739]: Upload finished. err: <nil> Aug 8 02:50:39.619819607 (17h11m14.134922187s) [upload=17749]: Starting upload Aug 8 02:50:39.619901137 (17h11m14.135003715s) [upload=17749]: Done starting upload Aug 8 02:50:39.620009105 (17h11m14.135111684s) [upload=17749]: Uploading segments... Aug 8 02:50:39.620061236 (17h11m14.135163825s) [upload=17749]: Got next segment. Inline: true Aug 8 02:50:39.620066774 (17h11m14.135169357s) [upload=17749]: Waiting for error group managing segments... Aug 8 02:50:39.620129403 (17h11m14.135231991s) [upload=17749]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:50:39.620134679 (17h11m14.135237257s) [upload=17749]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:50:39.620138422 (17h11m14.135241002s) [upload=17749]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:50:39.783502378 (17h11m14.298604959s) [upload=17749]: Done uploading segments... Aug 8 02:50:39.783519169 (17h11m14.298621746s) [upload=17749]: Upload finished. err: <nil> Aug 8 02:51:53.488163266 (17h12m28.003265854s) [upload=17769]: Starting upload Aug 8 02:51:53.488231606 (17h12m28.003334191s) [upload=17769]: Done starting upload Aug 8 02:51:53.488304051 (17h12m28.00340663s) [upload=17769]: Uploading segments... Aug 8 02:51:53.488381549 (17h12m28.003484136s) [upload=17769]: Got next segment. Inline: true Aug 8 02:51:53.488387602 (17h12m28.003490183s) [upload=17769]: Waiting for error group managing segments... Aug 8 02:51:53.488438849 (17h12m28.003541436s) [upload=17769]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:51:53.488445257 (17h12m28.003547838s) [upload=17769]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:51:53.488448976 (17h12m28.003551557s) [upload=17769]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:51:53.555883594 (17h12m28.070986169s) [upload=17769]: Done uploading segments... Aug 8 02:51:53.555899316 (17h12m28.071001892s) [upload=17769]: Upload finished. err: <nil> Aug 8 02:52:49.974130340 (17h13m24.489232916s) [upload=17782]: Starting upload Aug 8 02:52:49.974182339 (17h13m24.489284907s) [upload=17782]: Done starting upload Aug 8 02:52:49.974266182 (17h13m24.48936876s) [upload=17782]: Uploading segments... Aug 8 02:52:49.974303819 (17h13m24.489406465s) [upload=17782]: Got next segment. Inline: true Aug 8 02:52:49.974308907 (17h13m24.489411483s) [upload=17782]: Waiting for error group managing segments... Aug 8 02:52:49.974352082 (17h13m24.489454659s) [upload=17782]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:52:49.974356786 (17h13m24.489459362s) [upload=17782]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:52:49.974360363 (17h13m24.489462939s) [upload=17782]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:52:49.983269871 (17h13m24.498372455s) [upload=17782]: Done uploading segments... Aug 8 02:52:49.983289597 (17h13m24.498392175s) [upload=17782]: Upload finished. err: metaclient: Too Many Requests
Duration: 213.720091ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..] duration:
197.636597ms
FAIL (2025-08-08 02:54:26 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== m43.691381585s) [upload=17645]: Done uploading segments... Aug 8 02:45:09.176296212 (17h5m43.691398786s) [upload=17645]: Upload finished. err: <nil> Aug 8 02:46:04.444774241 (17h6m38.959876804s) [upload=17660]: Starting upload Aug 8 02:46:04.444862810 (17h6m38.959965425s) [upload=17660]: Done starting upload Aug 8 02:46:04.444937571 (17h6m38.96004014s) [upload=17660]: Uploading segments... Aug 8 02:46:04.445020035 (17h6m38.960122601s) [upload=17660]: Got next segment. Inline: true Aug 8 02:46:04.445028641 (17h6m38.960131209s) [upload=17660]: Waiting for error group managing segments... Aug 8 02:46:04.445085559 (17h6m38.960188125s) [upload=17660]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:46:04.445090655 (17h6m38.960193232s) [upload=17660]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:46:04.445094218 (17h6m38.960196793s) [upload=17660]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:46:04.519397453 (17h6m39.034500036s) [upload=17660]: Done uploading segments... Aug 8 02:46:04.519419431 (17h6m39.034522009s) [upload=17660]: Upload finished. err: <nil> Aug 8 02:46:29.848775596 (17h7m4.363878185s) [upload=17669]: Starting upload Aug 8 02:46:29.848845713 (17h7m4.363948289s) [upload=17669]: Done starting upload Aug 8 02:46:29.848924385 (17h7m4.364026954s) [upload=17669]: Uploading segments... Aug 8 02:46:29.848961594 (17h7m4.364064177s) [upload=17669]: Got next segment. Inline: true Aug 8 02:46:29.848968008 (17h7m4.364070587s) [upload=17669]: Waiting for error group managing segments... Aug 8 02:46:29.849009835 (17h7m4.364112411s) [upload=17669]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:46:29.849014457 (17h7m4.364117034s) [upload=17669]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:46:29.849018027 (17h7m4.364120603s) [upload=17669]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:46:29.914967479 (17h7m4.430070058s) [upload=17669]: Done uploading segments... Aug 8 02:46:29.914984083 (17h7m4.430086659s) [upload=17669]: Upload finished. err: <nil> Aug 8 02:47:07.009158285 (17h7m41.524260866s) [upload=17683]: Starting upload Aug 8 02:47:07.009236490 (17h7m41.524339073s) [upload=17683]: Done starting upload Aug 8 02:47:07.009381821 (17h7m41.5244844s) [upload=17683]: Uploading segments... Aug 8 02:47:07.009460862 (17h7m41.524563448s) [upload=17683]: Got next segment. Inline: true Aug 8 02:47:07.009467487 (17h7m41.524570067s) [upload=17683]: Waiting for error group managing segments... Aug 8 02:47:07.009557493 (17h7m41.524660055s) [upload=17683]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:47:07.009562936 (17h7m41.524665513s) [upload=17683]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:47:07.009566255 (17h7m41.524668832s) [upload=17683]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:47:07.108408364 (17h7m41.623510942s) [upload=17683]: Done uploading segments... Aug 8 02:47:07.108426523 (17h7m41.623529101s) [upload=17683]: Upload finished. err: <nil> Aug 8 02:47:43.954966497 (17h8m18.470069065s) [upload=17694]: Starting upload Aug 8 02:47:43.955070525 (17h8m18.470173105s) [upload=17694]: Done starting upload Aug 8 02:47:43.955164363 (17h8m18.470266931s) [upload=17694]: Uploading segments... Aug 8 02:47:43.955233625 (17h8m18.470336207s) [upload=17694]: Got next segment. Inline: true Aug 8 02:47:43.955238449 (17h8m18.470341026s) [upload=17694]: Waiting for error group managing segments... Aug 8 02:47:43.955292283 (17h8m18.470394859s) [upload=17694]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:47:43.955296757 (17h8m18.470399334s) [upload=17694]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:47:43.955299998 (17h8m18.470402574s) [upload=17694]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:47:44.124192924 (17h8m18.639295505s) [upload=17694]: Done uploading segments... Aug 8 02:47:44.124209329 (17h8m18.639311892s) [upload=17694]: Upload finished. err: <nil> Aug 8 02:48:43.731612500 (17h9m18.246715079s) [upload=17714]: Starting upload Aug 8 02:48:43.731667994 (17h9m18.246770571s) [upload=17714]: Done starting upload Aug 8 02:48:43.731765653 (17h9m18.246868236s) [upload=17714]: Uploading segments... Aug 8 02:48:43.731812427 (17h9m18.246915014s) [upload=17714]: Got next segment. Inline: true Aug 8 02:48:43.731833486 (17h9m18.24693607s) [upload=17714]: Waiting for error group managing segments... Aug 8 02:48:43.731883643 (17h9m18.246986222s) [upload=17714]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:48:43.731888532 (17h9m18.246991109s) [upload=17714]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:48:43.731891882 (17h9m18.246994458s) [upload=17714]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:48:44.128878769 (17h9m18.643981354s) [upload=17714]: Done uploading segments... Aug 8 02:48:44.128904173 (17h9m18.644006818s) [upload=17714]: Upload finished. err: <nil> Aug 8 02:49:38.867826030 (17h10m13.382928597s) [upload=17730]: Starting upload Aug 8 02:49:38.867910640 (17h10m13.38301321s) [upload=17730]: Done starting upload Aug 8 02:49:38.868107732 (17h10m13.3832103s) [upload=17730]: Uploading segments... Aug 8 02:49:38.868198737 (17h10m13.383301322s) [upload=17730]: Got next segment. Inline: true Aug 8 02:49:38.868206330 (17h10m13.38330891s) [upload=17730]: Waiting for error group managing segments... Aug 8 02:49:38.868306560 (17h10m13.383409144s) [upload=17730]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:49:38.868323606 (17h10m13.383426205s) [upload=17730]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:49:38.868327207 (17h10m13.383429792s) [upload=17730]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:49:38.988012432 (17h10m13.503115008s) [upload=17730]: Done uploading segments... Aug 8 02:49:38.988031058 (17h10m13.503133625s) [upload=17730]: Upload finished. err: <nil> Aug 8 02:50:07.278685760 (17h10m41.793788343s) [upload=17739]: Starting upload Aug 8 02:50:07.278829106 (17h10m41.793931683s) [upload=17739]: Done starting upload Aug 8 02:50:07.278912448 (17h10m41.79401501s) [upload=17739]: Uploading segments... Aug 8 02:50:07.278959874 (17h10m41.79406246s) [upload=17739]: Got next segment. Inline: true Aug 8 02:50:07.278965894 (17h10m41.79406848s) [upload=17739]: Waiting for error group managing segments... Aug 8 02:50:07.279051579 (17h10m41.794154156s) [upload=17739]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:50:07.279058068 (17h10m41.794160653s) [upload=17739]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:50:07.279062000 (17h10m41.794164578s) [upload=17739]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:50:07.373452172 (17h10m41.88855475s) [upload=17739]: Done uploading segments... Aug 8 02:50:07.373481935 (17h10m41.888584519s) [upload=17739]: Upload finished. err: <nil> Aug 8 02:50:39.619819607 (17h11m14.134922187s) [upload=17749]: Starting upload Aug 8 02:50:39.619901137 (17h11m14.135003715s) [upload=17749]: Done starting upload Aug 8 02:50:39.620009105 (17h11m14.135111684s) [upload=17749]: Uploading segments... Aug 8 02:50:39.620061236 (17h11m14.135163825s) [upload=17749]: Got next segment. Inline: true Aug 8 02:50:39.620066774 (17h11m14.135169357s) [upload=17749]: Waiting for error group managing segments... Aug 8 02:50:39.620129403 (17h11m14.135231991s) [upload=17749]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:50:39.620134679 (17h11m14.135237257s) [upload=17749]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:50:39.620138422 (17h11m14.135241002s) [upload=17749]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:50:39.783502378 (17h11m14.298604959s) [upload=17749]: Done uploading segments... Aug 8 02:50:39.783519169 (17h11m14.298621746s) [upload=17749]: Upload finished. err: <nil> Aug 8 02:51:53.488163266 (17h12m28.003265854s) [upload=17769]: Starting upload Aug 8 02:51:53.488231606 (17h12m28.003334191s) [upload=17769]: Done starting upload Aug 8 02:51:53.488304051 (17h12m28.00340663s) [upload=17769]: Uploading segments... Aug 8 02:51:53.488381549 (17h12m28.003484136s) [upload=17769]: Got next segment. Inline: true Aug 8 02:51:53.488387602 (17h12m28.003490183s) [upload=17769]: Waiting for error group managing segments... Aug 8 02:51:53.488438849 (17h12m28.003541436s) [upload=17769]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:51:53.488445257 (17h12m28.003547838s) [upload=17769]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:51:53.488448976 (17h12m28.003551557s) [upload=17769]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:51:53.555883594 (17h12m28.070986169s) [upload=17769]: Done uploading segments... Aug 8 02:51:53.555899316 (17h12m28.071001892s) [upload=17769]: Upload finished. err: <nil> Aug 8 02:52:49.974130340 (17h13m24.489232916s) [upload=17782]: Starting upload Aug 8 02:52:49.974182339 (17h13m24.489284907s) [upload=17782]: Done starting upload Aug 8 02:52:49.974266182 (17h13m24.48936876s) [upload=17782]: Uploading segments... Aug 8 02:52:49.974303819 (17h13m24.489406465s) [upload=17782]: Got next segment. Inline: true Aug 8 02:52:49.974308907 (17h13m24.489411483s) [upload=17782]: Waiting for error group managing segments... Aug 8 02:52:49.974352082 (17h13m24.489454659s) [upload=17782]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:52:49.974356786 (17h13m24.489459362s) [upload=17782]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:52:49.974360363 (17h13m24.489462939s) [upload=17782]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:52:49.983269871 (17h13m24.498372455s) [upload=17782]: Done uploading segments... Aug 8 02:52:49.983289597 (17h13m24.498392175s) [upload=17782]: Upload finished. err: metaclient: Too Many Requests
Duration: 197.666783ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.] duration:
275.568137ms
FAIL (2025-08-08 02:53:15 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== m43.691381585s) [upload=17645]: Done uploading segments... Aug 8 02:45:09.176296212 (17h5m43.691398786s) [upload=17645]: Upload finished. err: <nil> Aug 8 02:46:04.444774241 (17h6m38.959876804s) [upload=17660]: Starting upload Aug 8 02:46:04.444862810 (17h6m38.959965425s) [upload=17660]: Done starting upload Aug 8 02:46:04.444937571 (17h6m38.96004014s) [upload=17660]: Uploading segments... Aug 8 02:46:04.445020035 (17h6m38.960122601s) [upload=17660]: Got next segment. Inline: true Aug 8 02:46:04.445028641 (17h6m38.960131209s) [upload=17660]: Waiting for error group managing segments... Aug 8 02:46:04.445085559 (17h6m38.960188125s) [upload=17660]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:46:04.445090655 (17h6m38.960193232s) [upload=17660]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:46:04.445094218 (17h6m38.960196793s) [upload=17660]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:46:04.519397453 (17h6m39.034500036s) [upload=17660]: Done uploading segments... Aug 8 02:46:04.519419431 (17h6m39.034522009s) [upload=17660]: Upload finished. err: <nil> Aug 8 02:46:29.848775596 (17h7m4.363878185s) [upload=17669]: Starting upload Aug 8 02:46:29.848845713 (17h7m4.363948289s) [upload=17669]: Done starting upload Aug 8 02:46:29.848924385 (17h7m4.364026954s) [upload=17669]: Uploading segments... Aug 8 02:46:29.848961594 (17h7m4.364064177s) [upload=17669]: Got next segment. Inline: true Aug 8 02:46:29.848968008 (17h7m4.364070587s) [upload=17669]: Waiting for error group managing segments... Aug 8 02:46:29.849009835 (17h7m4.364112411s) [upload=17669]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:46:29.849014457 (17h7m4.364117034s) [upload=17669]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:46:29.849018027 (17h7m4.364120603s) [upload=17669]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:46:29.914967479 (17h7m4.430070058s) [upload=17669]: Done uploading segments... Aug 8 02:46:29.914984083 (17h7m4.430086659s) [upload=17669]: Upload finished. err: <nil> Aug 8 02:47:07.009158285 (17h7m41.524260866s) [upload=17683]: Starting upload Aug 8 02:47:07.009236490 (17h7m41.524339073s) [upload=17683]: Done starting upload Aug 8 02:47:07.009381821 (17h7m41.5244844s) [upload=17683]: Uploading segments... Aug 8 02:47:07.009460862 (17h7m41.524563448s) [upload=17683]: Got next segment. Inline: true Aug 8 02:47:07.009467487 (17h7m41.524570067s) [upload=17683]: Waiting for error group managing segments... Aug 8 02:47:07.009557493 (17h7m41.524660055s) [upload=17683]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:47:07.009562936 (17h7m41.524665513s) [upload=17683]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:47:07.009566255 (17h7m41.524668832s) [upload=17683]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:47:07.108408364 (17h7m41.623510942s) [upload=17683]: Done uploading segments... Aug 8 02:47:07.108426523 (17h7m41.623529101s) [upload=17683]: Upload finished. err: <nil> Aug 8 02:47:43.954966497 (17h8m18.470069065s) [upload=17694]: Starting upload Aug 8 02:47:43.955070525 (17h8m18.470173105s) [upload=17694]: Done starting upload Aug 8 02:47:43.955164363 (17h8m18.470266931s) [upload=17694]: Uploading segments... Aug 8 02:47:43.955233625 (17h8m18.470336207s) [upload=17694]: Got next segment. Inline: true Aug 8 02:47:43.955238449 (17h8m18.470341026s) [upload=17694]: Waiting for error group managing segments... Aug 8 02:47:43.955292283 (17h8m18.470394859s) [upload=17694]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:47:43.955296757 (17h8m18.470399334s) [upload=17694]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:47:43.955299998 (17h8m18.470402574s) [upload=17694]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:47:44.124192924 (17h8m18.639295505s) [upload=17694]: Done uploading segments... Aug 8 02:47:44.124209329 (17h8m18.639311892s) [upload=17694]: Upload finished. err: <nil> Aug 8 02:48:43.731612500 (17h9m18.246715079s) [upload=17714]: Starting upload Aug 8 02:48:43.731667994 (17h9m18.246770571s) [upload=17714]: Done starting upload Aug 8 02:48:43.731765653 (17h9m18.246868236s) [upload=17714]: Uploading segments... Aug 8 02:48:43.731812427 (17h9m18.246915014s) [upload=17714]: Got next segment. Inline: true Aug 8 02:48:43.731833486 (17h9m18.24693607s) [upload=17714]: Waiting for error group managing segments... Aug 8 02:48:43.731883643 (17h9m18.246986222s) [upload=17714]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:48:43.731888532 (17h9m18.246991109s) [upload=17714]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:48:43.731891882 (17h9m18.246994458s) [upload=17714]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:48:44.128878769 (17h9m18.643981354s) [upload=17714]: Done uploading segments... Aug 8 02:48:44.128904173 (17h9m18.644006818s) [upload=17714]: Upload finished. err: <nil> Aug 8 02:49:38.867826030 (17h10m13.382928597s) [upload=17730]: Starting upload Aug 8 02:49:38.867910640 (17h10m13.38301321s) [upload=17730]: Done starting upload Aug 8 02:49:38.868107732 (17h10m13.3832103s) [upload=17730]: Uploading segments... Aug 8 02:49:38.868198737 (17h10m13.383301322s) [upload=17730]: Got next segment. Inline: true Aug 8 02:49:38.868206330 (17h10m13.38330891s) [upload=17730]: Waiting for error group managing segments... Aug 8 02:49:38.868306560 (17h10m13.383409144s) [upload=17730]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:49:38.868323606 (17h10m13.383426205s) [upload=17730]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:49:38.868327207 (17h10m13.383429792s) [upload=17730]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:49:38.988012432 (17h10m13.503115008s) [upload=17730]: Done uploading segments... Aug 8 02:49:38.988031058 (17h10m13.503133625s) [upload=17730]: Upload finished. err: <nil> Aug 8 02:50:07.278685760 (17h10m41.793788343s) [upload=17739]: Starting upload Aug 8 02:50:07.278829106 (17h10m41.793931683s) [upload=17739]: Done starting upload Aug 8 02:50:07.278912448 (17h10m41.79401501s) [upload=17739]: Uploading segments... Aug 8 02:50:07.278959874 (17h10m41.79406246s) [upload=17739]: Got next segment. Inline: true Aug 8 02:50:07.278965894 (17h10m41.79406848s) [upload=17739]: Waiting for error group managing segments... Aug 8 02:50:07.279051579 (17h10m41.794154156s) [upload=17739]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:50:07.279058068 (17h10m41.794160653s) [upload=17739]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:50:07.279062000 (17h10m41.794164578s) [upload=17739]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:50:07.373452172 (17h10m41.88855475s) [upload=17739]: Done uploading segments... Aug 8 02:50:07.373481935 (17h10m41.888584519s) [upload=17739]: Upload finished. err: <nil> Aug 8 02:50:39.619819607 (17h11m14.134922187s) [upload=17749]: Starting upload Aug 8 02:50:39.619901137 (17h11m14.135003715s) [upload=17749]: Done starting upload Aug 8 02:50:39.620009105 (17h11m14.135111684s) [upload=17749]: Uploading segments... Aug 8 02:50:39.620061236 (17h11m14.135163825s) [upload=17749]: Got next segment. Inline: true Aug 8 02:50:39.620066774 (17h11m14.135169357s) [upload=17749]: Waiting for error group managing segments... Aug 8 02:50:39.620129403 (17h11m14.135231991s) [upload=17749]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:50:39.620134679 (17h11m14.135237257s) [upload=17749]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:50:39.620138422 (17h11m14.135241002s) [upload=17749]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:50:39.783502378 (17h11m14.298604959s) [upload=17749]: Done uploading segments... Aug 8 02:50:39.783519169 (17h11m14.298621746s) [upload=17749]: Upload finished. err: <nil> Aug 8 02:51:53.488163266 (17h12m28.003265854s) [upload=17769]: Starting upload Aug 8 02:51:53.488231606 (17h12m28.003334191s) [upload=17769]: Done starting upload Aug 8 02:51:53.488304051 (17h12m28.00340663s) [upload=17769]: Uploading segments... Aug 8 02:51:53.488381549 (17h12m28.003484136s) [upload=17769]: Got next segment. Inline: true Aug 8 02:51:53.488387602 (17h12m28.003490183s) [upload=17769]: Waiting for error group managing segments... Aug 8 02:51:53.488438849 (17h12m28.003541436s) [upload=17769]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:51:53.488445257 (17h12m28.003547838s) [upload=17769]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:51:53.488448976 (17h12m28.003551557s) [upload=17769]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:51:53.555883594 (17h12m28.070986169s) [upload=17769]: Done uploading segments... Aug 8 02:51:53.555899316 (17h12m28.071001892s) [upload=17769]: Upload finished. err: <nil> Aug 8 02:52:49.974130340 (17h13m24.489232916s) [upload=17782]: Starting upload Aug 8 02:52:49.974182339 (17h13m24.489284907s) [upload=17782]: Done starting upload Aug 8 02:52:49.974266182 (17h13m24.48936876s) [upload=17782]: Uploading segments... Aug 8 02:52:49.974303819 (17h13m24.489406465s) [upload=17782]: Got next segment. Inline: true Aug 8 02:52:49.974308907 (17h13m24.489411483s) [upload=17782]: Waiting for error group managing segments... Aug 8 02:52:49.974352082 (17h13m24.489454659s) [upload=17782]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:52:49.974356786 (17h13m24.489459362s) [upload=17782]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:52:49.974360363 (17h13m24.489462939s) [upload=17782]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:52:49.983269871 (17h13m24.498372455s) [upload=17782]: Done uploading segments... Aug 8 02:52:49.983289597 (17h13m24.498392175s) [upload=17782]: Upload finished. err: metaclient: Too Many Requests
Duration: 275.62004ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
42.323472ms
FAIL (2025-08-08 02:52:49 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== m43.691381585s) [upload=17645]: Done uploading segments... Aug 8 02:45:09.176296212 (17h5m43.691398786s) [upload=17645]: Upload finished. err: <nil> Aug 8 02:46:04.444774241 (17h6m38.959876804s) [upload=17660]: Starting upload Aug 8 02:46:04.444862810 (17h6m38.959965425s) [upload=17660]: Done starting upload Aug 8 02:46:04.444937571 (17h6m38.96004014s) [upload=17660]: Uploading segments... Aug 8 02:46:04.445020035 (17h6m38.960122601s) [upload=17660]: Got next segment. Inline: true Aug 8 02:46:04.445028641 (17h6m38.960131209s) [upload=17660]: Waiting for error group managing segments... Aug 8 02:46:04.445085559 (17h6m38.960188125s) [upload=17660]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:46:04.445090655 (17h6m38.960193232s) [upload=17660]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:46:04.445094218 (17h6m38.960196793s) [upload=17660]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:46:04.519397453 (17h6m39.034500036s) [upload=17660]: Done uploading segments... Aug 8 02:46:04.519419431 (17h6m39.034522009s) [upload=17660]: Upload finished. err: <nil> Aug 8 02:46:29.848775596 (17h7m4.363878185s) [upload=17669]: Starting upload Aug 8 02:46:29.848845713 (17h7m4.363948289s) [upload=17669]: Done starting upload Aug 8 02:46:29.848924385 (17h7m4.364026954s) [upload=17669]: Uploading segments... Aug 8 02:46:29.848961594 (17h7m4.364064177s) [upload=17669]: Got next segment. Inline: true Aug 8 02:46:29.848968008 (17h7m4.364070587s) [upload=17669]: Waiting for error group managing segments... Aug 8 02:46:29.849009835 (17h7m4.364112411s) [upload=17669]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:46:29.849014457 (17h7m4.364117034s) [upload=17669]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:46:29.849018027 (17h7m4.364120603s) [upload=17669]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:46:29.914967479 (17h7m4.430070058s) [upload=17669]: Done uploading segments... Aug 8 02:46:29.914984083 (17h7m4.430086659s) [upload=17669]: Upload finished. err: <nil> Aug 8 02:47:07.009158285 (17h7m41.524260866s) [upload=17683]: Starting upload Aug 8 02:47:07.009236490 (17h7m41.524339073s) [upload=17683]: Done starting upload Aug 8 02:47:07.009381821 (17h7m41.5244844s) [upload=17683]: Uploading segments... Aug 8 02:47:07.009460862 (17h7m41.524563448s) [upload=17683]: Got next segment. Inline: true Aug 8 02:47:07.009467487 (17h7m41.524570067s) [upload=17683]: Waiting for error group managing segments... Aug 8 02:47:07.009557493 (17h7m41.524660055s) [upload=17683]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:47:07.009562936 (17h7m41.524665513s) [upload=17683]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:47:07.009566255 (17h7m41.524668832s) [upload=17683]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:47:07.108408364 (17h7m41.623510942s) [upload=17683]: Done uploading segments... Aug 8 02:47:07.108426523 (17h7m41.623529101s) [upload=17683]: Upload finished. err: <nil> Aug 8 02:47:43.954966497 (17h8m18.470069065s) [upload=17694]: Starting upload Aug 8 02:47:43.955070525 (17h8m18.470173105s) [upload=17694]: Done starting upload Aug 8 02:47:43.955164363 (17h8m18.470266931s) [upload=17694]: Uploading segments... Aug 8 02:47:43.955233625 (17h8m18.470336207s) [upload=17694]: Got next segment. Inline: true Aug 8 02:47:43.955238449 (17h8m18.470341026s) [upload=17694]: Waiting for error group managing segments... Aug 8 02:47:43.955292283 (17h8m18.470394859s) [upload=17694]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:47:43.955296757 (17h8m18.470399334s) [upload=17694]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:47:43.955299998 (17h8m18.470402574s) [upload=17694]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:47:44.124192924 (17h8m18.639295505s) [upload=17694]: Done uploading segments... Aug 8 02:47:44.124209329 (17h8m18.639311892s) [upload=17694]: Upload finished. err: <nil> Aug 8 02:48:43.731612500 (17h9m18.246715079s) [upload=17714]: Starting upload Aug 8 02:48:43.731667994 (17h9m18.246770571s) [upload=17714]: Done starting upload Aug 8 02:48:43.731765653 (17h9m18.246868236s) [upload=17714]: Uploading segments... Aug 8 02:48:43.731812427 (17h9m18.246915014s) [upload=17714]: Got next segment. Inline: true Aug 8 02:48:43.731833486 (17h9m18.24693607s) [upload=17714]: Waiting for error group managing segments... Aug 8 02:48:43.731883643 (17h9m18.246986222s) [upload=17714]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:48:43.731888532 (17h9m18.246991109s) [upload=17714]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:48:43.731891882 (17h9m18.246994458s) [upload=17714]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:48:44.128878769 (17h9m18.643981354s) [upload=17714]: Done uploading segments... Aug 8 02:48:44.128904173 (17h9m18.644006818s) [upload=17714]: Upload finished. err: <nil> Aug 8 02:49:38.867826030 (17h10m13.382928597s) [upload=17730]: Starting upload Aug 8 02:49:38.867910640 (17h10m13.38301321s) [upload=17730]: Done starting upload Aug 8 02:49:38.868107732 (17h10m13.3832103s) [upload=17730]: Uploading segments... Aug 8 02:49:38.868198737 (17h10m13.383301322s) [upload=17730]: Got next segment. Inline: true Aug 8 02:49:38.868206330 (17h10m13.38330891s) [upload=17730]: Waiting for error group managing segments... Aug 8 02:49:38.868306560 (17h10m13.383409144s) [upload=17730]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:49:38.868323606 (17h10m13.383426205s) [upload=17730]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:49:38.868327207 (17h10m13.383429792s) [upload=17730]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:49:38.988012432 (17h10m13.503115008s) [upload=17730]: Done uploading segments... Aug 8 02:49:38.988031058 (17h10m13.503133625s) [upload=17730]: Upload finished. err: <nil> Aug 8 02:50:07.278685760 (17h10m41.793788343s) [upload=17739]: Starting upload Aug 8 02:50:07.278829106 (17h10m41.793931683s) [upload=17739]: Done starting upload Aug 8 02:50:07.278912448 (17h10m41.79401501s) [upload=17739]: Uploading segments... Aug 8 02:50:07.278959874 (17h10m41.79406246s) [upload=17739]: Got next segment. Inline: true Aug 8 02:50:07.278965894 (17h10m41.79406848s) [upload=17739]: Waiting for error group managing segments... Aug 8 02:50:07.279051579 (17h10m41.794154156s) [upload=17739]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:50:07.279058068 (17h10m41.794160653s) [upload=17739]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:50:07.279062000 (17h10m41.794164578s) [upload=17739]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:50:07.373452172 (17h10m41.88855475s) [upload=17739]: Done uploading segments... Aug 8 02:50:07.373481935 (17h10m41.888584519s) [upload=17739]: Upload finished. err: <nil> Aug 8 02:50:39.619819607 (17h11m14.134922187s) [upload=17749]: Starting upload Aug 8 02:50:39.619901137 (17h11m14.135003715s) [upload=17749]: Done starting upload Aug 8 02:50:39.620009105 (17h11m14.135111684s) [upload=17749]: Uploading segments... Aug 8 02:50:39.620061236 (17h11m14.135163825s) [upload=17749]: Got next segment. Inline: true Aug 8 02:50:39.620066774 (17h11m14.135169357s) [upload=17749]: Waiting for error group managing segments... Aug 8 02:50:39.620129403 (17h11m14.135231991s) [upload=17749]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:50:39.620134679 (17h11m14.135237257s) [upload=17749]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:50:39.620138422 (17h11m14.135241002s) [upload=17749]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:50:39.783502378 (17h11m14.298604959s) [upload=17749]: Done uploading segments... Aug 8 02:50:39.783519169 (17h11m14.298621746s) [upload=17749]: Upload finished. err: <nil> Aug 8 02:51:53.488163266 (17h12m28.003265854s) [upload=17769]: Starting upload Aug 8 02:51:53.488231606 (17h12m28.003334191s) [upload=17769]: Done starting upload Aug 8 02:51:53.488304051 (17h12m28.00340663s) [upload=17769]: Uploading segments... Aug 8 02:51:53.488381549 (17h12m28.003484136s) [upload=17769]: Got next segment. Inline: true Aug 8 02:51:53.488387602 (17h12m28.003490183s) [upload=17769]: Waiting for error group managing segments... Aug 8 02:51:53.488438849 (17h12m28.003541436s) [upload=17769]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:51:53.488445257 (17h12m28.003547838s) [upload=17769]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:51:53.488448976 (17h12m28.003551557s) [upload=17769]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:51:53.555883594 (17h12m28.070986169s) [upload=17769]: Done uploading segments... Aug 8 02:51:53.555899316 (17h12m28.071001892s) [upload=17769]: Upload finished. err: <nil> Aug 8 02:52:49.974130340 (17h13m24.489232916s) [upload=17782]: Starting upload Aug 8 02:52:49.974182339 (17h13m24.489284907s) [upload=17782]: Done starting upload Aug 8 02:52:49.974266182 (17h13m24.48936876s) [upload=17782]: Uploading segments... Aug 8 02:52:49.974303819 (17h13m24.489406465s) [upload=17782]: Got next segment. Inline: true Aug 8 02:52:49.974308907 (17h13m24.489411483s) [upload=17782]: Waiting for error group managing segments... Aug 8 02:52:49.974352082 (17h13m24.489454659s) [upload=17782]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 02:52:49.974356786 (17h13m24.489459362s) [upload=17782]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 02:52:49.974360363 (17h13m24.489462939s) [upload=17782]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 02:52:49.983269871 (17h13m24.498372455s) [upload=17782]: Done uploading segments... Aug 8 02:52:49.983289597 (17h13m24.498392175s) [upload=17782]: Upload finished. err: metaclient: Too Many Requests
Duration: 42.367819ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
30.258499ms target:
https://us1.storj.io/login
FAIL (2025-08-08 02:49:08 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.150.199.48:443: connect: connection refused
Duration: 30.277145ms
core
satellite
us1
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
29.368071ms target:
https://us1.storj.io/login
FAIL (2025-08-08 02:48:08 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.150.199.48:443: connect: connection refused
Duration: 29.396511ms
core
satellite
us1
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
29.031418ms target:
https://us1.storj.io/login
FAIL (2025-08-08 02:47:15 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.150.199.48:443: connect: connection refused
Duration: 29.04912ms
core
satellite
us1
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................] duration:
29.340755ms target:
https://us1.storj.io/login
FAIL (2025-08-08 02:46:53 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.150.199.48:443: connect: connection refused
Duration: 29.362712ms
core
satellite
us1
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
29.946759ms target:
https://us1.storj.io/login
FAIL (2025-08-08 02:46:21 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.150.199.48:443: connect: connection refused
Duration: 29.9715ms
core
satellite
us1
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................] duration:
30.162972ms target:
https://us1.storj.io/login
FAIL (2025-08-08 02:45:55 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.150.199.48:443: connect: connection refused
Duration: 30.185359ms
core
satellite
us1
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [......X.....................................................] duration:
30.116705ms target:
https://us1.storj.io/login
FAIL (2025-08-08 02:44:42 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.150.199.48:443: connect: connection refused
Duration: 30.152459ms
core
satellite
us1
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [.......X....................................................] duration:
31.802571ms target:
https://us1.storj.io/login
FAIL (2025-08-08 02:44:08 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.150.199.48:443: connect: connection refused
Duration: 31.848294ms
core
satellite
us1
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [........X...................................................] duration:
30.53899ms target:
https://us1.storj.io/login
FAIL (2025-08-08 02:43:10 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.150.199.48:443: connect: connection refused
Duration: 30.562581ms
core
satellite
us1
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [.........X..................................................] duration:
31.548224ms target:
https://us1.storj.io/login
FAIL (2025-08-08 02:42:23 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.150.199.48:443: connect: connection refused
Duration: 31.61268ms
core
satellite
us1
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [..........X.................................................] duration:
32.422577ms target:
https://us1.storj.io/login
FAIL (2025-08-08 02:41:23 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.150.199.48:443: connect: connection refused
Duration: 32.441531ms
core
satellite
us1
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [...........X................................................] duration:
35.628965ms target:
https://us1.storj.io/login
FAIL (2025-08-08 02:40:22 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.150.199.48:443: connect: connection refused
Duration: 35.659402ms
core
satellite
us1
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [............X...............................................] duration:
28.96208ms target:
https://us1.storj.io/login
FAIL (2025-08-08 02:39:46 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.150.199.48:443: connect: connection refused
Duration: 28.990409ms
core
satellite
us1
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [.............X..............................................] duration:
30.366225ms target:
https://us1.storj.io/login
FAIL (2025-08-08 02:38:38 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.150.199.48:443: connect: connection refused
Duration: 30.396027ms
core
satellite
us1
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [..............X.............................................] duration:
30.568137ms target:
https://us1.storj.io/login
FAIL (2025-08-08 02:38:10 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.150.199.48:443: connect: connection refused
Duration: 30.595215ms
core
satellite
us1
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [...............X............................................] duration:
30.58473ms target:
https://us1.storj.io/login
FAIL (2025-08-08 02:37:04 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.150.199.48:443: connect: connection refused
Duration: 30.611017ms
core
satellite
us1
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [................X...........................................] duration:
32.853493ms target:
https://us1.storj.io/login
FAIL (2025-08-08 02:36:12 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.150.199.48:443: connect: connection refused
Duration: 32.885405ms
core
satellite
us1
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [.................X..........................................] duration:
31.848425ms target:
https://us1.storj.io/login
FAIL (2025-08-08 02:35:53 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.150.199.48:443: connect: connection refused
Duration: 31.873175ms
core
satellite
us1
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [..................X.........................................] duration:
29.713699ms target:
https://us1.storj.io/login
FAIL (2025-08-08 02:35:22 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.150.199.48:443: connect: connection refused
Duration: 29.737631ms
core
satellite
us1
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [...................X........................................] duration:
30.92878ms target:
https://us1.storj.io/login
FAIL (2025-08-08 02:34:34 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.150.199.48:443: connect: connection refused
Duration: 30.94726ms
core
satellite
us1
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [....................X.......................................] duration:
35.879548ms target:
https://us1.storj.io/login
FAIL (2025-08-08 02:33:53 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.150.199.48:443: connect: connection refused
Duration: 35.903835ms
core
satellite
us1
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [.....................X......................................] duration:
39.061979ms target:
https://us1.storj.io/login
FAIL (2025-08-08 02:32:41 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.150.199.48:443: connect: connection refused
Duration: 39.084081ms
core
satellite
us1
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [......................X.....................................] duration:
37.357749ms target:
https://us1.storj.io/login
FAIL (2025-08-08 02:32:06 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.150.199.48:443: connect: connection refused
Duration: 37.382939ms
core
satellite
us1
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [.......................X....................................] duration:
42.169367ms target:
https://us1.storj.io/login
FAIL (2025-08-08 02:31:47 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.150.199.48:443: connect: connection refused
Duration: 42.195354ms
core
satellite
us1
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [........................X...................................] duration:
39.149074ms target:
https://us1.storj.io/login
FAIL (2025-08-08 02:30:47 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.150.199.48:443: connect: connection refused
Duration: 39.163532ms
core
satellite
us1
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [.........................X..................................] duration:
53.708194ms target:
https://us1.storj.io/login
FAIL (2025-08-08 02:30:01 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.150.199.48:443: connect: connection refused
Duration: 53.736327ms
core
satellite
us1
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [..........................X.................................] duration:
28.483534ms target:
https://us1.storj.io/login
FAIL (2025-08-08 02:29:18 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.150.199.48:443: connect: connection refused
Duration: 28.511354ms
core
satellite
us1
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [...........................X................................] duration:
31.843474ms target:
https://us1.storj.io/login
FAIL (2025-08-08 02:28:28 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.150.199.48:443: connect: connection refused
Duration: 31.865584ms
core
satellite
us1
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [............................X...............................] duration:
31.70814ms target:
https://us1.storj.io/login
FAIL (2025-08-08 02:27:46 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.150.199.48:443: connect: connection refused
Duration: 31.770034ms
core
satellite
us1
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [.............................X..............................] duration:
32.760356ms target:
https://us1.storj.io/login
FAIL (2025-08-08 02:27:09 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.150.199.48:443: connect: connection refused
Duration: 32.800554ms
core
satellite
us1
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [..............................X.............................] duration:
34.116079ms target:
https://us1.storj.io/login
FAIL (2025-08-08 02:26:16 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.150.199.48:443: connect: connection refused
Duration: 34.141852ms
core
satellite
us1
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [...............................X............................] duration:
32.136695ms target:
https://us1.storj.io/login
FAIL (2025-08-08 02:25:49 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.150.199.48:443: connect: connection refused
Duration: 32.162006ms
core
satellite
us1
|