Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............] duration:
371.003781ms
FAIL (2025-08-08 07:40:36 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
g_pos={0 0}, upload=22550]: Begin upload segment...
Aug 8 07:28:51.351770179 (21h49m25.866872766s) [seg_pos={0 0}, upload=22550]: Done begin upload segment.
Aug 8 07:28:51.351799155 (21h49m25.866901747s) [noise=true , address=104.36.82.23:4469 , piece_id=CI6CU5IBYK37MBIQW554EW6PPXVYA6ZF2CJPVZCIXMFFLXZ7UO7Q, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.351816053 (21h49m25.86691864s) [noise=true , address=193.149.164.143:4693 , piece_id=RCLLY6ABOIXYK6BBO5EWKTDDJ4MUH6IP2WGIHSTER4TEPSF2FYVA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.352227136 (21h49m25.867329713s) [noise=true , address=104.36.82.58:4286 , piece_id=36LFLTD2YWV3K7LM4M2GY3JIKLUDOZVRINQT3VLIFBRR55UCNA4Q, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.352322195 (21h49m25.867424772s) [noise=true , address=104.36.82.44:4880 , piece_id=HFXFL7ULX5OTYJFEOC45DI75DGDIHAQA6ZJIXHO22X4GPSEZN75Q, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.352611348 (21h49m25.867713924s) [noise=true , address=104.36.82.53:4139 , piece_id=IGOHVQJMZ3NWLCBKVPVHTVWVHDS43DABQPEMJUTB2TXAQPQOURSQ, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.352702579 (21h49m25.867805142s) [noise=true , address=45.91.116.7:4013 , piece_id=XG4M7HF2E5YIRJ7PVBEGZC2ADPUPAL2QL5TPUAFIYXZUP3BX2Y7A, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.353022021 (21h49m25.868124597s) [noise=true , address=104.36.82.16:4321 , piece_id=MTRDWQYPHECWBAH4ECUGJL2Y2LAUA2RWHX5J6XMR65NZOACBF6PA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.353464633 (21h49m25.868567211s) [noise=true , address=104.36.82.46:4487 , piece_id=JOSIX5WT5A36RXKJNWO5KLBEQOCNQCEZ7A2S4KYIBJHRKN3KT6RQ, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.354418304 (21h49m25.869520884s) [noise=true , address=104.36.82.38:4707 , piece_id=WZWQHYDMTUG2FOTZ6JJRVIXXFIPHJ3I2YIRQXMN7B2W7QGKAFY3A, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.354959848 (21h49m25.87006243s) [noise=true , address=104.36.82.47:4544 , piece_id=YDG3KOKUS5NVBWMQYJ7JJEO3PAXZKPZPUOUWUDCZ4RAVATW67NKQ, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.355461506 (21h49m25.870564086s) [noise=true , address=104.36.82.60:4363 , piece_id=QMR7CM4ASNAKG5PT2XUPFR4MIBHNFYJ7OZMASJ7Q6R2C225L655A, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.355910519 (21h49m25.871013101s) [noise=true , address=104.36.82.2:4741 , piece_id=3NWVL6DQCMDO5PQLSFU3VER7RSHGRAB6YRT32XBFUVLAXGEJQN7A, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.357467009 (21h49m25.872569585s) [noise=true , address=104.36.82.59:4322 , piece_id=NKCSGWGYWOIJWHSQUN2ZYOVZYIRML5G3DXVRJYDTAMHTEN2VES3Q, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.358628018 (21h49m25.873730598s) [noise=true , address=45.91.116.11:4120 , piece_id=274GAU5MCZAZOHB3ESPDSJP23Z5FZB636CWGN7ELS2HYY5OFZ7RA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.359105086 (21h49m25.874207666s) [noise=true , address=104.36.82.45:4907 , piece_id=ZF2SKF5O2WVNHM7CHQ7E4U7HVKQWCJGOZEVIDXO3OU4XHVUFPIGA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.360554866 (21h49m25.875657451s) [noise=true , address=104.36.82.36:4648 , piece_id=3WAWCS3XXWVX3ACYJG2OOS2R34SXOWYSHXVF6AMPFXJFCMYH4SZA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.363015621 (21h49m25.8781182s) [noise=true , address=104.36.82.62:4420 , piece_id=7S6ALI3DNOM7IURBTYY3DLALDQV7R7LLHPQCA4Q6G4HA2SOZ2YMA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.364089235 (21h49m25.87919182s) [noise=true , address=104.36.82.9:4008 , piece_id=EGIGCEJHELTHOJZWPFWNHCP672MIXAHZIQUK2WDHER77VVJ4NZZA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.467747752 (21h49m25.982850337s) [upload=22550]: Next returned nil segment
Aug 8 07:28:51.467767955 (21h49m25.982870543s) [upload=22550]: Waiting for error group managing segments...
Aug 8 07:28:51.473577577 (21h49m25.988680143s) [noise=true , address=45.91.116.7:4013 , piece_id=XG4M7HF2E5YIRJ7PVBEGZC2ADPUPAL2QL5TPUAFIYXZUP3BX2Y7A, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.478188638 (21h49m25.993291216s) [noise=true , address=104.36.82.16:4321 , piece_id=MTRDWQYPHECWBAH4ECUGJL2Y2LAUA2RWHX5J6XMR65NZOACBF6PA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.478342049 (21h49m25.993444627s) [noise=true , address=104.36.82.53:4139 , piece_id=IGOHVQJMZ3NWLCBKVPVHTVWVHDS43DABQPEMJUTB2TXAQPQOURSQ, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.478768394 (21h49m25.993870961s) [noise=true , address=104.36.82.36:4648 , piece_id=3WAWCS3XXWVX3ACYJG2OOS2R34SXOWYSHXVF6AMPFXJFCMYH4SZA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.479336172 (21h49m25.994438735s) [noise=true , address=104.36.82.59:4322 , piece_id=NKCSGWGYWOIJWHSQUN2ZYOVZYIRML5G3DXVRJYDTAMHTEN2VES3Q, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.479515883 (21h49m25.994618457s) [noise=true , address=104.36.82.58:4286 , piece_id=36LFLTD2YWV3K7LM4M2GY3JIKLUDOZVRINQT3VLIFBRR55UCNA4Q, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.480474985 (21h49m25.995577567s) [noise=true , address=104.36.82.2:4741 , piece_id=3NWVL6DQCMDO5PQLSFU3VER7RSHGRAB6YRT32XBFUVLAXGEJQN7A, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.481008628 (21h49m25.996111195s) [noise=true , address=104.36.82.38:4707 , piece_id=WZWQHYDMTUG2FOTZ6JJRVIXXFIPHJ3I2YIRQXMN7B2W7QGKAFY3A, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.481589774 (21h49m25.996692358s) [noise=true , address=104.36.82.60:4363 , piece_id=QMR7CM4ASNAKG5PT2XUPFR4MIBHNFYJ7OZMASJ7Q6R2C225L655A, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.482143054 (21h49m25.99724562s) [noise=true , address=104.36.82.46:4487 , piece_id=JOSIX5WT5A36RXKJNWO5KLBEQOCNQCEZ7A2S4KYIBJHRKN3KT6RQ, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.485172215 (21h49m26.000274779s) [noise=true , address=104.36.82.47:4544 , piece_id=YDG3KOKUS5NVBWMQYJ7JJEO3PAXZKPZPUOUWUDCZ4RAVATW67NKQ, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.486088591 (21h49m26.001191172s) [noise=true , address=104.36.82.44:4880 , piece_id=HFXFL7ULX5OTYJFEOC45DI75DGDIHAQA6ZJIXHO22X4GPSEZN75Q, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.486628345 (21h49m26.001730916s) [noise=true , address=104.36.82.23:4469 , piece_id=CI6CU5IBYK37MBIQW554EW6PPXVYA6ZF2CJPVZCIXMFFLXZ7UO7Q, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.487179958 (21h49m26.002282521s) [noise=true , address=104.36.82.62:4420 , piece_id=7S6ALI3DNOM7IURBTYY3DLALDQV7R7LLHPQCA4Q6G4HA2SOZ2YMA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.487192580 (21h49m26.002295147s) [seg_pos={0 0}, upload=22550]: Segment reached optimal threshold of 14 pieces.
Aug 8 07:28:51.487306760 (21h49m26.002409325s) [noise=true , address=104.36.82.45:4907 , piece_id=ZF2SKF5O2WVNHM7CHQ7E4U7HVKQWCJGOZEVIDXO3OU4XHVUFPIGA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12URx18Ckh7VaAKJ1hdQQucMRu5qiDrAraqaycjYqxETgwZ4uWS): protocol: expected piece hash; context canceled; EOF
Aug 8 07:28:51.487325587 (21h49m26.00242815s) [noise=true , address=104.36.82.9:4008 , piece_id=EGIGCEJHELTHOJZWPFWNHCP672MIXAHZIQUK2WDHER77VVJ4NZZA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.487748790 (21h49m26.002851367s) [noise=true , address=193.149.164.143:4693 , piece_id=RCLLY6ABOIXYK6BBO5EWKTDDJ4MUH6IP2WGIHSTER4TEPSF2FYVA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LHoCE9CnM32tbMNqAB7HpmArSVLTycv9o6kSSFNUJsoDeERkQ): protocol: read tcp 159.223.143.39:37688->193.149.164.143:4693: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 07:28:51.487858041 (21h49m26.002960604s) [noise=true , address=45.91.116.11:4120 , piece_id=274GAU5MCZAZOHB3ESPDSJP23Z5FZB636CWGN7ELS2HYY5OFZ7RA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.487890658 (21h49m26.002993237s) [seg_pos={0 0}, upload=22550]: Done waiting for segment. successful: 16 optimal: 14 errs: <nil>
Aug 8 07:28:51.487971404 (21h49m26.00307399s) [upload=22550]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 8 07:28:51.487976917 (21h49m26.003079481s) [upload=22550]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 8 07:28:51.647177827 (21h49m26.162280406s) [upload=22550]: Done uploading segments...
Aug 8 07:28:51.647194442 (21h49m26.162297008s) [upload=22550]: Upload finished. err: <nil>
Aug 8 07:29:34.528501362 (21h50m9.043603939s) [upload=22562]: Starting upload
Aug 8 07:29:34.528548565 (21h50m9.043651143s) [upload=22562]: Done starting upload
Aug 8 07:29:34.528615374 (21h50m9.043717942s) [upload=22562]: Uploading segments...
Aug 8 07:29:34.528903298 (21h50m9.044005874s) [upload=22562]: Got next segment. Inline: false
Aug 8 07:29:34.528940157 (21h50m9.044042733s) [upload=22562]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 8 07:29:34.528944789 (21h50m9.044047366s) [upload=22562]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 8 07:29:34.536881082 (21h50m9.051983672s) [upload=22562]: Next segment err: metaclient: Too Many Requests
Aug 8 07:29:34.536925024 (21h50m9.052027602s) [upload=22562]: Done uploading segments...
Aug 8 07:29:34.536943164 (21h50m9.052045741s) [upload=22562]: Upload finished. err: metaclient: Too Many Requests
Duration: 371.031701ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
31.697958ms target:
https://us1.storj.io/login
FAIL (2025-08-08 07:40:24 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.150.199.48:443: connect: connection refused
Duration: 31.721298ms
core
satellite
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [X..............................................X............] duration:
525.519059ms
FAIL (2025-08-08 07:40:10 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== nts... Aug 8 07:24:18.405184267 (21h44m52.920286834s) [upload=22475]: Upload finished. err: <nil> Aug 8 07:24:44.148856783 (21h45m18.663959349s) [upload=22481]: Starting upload Aug 8 07:24:44.148908040 (21h45m18.664010617s) [upload=22481]: Done starting upload Aug 8 07:24:44.148978727 (21h45m18.664081294s) [upload=22481]: Uploading segments... Aug 8 07:24:44.149031432 (21h45m18.664134016s) [upload=22481]: Got next segment. Inline: true Aug 8 07:24:44.149047929 (21h45m18.664150495s) [upload=22481]: Waiting for error group managing segments... Aug 8 07:24:44.149231742 (21h45m18.664334308s) [upload=22481]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:24:44.149238484 (21h45m18.664341061s) [upload=22481]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:24:44.149241695 (21h45m18.664344271s) [upload=22481]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:24:44.236175458 (21h45m18.751278039s) [upload=22481]: Done uploading segments... Aug 8 07:24:44.236197562 (21h45m18.75130015s) [upload=22481]: Upload finished. err: <nil> Aug 8 07:25:45.798907959 (21h46m20.31401054s) [upload=22496]: Starting upload Aug 8 07:25:45.798982852 (21h46m20.314085432s) [upload=22496]: Done starting upload Aug 8 07:25:45.799141617 (21h46m20.314244194s) [upload=22496]: Uploading segments... Aug 8 07:25:45.799187034 (21h46m20.314289621s) [upload=22496]: Got next segment. Inline: true Aug 8 07:25:45.799192794 (21h46m20.314295378s) [upload=22496]: Waiting for error group managing segments... Aug 8 07:25:45.799260506 (21h46m20.314363086s) [upload=22496]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:25:45.799265543 (21h46m20.314368123s) [upload=22496]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:25:45.799269614 (21h46m20.314372195s) [upload=22496]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:25:45.856873466 (21h46m20.371976033s) [upload=22496]: Done uploading segments... Aug 8 07:25:45.856894517 (21h46m20.371997083s) [upload=22496]: Upload finished. err: <nil> Aug 8 07:26:45.160306388 (21h47m19.67540897s) [upload=22512]: Starting upload Aug 8 07:26:45.160369840 (21h47m19.675472407s) [upload=22512]: Done starting upload Aug 8 07:26:45.160448308 (21h47m19.675550874s) [upload=22512]: Uploading segments... Aug 8 07:26:45.160512759 (21h47m19.675615339s) [upload=22512]: Got next segment. Inline: true Aug 8 07:26:45.160518821 (21h47m19.675621389s) [upload=22512]: Waiting for error group managing segments... Aug 8 07:26:45.160566486 (21h47m19.675669054s) [upload=22512]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:26:45.160571363 (21h47m19.675673943s) [upload=22512]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:26:45.160575116 (21h47m19.675677698s) [upload=22512]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:26:45.219521344 (21h47m19.73462392s) [upload=22512]: Done uploading segments... Aug 8 07:26:45.219537755 (21h47m19.73464032s) [upload=22512]: Upload finished. err: <nil> Aug 8 07:27:50.008745364 (21h48m24.523847971s) [upload=22529]: Starting upload Aug 8 07:27:50.008828662 (21h48m24.523931252s) [upload=22529]: Done starting upload Aug 8 07:27:50.008913720 (21h48m24.524016289s) [upload=22529]: Uploading segments... Aug 8 07:27:50.008968901 (21h48m24.524071486s) [upload=22529]: Got next segment. Inline: true Aug 8 07:27:50.008973893 (21h48m24.524076469s) [upload=22529]: Waiting for error group managing segments... Aug 8 07:27:50.009023439 (21h48m24.524126015s) [upload=22529]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:27:50.009028404 (21h48m24.524130983s) [upload=22529]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:27:50.009032740 (21h48m24.524135324s) [upload=22529]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:27:50.072222330 (21h48m24.587324909s) [upload=22529]: Done uploading segments... Aug 8 07:27:50.072260086 (21h48m24.587362652s) [upload=22529]: Upload finished. err: <nil> Aug 8 07:28:08.797868182 (21h48m43.312970759s) [upload=22537]: Starting upload Aug 8 07:28:08.797909617 (21h48m43.313012202s) [upload=22537]: Done starting upload Aug 8 07:28:08.797978107 (21h48m43.313080676s) [upload=22537]: Uploading segments... Aug 8 07:28:08.798028549 (21h48m43.313131134s) [upload=22537]: Got next segment. Inline: true Aug 8 07:28:08.798033746 (21h48m43.313136324s) [upload=22537]: Waiting for error group managing segments... Aug 8 07:28:08.798088133 (21h48m43.313190709s) [upload=22537]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:28:08.798092673 (21h48m43.313195249s) [upload=22537]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:28:08.798096028 (21h48m43.313198604s) [upload=22537]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:28:08.856181519 (21h48m43.371284096s) [upload=22537]: Done uploading segments... Aug 8 07:28:08.856219288 (21h48m43.371321856s) [upload=22537]: Upload finished. err: <nil> Aug 8 07:28:26.426406302 (21h49m0.941508883s) [upload=22543]: Starting upload Aug 8 07:28:26.426484520 (21h49m0.94158709s) [upload=22543]: Done starting upload Aug 8 07:28:26.426561912 (21h49m0.941664479s) [upload=22543]: Uploading segments... Aug 8 07:28:26.426613562 (21h49m0.941716147s) [upload=22543]: Got next segment. Inline: true Aug 8 07:28:26.426618334 (21h49m0.94172091s) [upload=22543]: Waiting for error group managing segments... Aug 8 07:28:26.426667358 (21h49m0.941769934s) [upload=22543]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:28:26.426672477 (21h49m0.941775052s) [upload=22543]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:28:26.426676980 (21h49m0.941779557s) [upload=22543]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:28:26.739786156 (21h49m1.254888734s) [upload=22543]: Done uploading segments... Aug 8 07:28:26.739818088 (21h49m1.254920663s) [upload=22543]: Upload finished. err: <nil> Aug 8 07:28:44.768272418 (21h49m19.283374995s) [upload=22547]: Starting upload Aug 8 07:28:44.768352182 (21h49m19.28345476s) [upload=22547]: Done starting upload Aug 8 07:28:44.768463680 (21h49m19.283566246s) [upload=22547]: Uploading segments... Aug 8 07:28:44.768552166 (21h49m19.283654733s) [upload=22547]: Got next segment. Inline: true Aug 8 07:28:44.768558003 (21h49m19.28366057s) [upload=22547]: Waiting for error group managing segments... Aug 8 07:28:44.768617460 (21h49m19.28372004s) [upload=22547]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:28:44.768622292 (21h49m19.283724868s) [upload=22547]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:28:44.768625505 (21h49m19.283728082s) [upload=22547]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:28:44.832641596 (21h49m19.347744163s) [upload=22547]: Done uploading segments... Aug 8 07:28:44.832661990 (21h49m19.347764559s) [upload=22547]: Upload finished. err: <nil> Aug 8 07:29:19.907807846 (21h49m54.422910428s) [upload=22558]: Starting upload Aug 8 07:29:19.907919209 (21h49m54.423021801s) [upload=22558]: Done starting upload Aug 8 07:29:19.908035700 (21h49m54.423138281s) [upload=22558]: Uploading segments... Aug 8 07:29:19.908115743 (21h49m54.423218334s) [upload=22558]: Got next segment. Inline: true Aug 8 07:29:19.908123216 (21h49m54.423225802s) [upload=22558]: Waiting for error group managing segments... Aug 8 07:29:19.908206488 (21h49m54.423309073s) [upload=22558]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:29:19.908213079 (21h49m54.423315659s) [upload=22558]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:29:19.908216828 (21h49m54.423319408s) [upload=22558]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:29:19.968416399 (21h49m54.483518987s) [upload=22558]: Done uploading segments... Aug 8 07:29:19.968437892 (21h49m54.48354047s) [upload=22558]: Upload finished. err: <nil> Aug 8 07:30:04.570811298 (21h50m39.085913875s) [upload=22574]: Starting upload Aug 8 07:30:04.570856561 (21h50m39.085959129s) [upload=22574]: Done starting upload Aug 8 07:30:04.570937395 (21h50m39.086039963s) [upload=22574]: Uploading segments... Aug 8 07:30:04.570993290 (21h50m39.086095879s) [upload=22574]: Got next segment. Inline: true Aug 8 07:30:04.570999714 (21h50m39.086102306s) [upload=22574]: Waiting for error group managing segments... Aug 8 07:30:04.571094735 (21h50m39.086197316s) [upload=22574]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:30:04.571101887 (21h50m39.086204467s) [upload=22574]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:30:04.571105790 (21h50m39.086208372s) [upload=22574]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:30:04.649336948 (21h50m39.164439536s) [upload=22574]: Done uploading segments... Aug 8 07:30:04.649362004 (21h50m39.164464585s) [upload=22574]: Upload finished. err: <nil> Aug 8 07:30:54.851564431 (21h51m29.366666997s) [upload=22587]: Starting upload Aug 8 07:30:54.851618044 (21h51m29.36672062s) [upload=22587]: Done starting upload Aug 8 07:30:54.851712059 (21h51m29.366814635s) [upload=22587]: Uploading segments... Aug 8 07:30:54.851783178 (21h51m29.36688576s) [upload=22587]: Got next segment. Inline: true Aug 8 07:30:54.851789517 (21h51m29.366892093s) [upload=22587]: Waiting for error group managing segments... Aug 8 07:30:54.851854337 (21h51m29.366956924s) [upload=22587]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:30:54.851861334 (21h51m29.366963924s) [upload=22587]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:30:54.851866059 (21h51m29.366968642s) [upload=22587]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:30:54.860593007 (21h51m29.375695595s) [upload=22587]: Done uploading segments... Aug 8 07:30:54.860627519 (21h51m29.375730095s) [upload=22587]: Upload finished. err: metaclient: Too Many Requests
Duration: 525.559774ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............................................X.............] duration:
579.59593ms
FAIL (2025-08-08 07:39:40 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
g_pos={0 0}, upload=22550]: Begin upload segment...
Aug 8 07:28:51.351770179 (21h49m25.866872766s) [seg_pos={0 0}, upload=22550]: Done begin upload segment.
Aug 8 07:28:51.351799155 (21h49m25.866901747s) [noise=true , address=104.36.82.23:4469 , piece_id=CI6CU5IBYK37MBIQW554EW6PPXVYA6ZF2CJPVZCIXMFFLXZ7UO7Q, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.351816053 (21h49m25.86691864s) [noise=true , address=193.149.164.143:4693 , piece_id=RCLLY6ABOIXYK6BBO5EWKTDDJ4MUH6IP2WGIHSTER4TEPSF2FYVA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.352227136 (21h49m25.867329713s) [noise=true , address=104.36.82.58:4286 , piece_id=36LFLTD2YWV3K7LM4M2GY3JIKLUDOZVRINQT3VLIFBRR55UCNA4Q, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.352322195 (21h49m25.867424772s) [noise=true , address=104.36.82.44:4880 , piece_id=HFXFL7ULX5OTYJFEOC45DI75DGDIHAQA6ZJIXHO22X4GPSEZN75Q, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.352611348 (21h49m25.867713924s) [noise=true , address=104.36.82.53:4139 , piece_id=IGOHVQJMZ3NWLCBKVPVHTVWVHDS43DABQPEMJUTB2TXAQPQOURSQ, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.352702579 (21h49m25.867805142s) [noise=true , address=45.91.116.7:4013 , piece_id=XG4M7HF2E5YIRJ7PVBEGZC2ADPUPAL2QL5TPUAFIYXZUP3BX2Y7A, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.353022021 (21h49m25.868124597s) [noise=true , address=104.36.82.16:4321 , piece_id=MTRDWQYPHECWBAH4ECUGJL2Y2LAUA2RWHX5J6XMR65NZOACBF6PA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.353464633 (21h49m25.868567211s) [noise=true , address=104.36.82.46:4487 , piece_id=JOSIX5WT5A36RXKJNWO5KLBEQOCNQCEZ7A2S4KYIBJHRKN3KT6RQ, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.354418304 (21h49m25.869520884s) [noise=true , address=104.36.82.38:4707 , piece_id=WZWQHYDMTUG2FOTZ6JJRVIXXFIPHJ3I2YIRQXMN7B2W7QGKAFY3A, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.354959848 (21h49m25.87006243s) [noise=true , address=104.36.82.47:4544 , piece_id=YDG3KOKUS5NVBWMQYJ7JJEO3PAXZKPZPUOUWUDCZ4RAVATW67NKQ, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.355461506 (21h49m25.870564086s) [noise=true , address=104.36.82.60:4363 , piece_id=QMR7CM4ASNAKG5PT2XUPFR4MIBHNFYJ7OZMASJ7Q6R2C225L655A, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.355910519 (21h49m25.871013101s) [noise=true , address=104.36.82.2:4741 , piece_id=3NWVL6DQCMDO5PQLSFU3VER7RSHGRAB6YRT32XBFUVLAXGEJQN7A, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.357467009 (21h49m25.872569585s) [noise=true , address=104.36.82.59:4322 , piece_id=NKCSGWGYWOIJWHSQUN2ZYOVZYIRML5G3DXVRJYDTAMHTEN2VES3Q, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.358628018 (21h49m25.873730598s) [noise=true , address=45.91.116.11:4120 , piece_id=274GAU5MCZAZOHB3ESPDSJP23Z5FZB636CWGN7ELS2HYY5OFZ7RA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.359105086 (21h49m25.874207666s) [noise=true , address=104.36.82.45:4907 , piece_id=ZF2SKF5O2WVNHM7CHQ7E4U7HVKQWCJGOZEVIDXO3OU4XHVUFPIGA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.360554866 (21h49m25.875657451s) [noise=true , address=104.36.82.36:4648 , piece_id=3WAWCS3XXWVX3ACYJG2OOS2R34SXOWYSHXVF6AMPFXJFCMYH4SZA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.363015621 (21h49m25.8781182s) [noise=true , address=104.36.82.62:4420 , piece_id=7S6ALI3DNOM7IURBTYY3DLALDQV7R7LLHPQCA4Q6G4HA2SOZ2YMA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.364089235 (21h49m25.87919182s) [noise=true , address=104.36.82.9:4008 , piece_id=EGIGCEJHELTHOJZWPFWNHCP672MIXAHZIQUK2WDHER77VVJ4NZZA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.467747752 (21h49m25.982850337s) [upload=22550]: Next returned nil segment
Aug 8 07:28:51.467767955 (21h49m25.982870543s) [upload=22550]: Waiting for error group managing segments...
Aug 8 07:28:51.473577577 (21h49m25.988680143s) [noise=true , address=45.91.116.7:4013 , piece_id=XG4M7HF2E5YIRJ7PVBEGZC2ADPUPAL2QL5TPUAFIYXZUP3BX2Y7A, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.478188638 (21h49m25.993291216s) [noise=true , address=104.36.82.16:4321 , piece_id=MTRDWQYPHECWBAH4ECUGJL2Y2LAUA2RWHX5J6XMR65NZOACBF6PA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.478342049 (21h49m25.993444627s) [noise=true , address=104.36.82.53:4139 , piece_id=IGOHVQJMZ3NWLCBKVPVHTVWVHDS43DABQPEMJUTB2TXAQPQOURSQ, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.478768394 (21h49m25.993870961s) [noise=true , address=104.36.82.36:4648 , piece_id=3WAWCS3XXWVX3ACYJG2OOS2R34SXOWYSHXVF6AMPFXJFCMYH4SZA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.479336172 (21h49m25.994438735s) [noise=true , address=104.36.82.59:4322 , piece_id=NKCSGWGYWOIJWHSQUN2ZYOVZYIRML5G3DXVRJYDTAMHTEN2VES3Q, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.479515883 (21h49m25.994618457s) [noise=true , address=104.36.82.58:4286 , piece_id=36LFLTD2YWV3K7LM4M2GY3JIKLUDOZVRINQT3VLIFBRR55UCNA4Q, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.480474985 (21h49m25.995577567s) [noise=true , address=104.36.82.2:4741 , piece_id=3NWVL6DQCMDO5PQLSFU3VER7RSHGRAB6YRT32XBFUVLAXGEJQN7A, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.481008628 (21h49m25.996111195s) [noise=true , address=104.36.82.38:4707 , piece_id=WZWQHYDMTUG2FOTZ6JJRVIXXFIPHJ3I2YIRQXMN7B2W7QGKAFY3A, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.481589774 (21h49m25.996692358s) [noise=true , address=104.36.82.60:4363 , piece_id=QMR7CM4ASNAKG5PT2XUPFR4MIBHNFYJ7OZMASJ7Q6R2C225L655A, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.482143054 (21h49m25.99724562s) [noise=true , address=104.36.82.46:4487 , piece_id=JOSIX5WT5A36RXKJNWO5KLBEQOCNQCEZ7A2S4KYIBJHRKN3KT6RQ, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.485172215 (21h49m26.000274779s) [noise=true , address=104.36.82.47:4544 , piece_id=YDG3KOKUS5NVBWMQYJ7JJEO3PAXZKPZPUOUWUDCZ4RAVATW67NKQ, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.486088591 (21h49m26.001191172s) [noise=true , address=104.36.82.44:4880 , piece_id=HFXFL7ULX5OTYJFEOC45DI75DGDIHAQA6ZJIXHO22X4GPSEZN75Q, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.486628345 (21h49m26.001730916s) [noise=true , address=104.36.82.23:4469 , piece_id=CI6CU5IBYK37MBIQW554EW6PPXVYA6ZF2CJPVZCIXMFFLXZ7UO7Q, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.487179958 (21h49m26.002282521s) [noise=true , address=104.36.82.62:4420 , piece_id=7S6ALI3DNOM7IURBTYY3DLALDQV7R7LLHPQCA4Q6G4HA2SOZ2YMA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.487192580 (21h49m26.002295147s) [seg_pos={0 0}, upload=22550]: Segment reached optimal threshold of 14 pieces.
Aug 8 07:28:51.487306760 (21h49m26.002409325s) [noise=true , address=104.36.82.45:4907 , piece_id=ZF2SKF5O2WVNHM7CHQ7E4U7HVKQWCJGOZEVIDXO3OU4XHVUFPIGA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12URx18Ckh7VaAKJ1hdQQucMRu5qiDrAraqaycjYqxETgwZ4uWS): protocol: expected piece hash; context canceled; EOF
Aug 8 07:28:51.487325587 (21h49m26.00242815s) [noise=true , address=104.36.82.9:4008 , piece_id=EGIGCEJHELTHOJZWPFWNHCP672MIXAHZIQUK2WDHER77VVJ4NZZA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.487748790 (21h49m26.002851367s) [noise=true , address=193.149.164.143:4693 , piece_id=RCLLY6ABOIXYK6BBO5EWKTDDJ4MUH6IP2WGIHSTER4TEPSF2FYVA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LHoCE9CnM32tbMNqAB7HpmArSVLTycv9o6kSSFNUJsoDeERkQ): protocol: read tcp 159.223.143.39:37688->193.149.164.143:4693: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 07:28:51.487858041 (21h49m26.002960604s) [noise=true , address=45.91.116.11:4120 , piece_id=274GAU5MCZAZOHB3ESPDSJP23Z5FZB636CWGN7ELS2HYY5OFZ7RA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.487890658 (21h49m26.002993237s) [seg_pos={0 0}, upload=22550]: Done waiting for segment. successful: 16 optimal: 14 errs: <nil>
Aug 8 07:28:51.487971404 (21h49m26.00307399s) [upload=22550]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 8 07:28:51.487976917 (21h49m26.003079481s) [upload=22550]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 8 07:28:51.647177827 (21h49m26.162280406s) [upload=22550]: Done uploading segments...
Aug 8 07:28:51.647194442 (21h49m26.162297008s) [upload=22550]: Upload finished. err: <nil>
Aug 8 07:29:34.528501362 (21h50m9.043603939s) [upload=22562]: Starting upload
Aug 8 07:29:34.528548565 (21h50m9.043651143s) [upload=22562]: Done starting upload
Aug 8 07:29:34.528615374 (21h50m9.043717942s) [upload=22562]: Uploading segments...
Aug 8 07:29:34.528903298 (21h50m9.044005874s) [upload=22562]: Got next segment. Inline: false
Aug 8 07:29:34.528940157 (21h50m9.044042733s) [upload=22562]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 8 07:29:34.528944789 (21h50m9.044047366s) [upload=22562]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 8 07:29:34.536881082 (21h50m9.051983672s) [upload=22562]: Next segment err: metaclient: Too Many Requests
Aug 8 07:29:34.536925024 (21h50m9.052027602s) [upload=22562]: Done uploading segments...
Aug 8 07:29:34.536943164 (21h50m9.052045741s) [upload=22562]: Upload finished. err: metaclient: Too Many Requests
Duration: 579.621817ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........] duration:
29.845635ms target:
https://us1.storj.io/login
FAIL (2025-08-08 07:39:19 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.150.199.48:443: connect: connection refused
Duration: 29.867691ms
core
satellite
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............] duration:
455.563864ms
FAIL (2025-08-08 07:39:10 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
g_pos={0 0}, upload=22550]: Begin upload segment...
Aug 8 07:28:51.351770179 (21h49m25.866872766s) [seg_pos={0 0}, upload=22550]: Done begin upload segment.
Aug 8 07:28:51.351799155 (21h49m25.866901747s) [noise=true , address=104.36.82.23:4469 , piece_id=CI6CU5IBYK37MBIQW554EW6PPXVYA6ZF2CJPVZCIXMFFLXZ7UO7Q, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.351816053 (21h49m25.86691864s) [noise=true , address=193.149.164.143:4693 , piece_id=RCLLY6ABOIXYK6BBO5EWKTDDJ4MUH6IP2WGIHSTER4TEPSF2FYVA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.352227136 (21h49m25.867329713s) [noise=true , address=104.36.82.58:4286 , piece_id=36LFLTD2YWV3K7LM4M2GY3JIKLUDOZVRINQT3VLIFBRR55UCNA4Q, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.352322195 (21h49m25.867424772s) [noise=true , address=104.36.82.44:4880 , piece_id=HFXFL7ULX5OTYJFEOC45DI75DGDIHAQA6ZJIXHO22X4GPSEZN75Q, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.352611348 (21h49m25.867713924s) [noise=true , address=104.36.82.53:4139 , piece_id=IGOHVQJMZ3NWLCBKVPVHTVWVHDS43DABQPEMJUTB2TXAQPQOURSQ, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.352702579 (21h49m25.867805142s) [noise=true , address=45.91.116.7:4013 , piece_id=XG4M7HF2E5YIRJ7PVBEGZC2ADPUPAL2QL5TPUAFIYXZUP3BX2Y7A, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.353022021 (21h49m25.868124597s) [noise=true , address=104.36.82.16:4321 , piece_id=MTRDWQYPHECWBAH4ECUGJL2Y2LAUA2RWHX5J6XMR65NZOACBF6PA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.353464633 (21h49m25.868567211s) [noise=true , address=104.36.82.46:4487 , piece_id=JOSIX5WT5A36RXKJNWO5KLBEQOCNQCEZ7A2S4KYIBJHRKN3KT6RQ, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.354418304 (21h49m25.869520884s) [noise=true , address=104.36.82.38:4707 , piece_id=WZWQHYDMTUG2FOTZ6JJRVIXXFIPHJ3I2YIRQXMN7B2W7QGKAFY3A, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.354959848 (21h49m25.87006243s) [noise=true , address=104.36.82.47:4544 , piece_id=YDG3KOKUS5NVBWMQYJ7JJEO3PAXZKPZPUOUWUDCZ4RAVATW67NKQ, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.355461506 (21h49m25.870564086s) [noise=true , address=104.36.82.60:4363 , piece_id=QMR7CM4ASNAKG5PT2XUPFR4MIBHNFYJ7OZMASJ7Q6R2C225L655A, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.355910519 (21h49m25.871013101s) [noise=true , address=104.36.82.2:4741 , piece_id=3NWVL6DQCMDO5PQLSFU3VER7RSHGRAB6YRT32XBFUVLAXGEJQN7A, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.357467009 (21h49m25.872569585s) [noise=true , address=104.36.82.59:4322 , piece_id=NKCSGWGYWOIJWHSQUN2ZYOVZYIRML5G3DXVRJYDTAMHTEN2VES3Q, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.358628018 (21h49m25.873730598s) [noise=true , address=45.91.116.11:4120 , piece_id=274GAU5MCZAZOHB3ESPDSJP23Z5FZB636CWGN7ELS2HYY5OFZ7RA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.359105086 (21h49m25.874207666s) [noise=true , address=104.36.82.45:4907 , piece_id=ZF2SKF5O2WVNHM7CHQ7E4U7HVKQWCJGOZEVIDXO3OU4XHVUFPIGA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.360554866 (21h49m25.875657451s) [noise=true , address=104.36.82.36:4648 , piece_id=3WAWCS3XXWVX3ACYJG2OOS2R34SXOWYSHXVF6AMPFXJFCMYH4SZA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.363015621 (21h49m25.8781182s) [noise=true , address=104.36.82.62:4420 , piece_id=7S6ALI3DNOM7IURBTYY3DLALDQV7R7LLHPQCA4Q6G4HA2SOZ2YMA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.364089235 (21h49m25.87919182s) [noise=true , address=104.36.82.9:4008 , piece_id=EGIGCEJHELTHOJZWPFWNHCP672MIXAHZIQUK2WDHER77VVJ4NZZA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.467747752 (21h49m25.982850337s) [upload=22550]: Next returned nil segment
Aug 8 07:28:51.467767955 (21h49m25.982870543s) [upload=22550]: Waiting for error group managing segments...
Aug 8 07:28:51.473577577 (21h49m25.988680143s) [noise=true , address=45.91.116.7:4013 , piece_id=XG4M7HF2E5YIRJ7PVBEGZC2ADPUPAL2QL5TPUAFIYXZUP3BX2Y7A, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.478188638 (21h49m25.993291216s) [noise=true , address=104.36.82.16:4321 , piece_id=MTRDWQYPHECWBAH4ECUGJL2Y2LAUA2RWHX5J6XMR65NZOACBF6PA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.478342049 (21h49m25.993444627s) [noise=true , address=104.36.82.53:4139 , piece_id=IGOHVQJMZ3NWLCBKVPVHTVWVHDS43DABQPEMJUTB2TXAQPQOURSQ, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.478768394 (21h49m25.993870961s) [noise=true , address=104.36.82.36:4648 , piece_id=3WAWCS3XXWVX3ACYJG2OOS2R34SXOWYSHXVF6AMPFXJFCMYH4SZA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.479336172 (21h49m25.994438735s) [noise=true , address=104.36.82.59:4322 , piece_id=NKCSGWGYWOIJWHSQUN2ZYOVZYIRML5G3DXVRJYDTAMHTEN2VES3Q, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.479515883 (21h49m25.994618457s) [noise=true , address=104.36.82.58:4286 , piece_id=36LFLTD2YWV3K7LM4M2GY3JIKLUDOZVRINQT3VLIFBRR55UCNA4Q, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.480474985 (21h49m25.995577567s) [noise=true , address=104.36.82.2:4741 , piece_id=3NWVL6DQCMDO5PQLSFU3VER7RSHGRAB6YRT32XBFUVLAXGEJQN7A, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.481008628 (21h49m25.996111195s) [noise=true , address=104.36.82.38:4707 , piece_id=WZWQHYDMTUG2FOTZ6JJRVIXXFIPHJ3I2YIRQXMN7B2W7QGKAFY3A, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.481589774 (21h49m25.996692358s) [noise=true , address=104.36.82.60:4363 , piece_id=QMR7CM4ASNAKG5PT2XUPFR4MIBHNFYJ7OZMASJ7Q6R2C225L655A, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.482143054 (21h49m25.99724562s) [noise=true , address=104.36.82.46:4487 , piece_id=JOSIX5WT5A36RXKJNWO5KLBEQOCNQCEZ7A2S4KYIBJHRKN3KT6RQ, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.485172215 (21h49m26.000274779s) [noise=true , address=104.36.82.47:4544 , piece_id=YDG3KOKUS5NVBWMQYJ7JJEO3PAXZKPZPUOUWUDCZ4RAVATW67NKQ, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.486088591 (21h49m26.001191172s) [noise=true , address=104.36.82.44:4880 , piece_id=HFXFL7ULX5OTYJFEOC45DI75DGDIHAQA6ZJIXHO22X4GPSEZN75Q, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.486628345 (21h49m26.001730916s) [noise=true , address=104.36.82.23:4469 , piece_id=CI6CU5IBYK37MBIQW554EW6PPXVYA6ZF2CJPVZCIXMFFLXZ7UO7Q, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.487179958 (21h49m26.002282521s) [noise=true , address=104.36.82.62:4420 , piece_id=7S6ALI3DNOM7IURBTYY3DLALDQV7R7LLHPQCA4Q6G4HA2SOZ2YMA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.487192580 (21h49m26.002295147s) [seg_pos={0 0}, upload=22550]: Segment reached optimal threshold of 14 pieces.
Aug 8 07:28:51.487306760 (21h49m26.002409325s) [noise=true , address=104.36.82.45:4907 , piece_id=ZF2SKF5O2WVNHM7CHQ7E4U7HVKQWCJGOZEVIDXO3OU4XHVUFPIGA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12URx18Ckh7VaAKJ1hdQQucMRu5qiDrAraqaycjYqxETgwZ4uWS): protocol: expected piece hash; context canceled; EOF
Aug 8 07:28:51.487325587 (21h49m26.00242815s) [noise=true , address=104.36.82.9:4008 , piece_id=EGIGCEJHELTHOJZWPFWNHCP672MIXAHZIQUK2WDHER77VVJ4NZZA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.487748790 (21h49m26.002851367s) [noise=true , address=193.149.164.143:4693 , piece_id=RCLLY6ABOIXYK6BBO5EWKTDDJ4MUH6IP2WGIHSTER4TEPSF2FYVA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LHoCE9CnM32tbMNqAB7HpmArSVLTycv9o6kSSFNUJsoDeERkQ): protocol: read tcp 159.223.143.39:37688->193.149.164.143:4693: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 07:28:51.487858041 (21h49m26.002960604s) [noise=true , address=45.91.116.11:4120 , piece_id=274GAU5MCZAZOHB3ESPDSJP23Z5FZB636CWGN7ELS2HYY5OFZ7RA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.487890658 (21h49m26.002993237s) [seg_pos={0 0}, upload=22550]: Done waiting for segment. successful: 16 optimal: 14 errs: <nil>
Aug 8 07:28:51.487971404 (21h49m26.00307399s) [upload=22550]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 8 07:28:51.487976917 (21h49m26.003079481s) [upload=22550]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 8 07:28:51.647177827 (21h49m26.162280406s) [upload=22550]: Done uploading segments...
Aug 8 07:28:51.647194442 (21h49m26.162297008s) [upload=22550]: Upload finished. err: <nil>
Aug 8 07:29:34.528501362 (21h50m9.043603939s) [upload=22562]: Starting upload
Aug 8 07:29:34.528548565 (21h50m9.043651143s) [upload=22562]: Done starting upload
Aug 8 07:29:34.528615374 (21h50m9.043717942s) [upload=22562]: Uploading segments...
Aug 8 07:29:34.528903298 (21h50m9.044005874s) [upload=22562]: Got next segment. Inline: false
Aug 8 07:29:34.528940157 (21h50m9.044042733s) [upload=22562]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 8 07:29:34.528944789 (21h50m9.044047366s) [upload=22562]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 8 07:29:34.536881082 (21h50m9.051983672s) [upload=22562]: Next segment err: metaclient: Too Many Requests
Aug 8 07:29:34.536925024 (21h50m9.052027602s) [upload=22562]: Done uploading segments...
Aug 8 07:29:34.536943164 (21h50m9.052045741s) [upload=22562]: Upload finished. err: metaclient: Too Many Requests
Duration: 455.622863ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [.X..............................................X...........] duration:
188.756307ms
FAIL (2025-08-08 07:39:05 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== nts... Aug 8 07:24:18.405184267 (21h44m52.920286834s) [upload=22475]: Upload finished. err: <nil> Aug 8 07:24:44.148856783 (21h45m18.663959349s) [upload=22481]: Starting upload Aug 8 07:24:44.148908040 (21h45m18.664010617s) [upload=22481]: Done starting upload Aug 8 07:24:44.148978727 (21h45m18.664081294s) [upload=22481]: Uploading segments... Aug 8 07:24:44.149031432 (21h45m18.664134016s) [upload=22481]: Got next segment. Inline: true Aug 8 07:24:44.149047929 (21h45m18.664150495s) [upload=22481]: Waiting for error group managing segments... Aug 8 07:24:44.149231742 (21h45m18.664334308s) [upload=22481]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:24:44.149238484 (21h45m18.664341061s) [upload=22481]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:24:44.149241695 (21h45m18.664344271s) [upload=22481]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:24:44.236175458 (21h45m18.751278039s) [upload=22481]: Done uploading segments... Aug 8 07:24:44.236197562 (21h45m18.75130015s) [upload=22481]: Upload finished. err: <nil> Aug 8 07:25:45.798907959 (21h46m20.31401054s) [upload=22496]: Starting upload Aug 8 07:25:45.798982852 (21h46m20.314085432s) [upload=22496]: Done starting upload Aug 8 07:25:45.799141617 (21h46m20.314244194s) [upload=22496]: Uploading segments... Aug 8 07:25:45.799187034 (21h46m20.314289621s) [upload=22496]: Got next segment. Inline: true Aug 8 07:25:45.799192794 (21h46m20.314295378s) [upload=22496]: Waiting for error group managing segments... Aug 8 07:25:45.799260506 (21h46m20.314363086s) [upload=22496]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:25:45.799265543 (21h46m20.314368123s) [upload=22496]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:25:45.799269614 (21h46m20.314372195s) [upload=22496]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:25:45.856873466 (21h46m20.371976033s) [upload=22496]: Done uploading segments... Aug 8 07:25:45.856894517 (21h46m20.371997083s) [upload=22496]: Upload finished. err: <nil> Aug 8 07:26:45.160306388 (21h47m19.67540897s) [upload=22512]: Starting upload Aug 8 07:26:45.160369840 (21h47m19.675472407s) [upload=22512]: Done starting upload Aug 8 07:26:45.160448308 (21h47m19.675550874s) [upload=22512]: Uploading segments... Aug 8 07:26:45.160512759 (21h47m19.675615339s) [upload=22512]: Got next segment. Inline: true Aug 8 07:26:45.160518821 (21h47m19.675621389s) [upload=22512]: Waiting for error group managing segments... Aug 8 07:26:45.160566486 (21h47m19.675669054s) [upload=22512]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:26:45.160571363 (21h47m19.675673943s) [upload=22512]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:26:45.160575116 (21h47m19.675677698s) [upload=22512]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:26:45.219521344 (21h47m19.73462392s) [upload=22512]: Done uploading segments... Aug 8 07:26:45.219537755 (21h47m19.73464032s) [upload=22512]: Upload finished. err: <nil> Aug 8 07:27:50.008745364 (21h48m24.523847971s) [upload=22529]: Starting upload Aug 8 07:27:50.008828662 (21h48m24.523931252s) [upload=22529]: Done starting upload Aug 8 07:27:50.008913720 (21h48m24.524016289s) [upload=22529]: Uploading segments... Aug 8 07:27:50.008968901 (21h48m24.524071486s) [upload=22529]: Got next segment. Inline: true Aug 8 07:27:50.008973893 (21h48m24.524076469s) [upload=22529]: Waiting for error group managing segments... Aug 8 07:27:50.009023439 (21h48m24.524126015s) [upload=22529]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:27:50.009028404 (21h48m24.524130983s) [upload=22529]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:27:50.009032740 (21h48m24.524135324s) [upload=22529]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:27:50.072222330 (21h48m24.587324909s) [upload=22529]: Done uploading segments... Aug 8 07:27:50.072260086 (21h48m24.587362652s) [upload=22529]: Upload finished. err: <nil> Aug 8 07:28:08.797868182 (21h48m43.312970759s) [upload=22537]: Starting upload Aug 8 07:28:08.797909617 (21h48m43.313012202s) [upload=22537]: Done starting upload Aug 8 07:28:08.797978107 (21h48m43.313080676s) [upload=22537]: Uploading segments... Aug 8 07:28:08.798028549 (21h48m43.313131134s) [upload=22537]: Got next segment. Inline: true Aug 8 07:28:08.798033746 (21h48m43.313136324s) [upload=22537]: Waiting for error group managing segments... Aug 8 07:28:08.798088133 (21h48m43.313190709s) [upload=22537]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:28:08.798092673 (21h48m43.313195249s) [upload=22537]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:28:08.798096028 (21h48m43.313198604s) [upload=22537]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:28:08.856181519 (21h48m43.371284096s) [upload=22537]: Done uploading segments... Aug 8 07:28:08.856219288 (21h48m43.371321856s) [upload=22537]: Upload finished. err: <nil> Aug 8 07:28:26.426406302 (21h49m0.941508883s) [upload=22543]: Starting upload Aug 8 07:28:26.426484520 (21h49m0.94158709s) [upload=22543]: Done starting upload Aug 8 07:28:26.426561912 (21h49m0.941664479s) [upload=22543]: Uploading segments... Aug 8 07:28:26.426613562 (21h49m0.941716147s) [upload=22543]: Got next segment. Inline: true Aug 8 07:28:26.426618334 (21h49m0.94172091s) [upload=22543]: Waiting for error group managing segments... Aug 8 07:28:26.426667358 (21h49m0.941769934s) [upload=22543]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:28:26.426672477 (21h49m0.941775052s) [upload=22543]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:28:26.426676980 (21h49m0.941779557s) [upload=22543]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:28:26.739786156 (21h49m1.254888734s) [upload=22543]: Done uploading segments... Aug 8 07:28:26.739818088 (21h49m1.254920663s) [upload=22543]: Upload finished. err: <nil> Aug 8 07:28:44.768272418 (21h49m19.283374995s) [upload=22547]: Starting upload Aug 8 07:28:44.768352182 (21h49m19.28345476s) [upload=22547]: Done starting upload Aug 8 07:28:44.768463680 (21h49m19.283566246s) [upload=22547]: Uploading segments... Aug 8 07:28:44.768552166 (21h49m19.283654733s) [upload=22547]: Got next segment. Inline: true Aug 8 07:28:44.768558003 (21h49m19.28366057s) [upload=22547]: Waiting for error group managing segments... Aug 8 07:28:44.768617460 (21h49m19.28372004s) [upload=22547]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:28:44.768622292 (21h49m19.283724868s) [upload=22547]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:28:44.768625505 (21h49m19.283728082s) [upload=22547]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:28:44.832641596 (21h49m19.347744163s) [upload=22547]: Done uploading segments... Aug 8 07:28:44.832661990 (21h49m19.347764559s) [upload=22547]: Upload finished. err: <nil> Aug 8 07:29:19.907807846 (21h49m54.422910428s) [upload=22558]: Starting upload Aug 8 07:29:19.907919209 (21h49m54.423021801s) [upload=22558]: Done starting upload Aug 8 07:29:19.908035700 (21h49m54.423138281s) [upload=22558]: Uploading segments... Aug 8 07:29:19.908115743 (21h49m54.423218334s) [upload=22558]: Got next segment. Inline: true Aug 8 07:29:19.908123216 (21h49m54.423225802s) [upload=22558]: Waiting for error group managing segments... Aug 8 07:29:19.908206488 (21h49m54.423309073s) [upload=22558]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:29:19.908213079 (21h49m54.423315659s) [upload=22558]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:29:19.908216828 (21h49m54.423319408s) [upload=22558]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:29:19.968416399 (21h49m54.483518987s) [upload=22558]: Done uploading segments... Aug 8 07:29:19.968437892 (21h49m54.48354047s) [upload=22558]: Upload finished. err: <nil> Aug 8 07:30:04.570811298 (21h50m39.085913875s) [upload=22574]: Starting upload Aug 8 07:30:04.570856561 (21h50m39.085959129s) [upload=22574]: Done starting upload Aug 8 07:30:04.570937395 (21h50m39.086039963s) [upload=22574]: Uploading segments... Aug 8 07:30:04.570993290 (21h50m39.086095879s) [upload=22574]: Got next segment. Inline: true Aug 8 07:30:04.570999714 (21h50m39.086102306s) [upload=22574]: Waiting for error group managing segments... Aug 8 07:30:04.571094735 (21h50m39.086197316s) [upload=22574]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:30:04.571101887 (21h50m39.086204467s) [upload=22574]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:30:04.571105790 (21h50m39.086208372s) [upload=22574]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:30:04.649336948 (21h50m39.164439536s) [upload=22574]: Done uploading segments... Aug 8 07:30:04.649362004 (21h50m39.164464585s) [upload=22574]: Upload finished. err: <nil> Aug 8 07:30:54.851564431 (21h51m29.366666997s) [upload=22587]: Starting upload Aug 8 07:30:54.851618044 (21h51m29.36672062s) [upload=22587]: Done starting upload Aug 8 07:30:54.851712059 (21h51m29.366814635s) [upload=22587]: Uploading segments... Aug 8 07:30:54.851783178 (21h51m29.36688576s) [upload=22587]: Got next segment. Inline: true Aug 8 07:30:54.851789517 (21h51m29.366892093s) [upload=22587]: Waiting for error group managing segments... Aug 8 07:30:54.851854337 (21h51m29.366956924s) [upload=22587]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:30:54.851861334 (21h51m29.366963924s) [upload=22587]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:30:54.851866059 (21h51m29.366968642s) [upload=22587]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:30:54.860593007 (21h51m29.375695595s) [upload=22587]: Done uploading segments... Aug 8 07:30:54.860627519 (21h51m29.375730095s) [upload=22587]: Upload finished. err: metaclient: Too Many Requests
Duration: 188.791359ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [..X..............................................X..........] duration:
176.278547ms
FAIL (2025-08-08 07:38:33 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== nts... Aug 8 07:24:18.405184267 (21h44m52.920286834s) [upload=22475]: Upload finished. err: <nil> Aug 8 07:24:44.148856783 (21h45m18.663959349s) [upload=22481]: Starting upload Aug 8 07:24:44.148908040 (21h45m18.664010617s) [upload=22481]: Done starting upload Aug 8 07:24:44.148978727 (21h45m18.664081294s) [upload=22481]: Uploading segments... Aug 8 07:24:44.149031432 (21h45m18.664134016s) [upload=22481]: Got next segment. Inline: true Aug 8 07:24:44.149047929 (21h45m18.664150495s) [upload=22481]: Waiting for error group managing segments... Aug 8 07:24:44.149231742 (21h45m18.664334308s) [upload=22481]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:24:44.149238484 (21h45m18.664341061s) [upload=22481]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:24:44.149241695 (21h45m18.664344271s) [upload=22481]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:24:44.236175458 (21h45m18.751278039s) [upload=22481]: Done uploading segments... Aug 8 07:24:44.236197562 (21h45m18.75130015s) [upload=22481]: Upload finished. err: <nil> Aug 8 07:25:45.798907959 (21h46m20.31401054s) [upload=22496]: Starting upload Aug 8 07:25:45.798982852 (21h46m20.314085432s) [upload=22496]: Done starting upload Aug 8 07:25:45.799141617 (21h46m20.314244194s) [upload=22496]: Uploading segments... Aug 8 07:25:45.799187034 (21h46m20.314289621s) [upload=22496]: Got next segment. Inline: true Aug 8 07:25:45.799192794 (21h46m20.314295378s) [upload=22496]: Waiting for error group managing segments... Aug 8 07:25:45.799260506 (21h46m20.314363086s) [upload=22496]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:25:45.799265543 (21h46m20.314368123s) [upload=22496]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:25:45.799269614 (21h46m20.314372195s) [upload=22496]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:25:45.856873466 (21h46m20.371976033s) [upload=22496]: Done uploading segments... Aug 8 07:25:45.856894517 (21h46m20.371997083s) [upload=22496]: Upload finished. err: <nil> Aug 8 07:26:45.160306388 (21h47m19.67540897s) [upload=22512]: Starting upload Aug 8 07:26:45.160369840 (21h47m19.675472407s) [upload=22512]: Done starting upload Aug 8 07:26:45.160448308 (21h47m19.675550874s) [upload=22512]: Uploading segments... Aug 8 07:26:45.160512759 (21h47m19.675615339s) [upload=22512]: Got next segment. Inline: true Aug 8 07:26:45.160518821 (21h47m19.675621389s) [upload=22512]: Waiting for error group managing segments... Aug 8 07:26:45.160566486 (21h47m19.675669054s) [upload=22512]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:26:45.160571363 (21h47m19.675673943s) [upload=22512]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:26:45.160575116 (21h47m19.675677698s) [upload=22512]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:26:45.219521344 (21h47m19.73462392s) [upload=22512]: Done uploading segments... Aug 8 07:26:45.219537755 (21h47m19.73464032s) [upload=22512]: Upload finished. err: <nil> Aug 8 07:27:50.008745364 (21h48m24.523847971s) [upload=22529]: Starting upload Aug 8 07:27:50.008828662 (21h48m24.523931252s) [upload=22529]: Done starting upload Aug 8 07:27:50.008913720 (21h48m24.524016289s) [upload=22529]: Uploading segments... Aug 8 07:27:50.008968901 (21h48m24.524071486s) [upload=22529]: Got next segment. Inline: true Aug 8 07:27:50.008973893 (21h48m24.524076469s) [upload=22529]: Waiting for error group managing segments... Aug 8 07:27:50.009023439 (21h48m24.524126015s) [upload=22529]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:27:50.009028404 (21h48m24.524130983s) [upload=22529]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:27:50.009032740 (21h48m24.524135324s) [upload=22529]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:27:50.072222330 (21h48m24.587324909s) [upload=22529]: Done uploading segments... Aug 8 07:27:50.072260086 (21h48m24.587362652s) [upload=22529]: Upload finished. err: <nil> Aug 8 07:28:08.797868182 (21h48m43.312970759s) [upload=22537]: Starting upload Aug 8 07:28:08.797909617 (21h48m43.313012202s) [upload=22537]: Done starting upload Aug 8 07:28:08.797978107 (21h48m43.313080676s) [upload=22537]: Uploading segments... Aug 8 07:28:08.798028549 (21h48m43.313131134s) [upload=22537]: Got next segment. Inline: true Aug 8 07:28:08.798033746 (21h48m43.313136324s) [upload=22537]: Waiting for error group managing segments... Aug 8 07:28:08.798088133 (21h48m43.313190709s) [upload=22537]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:28:08.798092673 (21h48m43.313195249s) [upload=22537]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:28:08.798096028 (21h48m43.313198604s) [upload=22537]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:28:08.856181519 (21h48m43.371284096s) [upload=22537]: Done uploading segments... Aug 8 07:28:08.856219288 (21h48m43.371321856s) [upload=22537]: Upload finished. err: <nil> Aug 8 07:28:26.426406302 (21h49m0.941508883s) [upload=22543]: Starting upload Aug 8 07:28:26.426484520 (21h49m0.94158709s) [upload=22543]: Done starting upload Aug 8 07:28:26.426561912 (21h49m0.941664479s) [upload=22543]: Uploading segments... Aug 8 07:28:26.426613562 (21h49m0.941716147s) [upload=22543]: Got next segment. Inline: true Aug 8 07:28:26.426618334 (21h49m0.94172091s) [upload=22543]: Waiting for error group managing segments... Aug 8 07:28:26.426667358 (21h49m0.941769934s) [upload=22543]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:28:26.426672477 (21h49m0.941775052s) [upload=22543]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:28:26.426676980 (21h49m0.941779557s) [upload=22543]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:28:26.739786156 (21h49m1.254888734s) [upload=22543]: Done uploading segments... Aug 8 07:28:26.739818088 (21h49m1.254920663s) [upload=22543]: Upload finished. err: <nil> Aug 8 07:28:44.768272418 (21h49m19.283374995s) [upload=22547]: Starting upload Aug 8 07:28:44.768352182 (21h49m19.28345476s) [upload=22547]: Done starting upload Aug 8 07:28:44.768463680 (21h49m19.283566246s) [upload=22547]: Uploading segments... Aug 8 07:28:44.768552166 (21h49m19.283654733s) [upload=22547]: Got next segment. Inline: true Aug 8 07:28:44.768558003 (21h49m19.28366057s) [upload=22547]: Waiting for error group managing segments... Aug 8 07:28:44.768617460 (21h49m19.28372004s) [upload=22547]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:28:44.768622292 (21h49m19.283724868s) [upload=22547]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:28:44.768625505 (21h49m19.283728082s) [upload=22547]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:28:44.832641596 (21h49m19.347744163s) [upload=22547]: Done uploading segments... Aug 8 07:28:44.832661990 (21h49m19.347764559s) [upload=22547]: Upload finished. err: <nil> Aug 8 07:29:19.907807846 (21h49m54.422910428s) [upload=22558]: Starting upload Aug 8 07:29:19.907919209 (21h49m54.423021801s) [upload=22558]: Done starting upload Aug 8 07:29:19.908035700 (21h49m54.423138281s) [upload=22558]: Uploading segments... Aug 8 07:29:19.908115743 (21h49m54.423218334s) [upload=22558]: Got next segment. Inline: true Aug 8 07:29:19.908123216 (21h49m54.423225802s) [upload=22558]: Waiting for error group managing segments... Aug 8 07:29:19.908206488 (21h49m54.423309073s) [upload=22558]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:29:19.908213079 (21h49m54.423315659s) [upload=22558]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:29:19.908216828 (21h49m54.423319408s) [upload=22558]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:29:19.968416399 (21h49m54.483518987s) [upload=22558]: Done uploading segments... Aug 8 07:29:19.968437892 (21h49m54.48354047s) [upload=22558]: Upload finished. err: <nil> Aug 8 07:30:04.570811298 (21h50m39.085913875s) [upload=22574]: Starting upload Aug 8 07:30:04.570856561 (21h50m39.085959129s) [upload=22574]: Done starting upload Aug 8 07:30:04.570937395 (21h50m39.086039963s) [upload=22574]: Uploading segments... Aug 8 07:30:04.570993290 (21h50m39.086095879s) [upload=22574]: Got next segment. Inline: true Aug 8 07:30:04.570999714 (21h50m39.086102306s) [upload=22574]: Waiting for error group managing segments... Aug 8 07:30:04.571094735 (21h50m39.086197316s) [upload=22574]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:30:04.571101887 (21h50m39.086204467s) [upload=22574]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:30:04.571105790 (21h50m39.086208372s) [upload=22574]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:30:04.649336948 (21h50m39.164439536s) [upload=22574]: Done uploading segments... Aug 8 07:30:04.649362004 (21h50m39.164464585s) [upload=22574]: Upload finished. err: <nil> Aug 8 07:30:54.851564431 (21h51m29.366666997s) [upload=22587]: Starting upload Aug 8 07:30:54.851618044 (21h51m29.36672062s) [upload=22587]: Done starting upload Aug 8 07:30:54.851712059 (21h51m29.366814635s) [upload=22587]: Uploading segments... Aug 8 07:30:54.851783178 (21h51m29.36688576s) [upload=22587]: Got next segment. Inline: true Aug 8 07:30:54.851789517 (21h51m29.366892093s) [upload=22587]: Waiting for error group managing segments... Aug 8 07:30:54.851854337 (21h51m29.366956924s) [upload=22587]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:30:54.851861334 (21h51m29.366963924s) [upload=22587]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:30:54.851866059 (21h51m29.366968642s) [upload=22587]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:30:54.860593007 (21h51m29.375695595s) [upload=22587]: Done uploading segments... Aug 8 07:30:54.860627519 (21h51m29.375730095s) [upload=22587]: Upload finished. err: metaclient: Too Many Requests
Duration: 176.320937ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [...X..............................................X.........] duration:
182.197831ms
FAIL (2025-08-08 07:38:14 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== nts... Aug 8 07:24:18.405184267 (21h44m52.920286834s) [upload=22475]: Upload finished. err: <nil> Aug 8 07:24:44.148856783 (21h45m18.663959349s) [upload=22481]: Starting upload Aug 8 07:24:44.148908040 (21h45m18.664010617s) [upload=22481]: Done starting upload Aug 8 07:24:44.148978727 (21h45m18.664081294s) [upload=22481]: Uploading segments... Aug 8 07:24:44.149031432 (21h45m18.664134016s) [upload=22481]: Got next segment. Inline: true Aug 8 07:24:44.149047929 (21h45m18.664150495s) [upload=22481]: Waiting for error group managing segments... Aug 8 07:24:44.149231742 (21h45m18.664334308s) [upload=22481]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:24:44.149238484 (21h45m18.664341061s) [upload=22481]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:24:44.149241695 (21h45m18.664344271s) [upload=22481]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:24:44.236175458 (21h45m18.751278039s) [upload=22481]: Done uploading segments... Aug 8 07:24:44.236197562 (21h45m18.75130015s) [upload=22481]: Upload finished. err: <nil> Aug 8 07:25:45.798907959 (21h46m20.31401054s) [upload=22496]: Starting upload Aug 8 07:25:45.798982852 (21h46m20.314085432s) [upload=22496]: Done starting upload Aug 8 07:25:45.799141617 (21h46m20.314244194s) [upload=22496]: Uploading segments... Aug 8 07:25:45.799187034 (21h46m20.314289621s) [upload=22496]: Got next segment. Inline: true Aug 8 07:25:45.799192794 (21h46m20.314295378s) [upload=22496]: Waiting for error group managing segments... Aug 8 07:25:45.799260506 (21h46m20.314363086s) [upload=22496]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:25:45.799265543 (21h46m20.314368123s) [upload=22496]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:25:45.799269614 (21h46m20.314372195s) [upload=22496]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:25:45.856873466 (21h46m20.371976033s) [upload=22496]: Done uploading segments... Aug 8 07:25:45.856894517 (21h46m20.371997083s) [upload=22496]: Upload finished. err: <nil> Aug 8 07:26:45.160306388 (21h47m19.67540897s) [upload=22512]: Starting upload Aug 8 07:26:45.160369840 (21h47m19.675472407s) [upload=22512]: Done starting upload Aug 8 07:26:45.160448308 (21h47m19.675550874s) [upload=22512]: Uploading segments... Aug 8 07:26:45.160512759 (21h47m19.675615339s) [upload=22512]: Got next segment. Inline: true Aug 8 07:26:45.160518821 (21h47m19.675621389s) [upload=22512]: Waiting for error group managing segments... Aug 8 07:26:45.160566486 (21h47m19.675669054s) [upload=22512]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:26:45.160571363 (21h47m19.675673943s) [upload=22512]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:26:45.160575116 (21h47m19.675677698s) [upload=22512]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:26:45.219521344 (21h47m19.73462392s) [upload=22512]: Done uploading segments... Aug 8 07:26:45.219537755 (21h47m19.73464032s) [upload=22512]: Upload finished. err: <nil> Aug 8 07:27:50.008745364 (21h48m24.523847971s) [upload=22529]: Starting upload Aug 8 07:27:50.008828662 (21h48m24.523931252s) [upload=22529]: Done starting upload Aug 8 07:27:50.008913720 (21h48m24.524016289s) [upload=22529]: Uploading segments... Aug 8 07:27:50.008968901 (21h48m24.524071486s) [upload=22529]: Got next segment. Inline: true Aug 8 07:27:50.008973893 (21h48m24.524076469s) [upload=22529]: Waiting for error group managing segments... Aug 8 07:27:50.009023439 (21h48m24.524126015s) [upload=22529]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:27:50.009028404 (21h48m24.524130983s) [upload=22529]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:27:50.009032740 (21h48m24.524135324s) [upload=22529]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:27:50.072222330 (21h48m24.587324909s) [upload=22529]: Done uploading segments... Aug 8 07:27:50.072260086 (21h48m24.587362652s) [upload=22529]: Upload finished. err: <nil> Aug 8 07:28:08.797868182 (21h48m43.312970759s) [upload=22537]: Starting upload Aug 8 07:28:08.797909617 (21h48m43.313012202s) [upload=22537]: Done starting upload Aug 8 07:28:08.797978107 (21h48m43.313080676s) [upload=22537]: Uploading segments... Aug 8 07:28:08.798028549 (21h48m43.313131134s) [upload=22537]: Got next segment. Inline: true Aug 8 07:28:08.798033746 (21h48m43.313136324s) [upload=22537]: Waiting for error group managing segments... Aug 8 07:28:08.798088133 (21h48m43.313190709s) [upload=22537]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:28:08.798092673 (21h48m43.313195249s) [upload=22537]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:28:08.798096028 (21h48m43.313198604s) [upload=22537]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:28:08.856181519 (21h48m43.371284096s) [upload=22537]: Done uploading segments... Aug 8 07:28:08.856219288 (21h48m43.371321856s) [upload=22537]: Upload finished. err: <nil> Aug 8 07:28:26.426406302 (21h49m0.941508883s) [upload=22543]: Starting upload Aug 8 07:28:26.426484520 (21h49m0.94158709s) [upload=22543]: Done starting upload Aug 8 07:28:26.426561912 (21h49m0.941664479s) [upload=22543]: Uploading segments... Aug 8 07:28:26.426613562 (21h49m0.941716147s) [upload=22543]: Got next segment. Inline: true Aug 8 07:28:26.426618334 (21h49m0.94172091s) [upload=22543]: Waiting for error group managing segments... Aug 8 07:28:26.426667358 (21h49m0.941769934s) [upload=22543]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:28:26.426672477 (21h49m0.941775052s) [upload=22543]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:28:26.426676980 (21h49m0.941779557s) [upload=22543]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:28:26.739786156 (21h49m1.254888734s) [upload=22543]: Done uploading segments... Aug 8 07:28:26.739818088 (21h49m1.254920663s) [upload=22543]: Upload finished. err: <nil> Aug 8 07:28:44.768272418 (21h49m19.283374995s) [upload=22547]: Starting upload Aug 8 07:28:44.768352182 (21h49m19.28345476s) [upload=22547]: Done starting upload Aug 8 07:28:44.768463680 (21h49m19.283566246s) [upload=22547]: Uploading segments... Aug 8 07:28:44.768552166 (21h49m19.283654733s) [upload=22547]: Got next segment. Inline: true Aug 8 07:28:44.768558003 (21h49m19.28366057s) [upload=22547]: Waiting for error group managing segments... Aug 8 07:28:44.768617460 (21h49m19.28372004s) [upload=22547]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:28:44.768622292 (21h49m19.283724868s) [upload=22547]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:28:44.768625505 (21h49m19.283728082s) [upload=22547]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:28:44.832641596 (21h49m19.347744163s) [upload=22547]: Done uploading segments... Aug 8 07:28:44.832661990 (21h49m19.347764559s) [upload=22547]: Upload finished. err: <nil> Aug 8 07:29:19.907807846 (21h49m54.422910428s) [upload=22558]: Starting upload Aug 8 07:29:19.907919209 (21h49m54.423021801s) [upload=22558]: Done starting upload Aug 8 07:29:19.908035700 (21h49m54.423138281s) [upload=22558]: Uploading segments... Aug 8 07:29:19.908115743 (21h49m54.423218334s) [upload=22558]: Got next segment. Inline: true Aug 8 07:29:19.908123216 (21h49m54.423225802s) [upload=22558]: Waiting for error group managing segments... Aug 8 07:29:19.908206488 (21h49m54.423309073s) [upload=22558]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:29:19.908213079 (21h49m54.423315659s) [upload=22558]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:29:19.908216828 (21h49m54.423319408s) [upload=22558]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:29:19.968416399 (21h49m54.483518987s) [upload=22558]: Done uploading segments... Aug 8 07:29:19.968437892 (21h49m54.48354047s) [upload=22558]: Upload finished. err: <nil> Aug 8 07:30:04.570811298 (21h50m39.085913875s) [upload=22574]: Starting upload Aug 8 07:30:04.570856561 (21h50m39.085959129s) [upload=22574]: Done starting upload Aug 8 07:30:04.570937395 (21h50m39.086039963s) [upload=22574]: Uploading segments... Aug 8 07:30:04.570993290 (21h50m39.086095879s) [upload=22574]: Got next segment. Inline: true Aug 8 07:30:04.570999714 (21h50m39.086102306s) [upload=22574]: Waiting for error group managing segments... Aug 8 07:30:04.571094735 (21h50m39.086197316s) [upload=22574]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:30:04.571101887 (21h50m39.086204467s) [upload=22574]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:30:04.571105790 (21h50m39.086208372s) [upload=22574]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:30:04.649336948 (21h50m39.164439536s) [upload=22574]: Done uploading segments... Aug 8 07:30:04.649362004 (21h50m39.164464585s) [upload=22574]: Upload finished. err: <nil> Aug 8 07:30:54.851564431 (21h51m29.366666997s) [upload=22587]: Starting upload Aug 8 07:30:54.851618044 (21h51m29.36672062s) [upload=22587]: Done starting upload Aug 8 07:30:54.851712059 (21h51m29.366814635s) [upload=22587]: Uploading segments... Aug 8 07:30:54.851783178 (21h51m29.36688576s) [upload=22587]: Got next segment. Inline: true Aug 8 07:30:54.851789517 (21h51m29.366892093s) [upload=22587]: Waiting for error group managing segments... Aug 8 07:30:54.851854337 (21h51m29.366956924s) [upload=22587]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:30:54.851861334 (21h51m29.366963924s) [upload=22587]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:30:54.851866059 (21h51m29.366968642s) [upload=22587]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:30:54.860593007 (21h51m29.375695595s) [upload=22587]: Done uploading segments... Aug 8 07:30:54.860627519 (21h51m29.375730095s) [upload=22587]: Upload finished. err: metaclient: Too Many Requests
Duration: 182.278136ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......] duration:
32.435003ms target:
https://us1.storj.io/login
FAIL (2025-08-08 07:38:14 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.150.199.48:443: connect: connection refused
Duration: 32.455544ms
core
satellite
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........] duration:
398.853535ms
FAIL (2025-08-08 07:38:04 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
g_pos={0 0}, upload=22550]: Begin upload segment...
Aug 8 07:28:51.351770179 (21h49m25.866872766s) [seg_pos={0 0}, upload=22550]: Done begin upload segment.
Aug 8 07:28:51.351799155 (21h49m25.866901747s) [noise=true , address=104.36.82.23:4469 , piece_id=CI6CU5IBYK37MBIQW554EW6PPXVYA6ZF2CJPVZCIXMFFLXZ7UO7Q, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.351816053 (21h49m25.86691864s) [noise=true , address=193.149.164.143:4693 , piece_id=RCLLY6ABOIXYK6BBO5EWKTDDJ4MUH6IP2WGIHSTER4TEPSF2FYVA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.352227136 (21h49m25.867329713s) [noise=true , address=104.36.82.58:4286 , piece_id=36LFLTD2YWV3K7LM4M2GY3JIKLUDOZVRINQT3VLIFBRR55UCNA4Q, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.352322195 (21h49m25.867424772s) [noise=true , address=104.36.82.44:4880 , piece_id=HFXFL7ULX5OTYJFEOC45DI75DGDIHAQA6ZJIXHO22X4GPSEZN75Q, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.352611348 (21h49m25.867713924s) [noise=true , address=104.36.82.53:4139 , piece_id=IGOHVQJMZ3NWLCBKVPVHTVWVHDS43DABQPEMJUTB2TXAQPQOURSQ, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.352702579 (21h49m25.867805142s) [noise=true , address=45.91.116.7:4013 , piece_id=XG4M7HF2E5YIRJ7PVBEGZC2ADPUPAL2QL5TPUAFIYXZUP3BX2Y7A, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.353022021 (21h49m25.868124597s) [noise=true , address=104.36.82.16:4321 , piece_id=MTRDWQYPHECWBAH4ECUGJL2Y2LAUA2RWHX5J6XMR65NZOACBF6PA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.353464633 (21h49m25.868567211s) [noise=true , address=104.36.82.46:4487 , piece_id=JOSIX5WT5A36RXKJNWO5KLBEQOCNQCEZ7A2S4KYIBJHRKN3KT6RQ, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.354418304 (21h49m25.869520884s) [noise=true , address=104.36.82.38:4707 , piece_id=WZWQHYDMTUG2FOTZ6JJRVIXXFIPHJ3I2YIRQXMN7B2W7QGKAFY3A, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.354959848 (21h49m25.87006243s) [noise=true , address=104.36.82.47:4544 , piece_id=YDG3KOKUS5NVBWMQYJ7JJEO3PAXZKPZPUOUWUDCZ4RAVATW67NKQ, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.355461506 (21h49m25.870564086s) [noise=true , address=104.36.82.60:4363 , piece_id=QMR7CM4ASNAKG5PT2XUPFR4MIBHNFYJ7OZMASJ7Q6R2C225L655A, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.355910519 (21h49m25.871013101s) [noise=true , address=104.36.82.2:4741 , piece_id=3NWVL6DQCMDO5PQLSFU3VER7RSHGRAB6YRT32XBFUVLAXGEJQN7A, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.357467009 (21h49m25.872569585s) [noise=true , address=104.36.82.59:4322 , piece_id=NKCSGWGYWOIJWHSQUN2ZYOVZYIRML5G3DXVRJYDTAMHTEN2VES3Q, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.358628018 (21h49m25.873730598s) [noise=true , address=45.91.116.11:4120 , piece_id=274GAU5MCZAZOHB3ESPDSJP23Z5FZB636CWGN7ELS2HYY5OFZ7RA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.359105086 (21h49m25.874207666s) [noise=true , address=104.36.82.45:4907 , piece_id=ZF2SKF5O2WVNHM7CHQ7E4U7HVKQWCJGOZEVIDXO3OU4XHVUFPIGA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.360554866 (21h49m25.875657451s) [noise=true , address=104.36.82.36:4648 , piece_id=3WAWCS3XXWVX3ACYJG2OOS2R34SXOWYSHXVF6AMPFXJFCMYH4SZA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.363015621 (21h49m25.8781182s) [noise=true , address=104.36.82.62:4420 , piece_id=7S6ALI3DNOM7IURBTYY3DLALDQV7R7LLHPQCA4Q6G4HA2SOZ2YMA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.364089235 (21h49m25.87919182s) [noise=true , address=104.36.82.9:4008 , piece_id=EGIGCEJHELTHOJZWPFWNHCP672MIXAHZIQUK2WDHER77VVJ4NZZA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.467747752 (21h49m25.982850337s) [upload=22550]: Next returned nil segment
Aug 8 07:28:51.467767955 (21h49m25.982870543s) [upload=22550]: Waiting for error group managing segments...
Aug 8 07:28:51.473577577 (21h49m25.988680143s) [noise=true , address=45.91.116.7:4013 , piece_id=XG4M7HF2E5YIRJ7PVBEGZC2ADPUPAL2QL5TPUAFIYXZUP3BX2Y7A, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.478188638 (21h49m25.993291216s) [noise=true , address=104.36.82.16:4321 , piece_id=MTRDWQYPHECWBAH4ECUGJL2Y2LAUA2RWHX5J6XMR65NZOACBF6PA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.478342049 (21h49m25.993444627s) [noise=true , address=104.36.82.53:4139 , piece_id=IGOHVQJMZ3NWLCBKVPVHTVWVHDS43DABQPEMJUTB2TXAQPQOURSQ, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.478768394 (21h49m25.993870961s) [noise=true , address=104.36.82.36:4648 , piece_id=3WAWCS3XXWVX3ACYJG2OOS2R34SXOWYSHXVF6AMPFXJFCMYH4SZA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.479336172 (21h49m25.994438735s) [noise=true , address=104.36.82.59:4322 , piece_id=NKCSGWGYWOIJWHSQUN2ZYOVZYIRML5G3DXVRJYDTAMHTEN2VES3Q, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.479515883 (21h49m25.994618457s) [noise=true , address=104.36.82.58:4286 , piece_id=36LFLTD2YWV3K7LM4M2GY3JIKLUDOZVRINQT3VLIFBRR55UCNA4Q, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.480474985 (21h49m25.995577567s) [noise=true , address=104.36.82.2:4741 , piece_id=3NWVL6DQCMDO5PQLSFU3VER7RSHGRAB6YRT32XBFUVLAXGEJQN7A, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.481008628 (21h49m25.996111195s) [noise=true , address=104.36.82.38:4707 , piece_id=WZWQHYDMTUG2FOTZ6JJRVIXXFIPHJ3I2YIRQXMN7B2W7QGKAFY3A, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.481589774 (21h49m25.996692358s) [noise=true , address=104.36.82.60:4363 , piece_id=QMR7CM4ASNAKG5PT2XUPFR4MIBHNFYJ7OZMASJ7Q6R2C225L655A, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.482143054 (21h49m25.99724562s) [noise=true , address=104.36.82.46:4487 , piece_id=JOSIX5WT5A36RXKJNWO5KLBEQOCNQCEZ7A2S4KYIBJHRKN3KT6RQ, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.485172215 (21h49m26.000274779s) [noise=true , address=104.36.82.47:4544 , piece_id=YDG3KOKUS5NVBWMQYJ7JJEO3PAXZKPZPUOUWUDCZ4RAVATW67NKQ, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.486088591 (21h49m26.001191172s) [noise=true , address=104.36.82.44:4880 , piece_id=HFXFL7ULX5OTYJFEOC45DI75DGDIHAQA6ZJIXHO22X4GPSEZN75Q, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.486628345 (21h49m26.001730916s) [noise=true , address=104.36.82.23:4469 , piece_id=CI6CU5IBYK37MBIQW554EW6PPXVYA6ZF2CJPVZCIXMFFLXZ7UO7Q, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.487179958 (21h49m26.002282521s) [noise=true , address=104.36.82.62:4420 , piece_id=7S6ALI3DNOM7IURBTYY3DLALDQV7R7LLHPQCA4Q6G4HA2SOZ2YMA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.487192580 (21h49m26.002295147s) [seg_pos={0 0}, upload=22550]: Segment reached optimal threshold of 14 pieces.
Aug 8 07:28:51.487306760 (21h49m26.002409325s) [noise=true , address=104.36.82.45:4907 , piece_id=ZF2SKF5O2WVNHM7CHQ7E4U7HVKQWCJGOZEVIDXO3OU4XHVUFPIGA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12URx18Ckh7VaAKJ1hdQQucMRu5qiDrAraqaycjYqxETgwZ4uWS): protocol: expected piece hash; context canceled; EOF
Aug 8 07:28:51.487325587 (21h49m26.00242815s) [noise=true , address=104.36.82.9:4008 , piece_id=EGIGCEJHELTHOJZWPFWNHCP672MIXAHZIQUK2WDHER77VVJ4NZZA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.487748790 (21h49m26.002851367s) [noise=true , address=193.149.164.143:4693 , piece_id=RCLLY6ABOIXYK6BBO5EWKTDDJ4MUH6IP2WGIHSTER4TEPSF2FYVA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LHoCE9CnM32tbMNqAB7HpmArSVLTycv9o6kSSFNUJsoDeERkQ): protocol: read tcp 159.223.143.39:37688->193.149.164.143:4693: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 07:28:51.487858041 (21h49m26.002960604s) [noise=true , address=45.91.116.11:4120 , piece_id=274GAU5MCZAZOHB3ESPDSJP23Z5FZB636CWGN7ELS2HYY5OFZ7RA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.487890658 (21h49m26.002993237s) [seg_pos={0 0}, upload=22550]: Done waiting for segment. successful: 16 optimal: 14 errs: <nil>
Aug 8 07:28:51.487971404 (21h49m26.00307399s) [upload=22550]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 8 07:28:51.487976917 (21h49m26.003079481s) [upload=22550]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 8 07:28:51.647177827 (21h49m26.162280406s) [upload=22550]: Done uploading segments...
Aug 8 07:28:51.647194442 (21h49m26.162297008s) [upload=22550]: Upload finished. err: <nil>
Aug 8 07:29:34.528501362 (21h50m9.043603939s) [upload=22562]: Starting upload
Aug 8 07:29:34.528548565 (21h50m9.043651143s) [upload=22562]: Done starting upload
Aug 8 07:29:34.528615374 (21h50m9.043717942s) [upload=22562]: Uploading segments...
Aug 8 07:29:34.528903298 (21h50m9.044005874s) [upload=22562]: Got next segment. Inline: false
Aug 8 07:29:34.528940157 (21h50m9.044042733s) [upload=22562]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 8 07:29:34.528944789 (21h50m9.044047366s) [upload=22562]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 8 07:29:34.536881082 (21h50m9.051983672s) [upload=22562]: Next segment err: metaclient: Too Many Requests
Aug 8 07:29:34.536925024 (21h50m9.052027602s) [upload=22562]: Done uploading segments...
Aug 8 07:29:34.536943164 (21h50m9.052045741s) [upload=22562]: Upload finished. err: metaclient: Too Many Requests
Duration: 398.881229ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......] duration:
31.798993ms target:
https://us1.storj.io/login
FAIL (2025-08-08 07:37:49 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.150.199.48:443: connect: connection refused
Duration: 31.897282ms
core
satellite
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [....X..............................................X........] duration:
465.618971ms
FAIL (2025-08-08 07:37:45 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== nts... Aug 8 07:24:18.405184267 (21h44m52.920286834s) [upload=22475]: Upload finished. err: <nil> Aug 8 07:24:44.148856783 (21h45m18.663959349s) [upload=22481]: Starting upload Aug 8 07:24:44.148908040 (21h45m18.664010617s) [upload=22481]: Done starting upload Aug 8 07:24:44.148978727 (21h45m18.664081294s) [upload=22481]: Uploading segments... Aug 8 07:24:44.149031432 (21h45m18.664134016s) [upload=22481]: Got next segment. Inline: true Aug 8 07:24:44.149047929 (21h45m18.664150495s) [upload=22481]: Waiting for error group managing segments... Aug 8 07:24:44.149231742 (21h45m18.664334308s) [upload=22481]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:24:44.149238484 (21h45m18.664341061s) [upload=22481]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:24:44.149241695 (21h45m18.664344271s) [upload=22481]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:24:44.236175458 (21h45m18.751278039s) [upload=22481]: Done uploading segments... Aug 8 07:24:44.236197562 (21h45m18.75130015s) [upload=22481]: Upload finished. err: <nil> Aug 8 07:25:45.798907959 (21h46m20.31401054s) [upload=22496]: Starting upload Aug 8 07:25:45.798982852 (21h46m20.314085432s) [upload=22496]: Done starting upload Aug 8 07:25:45.799141617 (21h46m20.314244194s) [upload=22496]: Uploading segments... Aug 8 07:25:45.799187034 (21h46m20.314289621s) [upload=22496]: Got next segment. Inline: true Aug 8 07:25:45.799192794 (21h46m20.314295378s) [upload=22496]: Waiting for error group managing segments... Aug 8 07:25:45.799260506 (21h46m20.314363086s) [upload=22496]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:25:45.799265543 (21h46m20.314368123s) [upload=22496]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:25:45.799269614 (21h46m20.314372195s) [upload=22496]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:25:45.856873466 (21h46m20.371976033s) [upload=22496]: Done uploading segments... Aug 8 07:25:45.856894517 (21h46m20.371997083s) [upload=22496]: Upload finished. err: <nil> Aug 8 07:26:45.160306388 (21h47m19.67540897s) [upload=22512]: Starting upload Aug 8 07:26:45.160369840 (21h47m19.675472407s) [upload=22512]: Done starting upload Aug 8 07:26:45.160448308 (21h47m19.675550874s) [upload=22512]: Uploading segments... Aug 8 07:26:45.160512759 (21h47m19.675615339s) [upload=22512]: Got next segment. Inline: true Aug 8 07:26:45.160518821 (21h47m19.675621389s) [upload=22512]: Waiting for error group managing segments... Aug 8 07:26:45.160566486 (21h47m19.675669054s) [upload=22512]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:26:45.160571363 (21h47m19.675673943s) [upload=22512]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:26:45.160575116 (21h47m19.675677698s) [upload=22512]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:26:45.219521344 (21h47m19.73462392s) [upload=22512]: Done uploading segments... Aug 8 07:26:45.219537755 (21h47m19.73464032s) [upload=22512]: Upload finished. err: <nil> Aug 8 07:27:50.008745364 (21h48m24.523847971s) [upload=22529]: Starting upload Aug 8 07:27:50.008828662 (21h48m24.523931252s) [upload=22529]: Done starting upload Aug 8 07:27:50.008913720 (21h48m24.524016289s) [upload=22529]: Uploading segments... Aug 8 07:27:50.008968901 (21h48m24.524071486s) [upload=22529]: Got next segment. Inline: true Aug 8 07:27:50.008973893 (21h48m24.524076469s) [upload=22529]: Waiting for error group managing segments... Aug 8 07:27:50.009023439 (21h48m24.524126015s) [upload=22529]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:27:50.009028404 (21h48m24.524130983s) [upload=22529]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:27:50.009032740 (21h48m24.524135324s) [upload=22529]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:27:50.072222330 (21h48m24.587324909s) [upload=22529]: Done uploading segments... Aug 8 07:27:50.072260086 (21h48m24.587362652s) [upload=22529]: Upload finished. err: <nil> Aug 8 07:28:08.797868182 (21h48m43.312970759s) [upload=22537]: Starting upload Aug 8 07:28:08.797909617 (21h48m43.313012202s) [upload=22537]: Done starting upload Aug 8 07:28:08.797978107 (21h48m43.313080676s) [upload=22537]: Uploading segments... Aug 8 07:28:08.798028549 (21h48m43.313131134s) [upload=22537]: Got next segment. Inline: true Aug 8 07:28:08.798033746 (21h48m43.313136324s) [upload=22537]: Waiting for error group managing segments... Aug 8 07:28:08.798088133 (21h48m43.313190709s) [upload=22537]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:28:08.798092673 (21h48m43.313195249s) [upload=22537]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:28:08.798096028 (21h48m43.313198604s) [upload=22537]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:28:08.856181519 (21h48m43.371284096s) [upload=22537]: Done uploading segments... Aug 8 07:28:08.856219288 (21h48m43.371321856s) [upload=22537]: Upload finished. err: <nil> Aug 8 07:28:26.426406302 (21h49m0.941508883s) [upload=22543]: Starting upload Aug 8 07:28:26.426484520 (21h49m0.94158709s) [upload=22543]: Done starting upload Aug 8 07:28:26.426561912 (21h49m0.941664479s) [upload=22543]: Uploading segments... Aug 8 07:28:26.426613562 (21h49m0.941716147s) [upload=22543]: Got next segment. Inline: true Aug 8 07:28:26.426618334 (21h49m0.94172091s) [upload=22543]: Waiting for error group managing segments... Aug 8 07:28:26.426667358 (21h49m0.941769934s) [upload=22543]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:28:26.426672477 (21h49m0.941775052s) [upload=22543]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:28:26.426676980 (21h49m0.941779557s) [upload=22543]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:28:26.739786156 (21h49m1.254888734s) [upload=22543]: Done uploading segments... Aug 8 07:28:26.739818088 (21h49m1.254920663s) [upload=22543]: Upload finished. err: <nil> Aug 8 07:28:44.768272418 (21h49m19.283374995s) [upload=22547]: Starting upload Aug 8 07:28:44.768352182 (21h49m19.28345476s) [upload=22547]: Done starting upload Aug 8 07:28:44.768463680 (21h49m19.283566246s) [upload=22547]: Uploading segments... Aug 8 07:28:44.768552166 (21h49m19.283654733s) [upload=22547]: Got next segment. Inline: true Aug 8 07:28:44.768558003 (21h49m19.28366057s) [upload=22547]: Waiting for error group managing segments... Aug 8 07:28:44.768617460 (21h49m19.28372004s) [upload=22547]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:28:44.768622292 (21h49m19.283724868s) [upload=22547]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:28:44.768625505 (21h49m19.283728082s) [upload=22547]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:28:44.832641596 (21h49m19.347744163s) [upload=22547]: Done uploading segments... Aug 8 07:28:44.832661990 (21h49m19.347764559s) [upload=22547]: Upload finished. err: <nil> Aug 8 07:29:19.907807846 (21h49m54.422910428s) [upload=22558]: Starting upload Aug 8 07:29:19.907919209 (21h49m54.423021801s) [upload=22558]: Done starting upload Aug 8 07:29:19.908035700 (21h49m54.423138281s) [upload=22558]: Uploading segments... Aug 8 07:29:19.908115743 (21h49m54.423218334s) [upload=22558]: Got next segment. Inline: true Aug 8 07:29:19.908123216 (21h49m54.423225802s) [upload=22558]: Waiting for error group managing segments... Aug 8 07:29:19.908206488 (21h49m54.423309073s) [upload=22558]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:29:19.908213079 (21h49m54.423315659s) [upload=22558]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:29:19.908216828 (21h49m54.423319408s) [upload=22558]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:29:19.968416399 (21h49m54.483518987s) [upload=22558]: Done uploading segments... Aug 8 07:29:19.968437892 (21h49m54.48354047s) [upload=22558]: Upload finished. err: <nil> Aug 8 07:30:04.570811298 (21h50m39.085913875s) [upload=22574]: Starting upload Aug 8 07:30:04.570856561 (21h50m39.085959129s) [upload=22574]: Done starting upload Aug 8 07:30:04.570937395 (21h50m39.086039963s) [upload=22574]: Uploading segments... Aug 8 07:30:04.570993290 (21h50m39.086095879s) [upload=22574]: Got next segment. Inline: true Aug 8 07:30:04.570999714 (21h50m39.086102306s) [upload=22574]: Waiting for error group managing segments... Aug 8 07:30:04.571094735 (21h50m39.086197316s) [upload=22574]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:30:04.571101887 (21h50m39.086204467s) [upload=22574]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:30:04.571105790 (21h50m39.086208372s) [upload=22574]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:30:04.649336948 (21h50m39.164439536s) [upload=22574]: Done uploading segments... Aug 8 07:30:04.649362004 (21h50m39.164464585s) [upload=22574]: Upload finished. err: <nil> Aug 8 07:30:54.851564431 (21h51m29.366666997s) [upload=22587]: Starting upload Aug 8 07:30:54.851618044 (21h51m29.36672062s) [upload=22587]: Done starting upload Aug 8 07:30:54.851712059 (21h51m29.366814635s) [upload=22587]: Uploading segments... Aug 8 07:30:54.851783178 (21h51m29.36688576s) [upload=22587]: Got next segment. Inline: true Aug 8 07:30:54.851789517 (21h51m29.366892093s) [upload=22587]: Waiting for error group managing segments... Aug 8 07:30:54.851854337 (21h51m29.366956924s) [upload=22587]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:30:54.851861334 (21h51m29.366963924s) [upload=22587]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:30:54.851866059 (21h51m29.366968642s) [upload=22587]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:30:54.860593007 (21h51m29.375695595s) [upload=22587]: Done uploading segments... Aug 8 07:30:54.860627519 (21h51m29.375730095s) [upload=22587]: Upload finished. err: metaclient: Too Many Requests
Duration: 465.646191ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........] duration:
750.394533ms
FAIL (2025-08-08 07:37:00 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
g_pos={0 0}, upload=22550]: Begin upload segment...
Aug 8 07:28:51.351770179 (21h49m25.866872766s) [seg_pos={0 0}, upload=22550]: Done begin upload segment.
Aug 8 07:28:51.351799155 (21h49m25.866901747s) [noise=true , address=104.36.82.23:4469 , piece_id=CI6CU5IBYK37MBIQW554EW6PPXVYA6ZF2CJPVZCIXMFFLXZ7UO7Q, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.351816053 (21h49m25.86691864s) [noise=true , address=193.149.164.143:4693 , piece_id=RCLLY6ABOIXYK6BBO5EWKTDDJ4MUH6IP2WGIHSTER4TEPSF2FYVA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.352227136 (21h49m25.867329713s) [noise=true , address=104.36.82.58:4286 , piece_id=36LFLTD2YWV3K7LM4M2GY3JIKLUDOZVRINQT3VLIFBRR55UCNA4Q, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.352322195 (21h49m25.867424772s) [noise=true , address=104.36.82.44:4880 , piece_id=HFXFL7ULX5OTYJFEOC45DI75DGDIHAQA6ZJIXHO22X4GPSEZN75Q, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.352611348 (21h49m25.867713924s) [noise=true , address=104.36.82.53:4139 , piece_id=IGOHVQJMZ3NWLCBKVPVHTVWVHDS43DABQPEMJUTB2TXAQPQOURSQ, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.352702579 (21h49m25.867805142s) [noise=true , address=45.91.116.7:4013 , piece_id=XG4M7HF2E5YIRJ7PVBEGZC2ADPUPAL2QL5TPUAFIYXZUP3BX2Y7A, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.353022021 (21h49m25.868124597s) [noise=true , address=104.36.82.16:4321 , piece_id=MTRDWQYPHECWBAH4ECUGJL2Y2LAUA2RWHX5J6XMR65NZOACBF6PA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.353464633 (21h49m25.868567211s) [noise=true , address=104.36.82.46:4487 , piece_id=JOSIX5WT5A36RXKJNWO5KLBEQOCNQCEZ7A2S4KYIBJHRKN3KT6RQ, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.354418304 (21h49m25.869520884s) [noise=true , address=104.36.82.38:4707 , piece_id=WZWQHYDMTUG2FOTZ6JJRVIXXFIPHJ3I2YIRQXMN7B2W7QGKAFY3A, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.354959848 (21h49m25.87006243s) [noise=true , address=104.36.82.47:4544 , piece_id=YDG3KOKUS5NVBWMQYJ7JJEO3PAXZKPZPUOUWUDCZ4RAVATW67NKQ, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.355461506 (21h49m25.870564086s) [noise=true , address=104.36.82.60:4363 , piece_id=QMR7CM4ASNAKG5PT2XUPFR4MIBHNFYJ7OZMASJ7Q6R2C225L655A, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.355910519 (21h49m25.871013101s) [noise=true , address=104.36.82.2:4741 , piece_id=3NWVL6DQCMDO5PQLSFU3VER7RSHGRAB6YRT32XBFUVLAXGEJQN7A, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.357467009 (21h49m25.872569585s) [noise=true , address=104.36.82.59:4322 , piece_id=NKCSGWGYWOIJWHSQUN2ZYOVZYIRML5G3DXVRJYDTAMHTEN2VES3Q, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.358628018 (21h49m25.873730598s) [noise=true , address=45.91.116.11:4120 , piece_id=274GAU5MCZAZOHB3ESPDSJP23Z5FZB636CWGN7ELS2HYY5OFZ7RA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.359105086 (21h49m25.874207666s) [noise=true , address=104.36.82.45:4907 , piece_id=ZF2SKF5O2WVNHM7CHQ7E4U7HVKQWCJGOZEVIDXO3OU4XHVUFPIGA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.360554866 (21h49m25.875657451s) [noise=true , address=104.36.82.36:4648 , piece_id=3WAWCS3XXWVX3ACYJG2OOS2R34SXOWYSHXVF6AMPFXJFCMYH4SZA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.363015621 (21h49m25.8781182s) [noise=true , address=104.36.82.62:4420 , piece_id=7S6ALI3DNOM7IURBTYY3DLALDQV7R7LLHPQCA4Q6G4HA2SOZ2YMA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.364089235 (21h49m25.87919182s) [noise=true , address=104.36.82.9:4008 , piece_id=EGIGCEJHELTHOJZWPFWNHCP672MIXAHZIQUK2WDHER77VVJ4NZZA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.467747752 (21h49m25.982850337s) [upload=22550]: Next returned nil segment
Aug 8 07:28:51.467767955 (21h49m25.982870543s) [upload=22550]: Waiting for error group managing segments...
Aug 8 07:28:51.473577577 (21h49m25.988680143s) [noise=true , address=45.91.116.7:4013 , piece_id=XG4M7HF2E5YIRJ7PVBEGZC2ADPUPAL2QL5TPUAFIYXZUP3BX2Y7A, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.478188638 (21h49m25.993291216s) [noise=true , address=104.36.82.16:4321 , piece_id=MTRDWQYPHECWBAH4ECUGJL2Y2LAUA2RWHX5J6XMR65NZOACBF6PA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.478342049 (21h49m25.993444627s) [noise=true , address=104.36.82.53:4139 , piece_id=IGOHVQJMZ3NWLCBKVPVHTVWVHDS43DABQPEMJUTB2TXAQPQOURSQ, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.478768394 (21h49m25.993870961s) [noise=true , address=104.36.82.36:4648 , piece_id=3WAWCS3XXWVX3ACYJG2OOS2R34SXOWYSHXVF6AMPFXJFCMYH4SZA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.479336172 (21h49m25.994438735s) [noise=true , address=104.36.82.59:4322 , piece_id=NKCSGWGYWOIJWHSQUN2ZYOVZYIRML5G3DXVRJYDTAMHTEN2VES3Q, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.479515883 (21h49m25.994618457s) [noise=true , address=104.36.82.58:4286 , piece_id=36LFLTD2YWV3K7LM4M2GY3JIKLUDOZVRINQT3VLIFBRR55UCNA4Q, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.480474985 (21h49m25.995577567s) [noise=true , address=104.36.82.2:4741 , piece_id=3NWVL6DQCMDO5PQLSFU3VER7RSHGRAB6YRT32XBFUVLAXGEJQN7A, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.481008628 (21h49m25.996111195s) [noise=true , address=104.36.82.38:4707 , piece_id=WZWQHYDMTUG2FOTZ6JJRVIXXFIPHJ3I2YIRQXMN7B2W7QGKAFY3A, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.481589774 (21h49m25.996692358s) [noise=true , address=104.36.82.60:4363 , piece_id=QMR7CM4ASNAKG5PT2XUPFR4MIBHNFYJ7OZMASJ7Q6R2C225L655A, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.482143054 (21h49m25.99724562s) [noise=true , address=104.36.82.46:4487 , piece_id=JOSIX5WT5A36RXKJNWO5KLBEQOCNQCEZ7A2S4KYIBJHRKN3KT6RQ, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.485172215 (21h49m26.000274779s) [noise=true , address=104.36.82.47:4544 , piece_id=YDG3KOKUS5NVBWMQYJ7JJEO3PAXZKPZPUOUWUDCZ4RAVATW67NKQ, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.486088591 (21h49m26.001191172s) [noise=true , address=104.36.82.44:4880 , piece_id=HFXFL7ULX5OTYJFEOC45DI75DGDIHAQA6ZJIXHO22X4GPSEZN75Q, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.486628345 (21h49m26.001730916s) [noise=true , address=104.36.82.23:4469 , piece_id=CI6CU5IBYK37MBIQW554EW6PPXVYA6ZF2CJPVZCIXMFFLXZ7UO7Q, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.487179958 (21h49m26.002282521s) [noise=true , address=104.36.82.62:4420 , piece_id=7S6ALI3DNOM7IURBTYY3DLALDQV7R7LLHPQCA4Q6G4HA2SOZ2YMA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.487192580 (21h49m26.002295147s) [seg_pos={0 0}, upload=22550]: Segment reached optimal threshold of 14 pieces.
Aug 8 07:28:51.487306760 (21h49m26.002409325s) [noise=true , address=104.36.82.45:4907 , piece_id=ZF2SKF5O2WVNHM7CHQ7E4U7HVKQWCJGOZEVIDXO3OU4XHVUFPIGA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12URx18Ckh7VaAKJ1hdQQucMRu5qiDrAraqaycjYqxETgwZ4uWS): protocol: expected piece hash; context canceled; EOF
Aug 8 07:28:51.487325587 (21h49m26.00242815s) [noise=true , address=104.36.82.9:4008 , piece_id=EGIGCEJHELTHOJZWPFWNHCP672MIXAHZIQUK2WDHER77VVJ4NZZA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.487748790 (21h49m26.002851367s) [noise=true , address=193.149.164.143:4693 , piece_id=RCLLY6ABOIXYK6BBO5EWKTDDJ4MUH6IP2WGIHSTER4TEPSF2FYVA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LHoCE9CnM32tbMNqAB7HpmArSVLTycv9o6kSSFNUJsoDeERkQ): protocol: read tcp 159.223.143.39:37688->193.149.164.143:4693: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 07:28:51.487858041 (21h49m26.002960604s) [noise=true , address=45.91.116.11:4120 , piece_id=274GAU5MCZAZOHB3ESPDSJP23Z5FZB636CWGN7ELS2HYY5OFZ7RA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.487890658 (21h49m26.002993237s) [seg_pos={0 0}, upload=22550]: Done waiting for segment. successful: 16 optimal: 14 errs: <nil>
Aug 8 07:28:51.487971404 (21h49m26.00307399s) [upload=22550]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 8 07:28:51.487976917 (21h49m26.003079481s) [upload=22550]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 8 07:28:51.647177827 (21h49m26.162280406s) [upload=22550]: Done uploading segments...
Aug 8 07:28:51.647194442 (21h49m26.162297008s) [upload=22550]: Upload finished. err: <nil>
Aug 8 07:29:34.528501362 (21h50m9.043603939s) [upload=22562]: Starting upload
Aug 8 07:29:34.528548565 (21h50m9.043651143s) [upload=22562]: Done starting upload
Aug 8 07:29:34.528615374 (21h50m9.043717942s) [upload=22562]: Uploading segments...
Aug 8 07:29:34.528903298 (21h50m9.044005874s) [upload=22562]: Got next segment. Inline: false
Aug 8 07:29:34.528940157 (21h50m9.044042733s) [upload=22562]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 8 07:29:34.528944789 (21h50m9.044047366s) [upload=22562]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 8 07:29:34.536881082 (21h50m9.051983672s) [upload=22562]: Next segment err: metaclient: Too Many Requests
Aug 8 07:29:34.536925024 (21h50m9.052027602s) [upload=22562]: Done uploading segments...
Aug 8 07:29:34.536943164 (21h50m9.052045741s) [upload=22562]: Upload finished. err: metaclient: Too Many Requests
Duration: 750.424416ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [.....X..............................................X.......] duration:
177.575446ms
FAIL (2025-08-08 07:36:51 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== nts... Aug 8 07:24:18.405184267 (21h44m52.920286834s) [upload=22475]: Upload finished. err: <nil> Aug 8 07:24:44.148856783 (21h45m18.663959349s) [upload=22481]: Starting upload Aug 8 07:24:44.148908040 (21h45m18.664010617s) [upload=22481]: Done starting upload Aug 8 07:24:44.148978727 (21h45m18.664081294s) [upload=22481]: Uploading segments... Aug 8 07:24:44.149031432 (21h45m18.664134016s) [upload=22481]: Got next segment. Inline: true Aug 8 07:24:44.149047929 (21h45m18.664150495s) [upload=22481]: Waiting for error group managing segments... Aug 8 07:24:44.149231742 (21h45m18.664334308s) [upload=22481]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:24:44.149238484 (21h45m18.664341061s) [upload=22481]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:24:44.149241695 (21h45m18.664344271s) [upload=22481]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:24:44.236175458 (21h45m18.751278039s) [upload=22481]: Done uploading segments... Aug 8 07:24:44.236197562 (21h45m18.75130015s) [upload=22481]: Upload finished. err: <nil> Aug 8 07:25:45.798907959 (21h46m20.31401054s) [upload=22496]: Starting upload Aug 8 07:25:45.798982852 (21h46m20.314085432s) [upload=22496]: Done starting upload Aug 8 07:25:45.799141617 (21h46m20.314244194s) [upload=22496]: Uploading segments... Aug 8 07:25:45.799187034 (21h46m20.314289621s) [upload=22496]: Got next segment. Inline: true Aug 8 07:25:45.799192794 (21h46m20.314295378s) [upload=22496]: Waiting for error group managing segments... Aug 8 07:25:45.799260506 (21h46m20.314363086s) [upload=22496]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:25:45.799265543 (21h46m20.314368123s) [upload=22496]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:25:45.799269614 (21h46m20.314372195s) [upload=22496]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:25:45.856873466 (21h46m20.371976033s) [upload=22496]: Done uploading segments... Aug 8 07:25:45.856894517 (21h46m20.371997083s) [upload=22496]: Upload finished. err: <nil> Aug 8 07:26:45.160306388 (21h47m19.67540897s) [upload=22512]: Starting upload Aug 8 07:26:45.160369840 (21h47m19.675472407s) [upload=22512]: Done starting upload Aug 8 07:26:45.160448308 (21h47m19.675550874s) [upload=22512]: Uploading segments... Aug 8 07:26:45.160512759 (21h47m19.675615339s) [upload=22512]: Got next segment. Inline: true Aug 8 07:26:45.160518821 (21h47m19.675621389s) [upload=22512]: Waiting for error group managing segments... Aug 8 07:26:45.160566486 (21h47m19.675669054s) [upload=22512]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:26:45.160571363 (21h47m19.675673943s) [upload=22512]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:26:45.160575116 (21h47m19.675677698s) [upload=22512]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:26:45.219521344 (21h47m19.73462392s) [upload=22512]: Done uploading segments... Aug 8 07:26:45.219537755 (21h47m19.73464032s) [upload=22512]: Upload finished. err: <nil> Aug 8 07:27:50.008745364 (21h48m24.523847971s) [upload=22529]: Starting upload Aug 8 07:27:50.008828662 (21h48m24.523931252s) [upload=22529]: Done starting upload Aug 8 07:27:50.008913720 (21h48m24.524016289s) [upload=22529]: Uploading segments... Aug 8 07:27:50.008968901 (21h48m24.524071486s) [upload=22529]: Got next segment. Inline: true Aug 8 07:27:50.008973893 (21h48m24.524076469s) [upload=22529]: Waiting for error group managing segments... Aug 8 07:27:50.009023439 (21h48m24.524126015s) [upload=22529]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:27:50.009028404 (21h48m24.524130983s) [upload=22529]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:27:50.009032740 (21h48m24.524135324s) [upload=22529]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:27:50.072222330 (21h48m24.587324909s) [upload=22529]: Done uploading segments... Aug 8 07:27:50.072260086 (21h48m24.587362652s) [upload=22529]: Upload finished. err: <nil> Aug 8 07:28:08.797868182 (21h48m43.312970759s) [upload=22537]: Starting upload Aug 8 07:28:08.797909617 (21h48m43.313012202s) [upload=22537]: Done starting upload Aug 8 07:28:08.797978107 (21h48m43.313080676s) [upload=22537]: Uploading segments... Aug 8 07:28:08.798028549 (21h48m43.313131134s) [upload=22537]: Got next segment. Inline: true Aug 8 07:28:08.798033746 (21h48m43.313136324s) [upload=22537]: Waiting for error group managing segments... Aug 8 07:28:08.798088133 (21h48m43.313190709s) [upload=22537]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:28:08.798092673 (21h48m43.313195249s) [upload=22537]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:28:08.798096028 (21h48m43.313198604s) [upload=22537]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:28:08.856181519 (21h48m43.371284096s) [upload=22537]: Done uploading segments... Aug 8 07:28:08.856219288 (21h48m43.371321856s) [upload=22537]: Upload finished. err: <nil> Aug 8 07:28:26.426406302 (21h49m0.941508883s) [upload=22543]: Starting upload Aug 8 07:28:26.426484520 (21h49m0.94158709s) [upload=22543]: Done starting upload Aug 8 07:28:26.426561912 (21h49m0.941664479s) [upload=22543]: Uploading segments... Aug 8 07:28:26.426613562 (21h49m0.941716147s) [upload=22543]: Got next segment. Inline: true Aug 8 07:28:26.426618334 (21h49m0.94172091s) [upload=22543]: Waiting for error group managing segments... Aug 8 07:28:26.426667358 (21h49m0.941769934s) [upload=22543]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:28:26.426672477 (21h49m0.941775052s) [upload=22543]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:28:26.426676980 (21h49m0.941779557s) [upload=22543]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:28:26.739786156 (21h49m1.254888734s) [upload=22543]: Done uploading segments... Aug 8 07:28:26.739818088 (21h49m1.254920663s) [upload=22543]: Upload finished. err: <nil> Aug 8 07:28:44.768272418 (21h49m19.283374995s) [upload=22547]: Starting upload Aug 8 07:28:44.768352182 (21h49m19.28345476s) [upload=22547]: Done starting upload Aug 8 07:28:44.768463680 (21h49m19.283566246s) [upload=22547]: Uploading segments... Aug 8 07:28:44.768552166 (21h49m19.283654733s) [upload=22547]: Got next segment. Inline: true Aug 8 07:28:44.768558003 (21h49m19.28366057s) [upload=22547]: Waiting for error group managing segments... Aug 8 07:28:44.768617460 (21h49m19.28372004s) [upload=22547]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:28:44.768622292 (21h49m19.283724868s) [upload=22547]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:28:44.768625505 (21h49m19.283728082s) [upload=22547]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:28:44.832641596 (21h49m19.347744163s) [upload=22547]: Done uploading segments... Aug 8 07:28:44.832661990 (21h49m19.347764559s) [upload=22547]: Upload finished. err: <nil> Aug 8 07:29:19.907807846 (21h49m54.422910428s) [upload=22558]: Starting upload Aug 8 07:29:19.907919209 (21h49m54.423021801s) [upload=22558]: Done starting upload Aug 8 07:29:19.908035700 (21h49m54.423138281s) [upload=22558]: Uploading segments... Aug 8 07:29:19.908115743 (21h49m54.423218334s) [upload=22558]: Got next segment. Inline: true Aug 8 07:29:19.908123216 (21h49m54.423225802s) [upload=22558]: Waiting for error group managing segments... Aug 8 07:29:19.908206488 (21h49m54.423309073s) [upload=22558]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:29:19.908213079 (21h49m54.423315659s) [upload=22558]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:29:19.908216828 (21h49m54.423319408s) [upload=22558]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:29:19.968416399 (21h49m54.483518987s) [upload=22558]: Done uploading segments... Aug 8 07:29:19.968437892 (21h49m54.48354047s) [upload=22558]: Upload finished. err: <nil> Aug 8 07:30:04.570811298 (21h50m39.085913875s) [upload=22574]: Starting upload Aug 8 07:30:04.570856561 (21h50m39.085959129s) [upload=22574]: Done starting upload Aug 8 07:30:04.570937395 (21h50m39.086039963s) [upload=22574]: Uploading segments... Aug 8 07:30:04.570993290 (21h50m39.086095879s) [upload=22574]: Got next segment. Inline: true Aug 8 07:30:04.570999714 (21h50m39.086102306s) [upload=22574]: Waiting for error group managing segments... Aug 8 07:30:04.571094735 (21h50m39.086197316s) [upload=22574]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:30:04.571101887 (21h50m39.086204467s) [upload=22574]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:30:04.571105790 (21h50m39.086208372s) [upload=22574]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:30:04.649336948 (21h50m39.164439536s) [upload=22574]: Done uploading segments... Aug 8 07:30:04.649362004 (21h50m39.164464585s) [upload=22574]: Upload finished. err: <nil> Aug 8 07:30:54.851564431 (21h51m29.366666997s) [upload=22587]: Starting upload Aug 8 07:30:54.851618044 (21h51m29.36672062s) [upload=22587]: Done starting upload Aug 8 07:30:54.851712059 (21h51m29.366814635s) [upload=22587]: Uploading segments... Aug 8 07:30:54.851783178 (21h51m29.36688576s) [upload=22587]: Got next segment. Inline: true Aug 8 07:30:54.851789517 (21h51m29.366892093s) [upload=22587]: Waiting for error group managing segments... Aug 8 07:30:54.851854337 (21h51m29.366956924s) [upload=22587]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:30:54.851861334 (21h51m29.366963924s) [upload=22587]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:30:54.851866059 (21h51m29.366968642s) [upload=22587]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:30:54.860593007 (21h51m29.375695595s) [upload=22587]: Done uploading segments... Aug 8 07:30:54.860627519 (21h51m29.375730095s) [upload=22587]: Upload finished. err: metaclient: Too Many Requests
Duration: 177.608569ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....] duration:
29.773463ms target:
https://us1.storj.io/login
FAIL (2025-08-08 07:36:38 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.150.199.48:443: connect: connection refused
Duration: 29.803956ms
core
satellite
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
390.115621ms
FAIL (2025-08-08 07:36:13 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
g_pos={0 0}, upload=22550]: Begin upload segment...
Aug 8 07:28:51.351770179 (21h49m25.866872766s) [seg_pos={0 0}, upload=22550]: Done begin upload segment.
Aug 8 07:28:51.351799155 (21h49m25.866901747s) [noise=true , address=104.36.82.23:4469 , piece_id=CI6CU5IBYK37MBIQW554EW6PPXVYA6ZF2CJPVZCIXMFFLXZ7UO7Q, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.351816053 (21h49m25.86691864s) [noise=true , address=193.149.164.143:4693 , piece_id=RCLLY6ABOIXYK6BBO5EWKTDDJ4MUH6IP2WGIHSTER4TEPSF2FYVA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.352227136 (21h49m25.867329713s) [noise=true , address=104.36.82.58:4286 , piece_id=36LFLTD2YWV3K7LM4M2GY3JIKLUDOZVRINQT3VLIFBRR55UCNA4Q, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.352322195 (21h49m25.867424772s) [noise=true , address=104.36.82.44:4880 , piece_id=HFXFL7ULX5OTYJFEOC45DI75DGDIHAQA6ZJIXHO22X4GPSEZN75Q, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.352611348 (21h49m25.867713924s) [noise=true , address=104.36.82.53:4139 , piece_id=IGOHVQJMZ3NWLCBKVPVHTVWVHDS43DABQPEMJUTB2TXAQPQOURSQ, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.352702579 (21h49m25.867805142s) [noise=true , address=45.91.116.7:4013 , piece_id=XG4M7HF2E5YIRJ7PVBEGZC2ADPUPAL2QL5TPUAFIYXZUP3BX2Y7A, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.353022021 (21h49m25.868124597s) [noise=true , address=104.36.82.16:4321 , piece_id=MTRDWQYPHECWBAH4ECUGJL2Y2LAUA2RWHX5J6XMR65NZOACBF6PA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.353464633 (21h49m25.868567211s) [noise=true , address=104.36.82.46:4487 , piece_id=JOSIX5WT5A36RXKJNWO5KLBEQOCNQCEZ7A2S4KYIBJHRKN3KT6RQ, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.354418304 (21h49m25.869520884s) [noise=true , address=104.36.82.38:4707 , piece_id=WZWQHYDMTUG2FOTZ6JJRVIXXFIPHJ3I2YIRQXMN7B2W7QGKAFY3A, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.354959848 (21h49m25.87006243s) [noise=true , address=104.36.82.47:4544 , piece_id=YDG3KOKUS5NVBWMQYJ7JJEO3PAXZKPZPUOUWUDCZ4RAVATW67NKQ, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.355461506 (21h49m25.870564086s) [noise=true , address=104.36.82.60:4363 , piece_id=QMR7CM4ASNAKG5PT2XUPFR4MIBHNFYJ7OZMASJ7Q6R2C225L655A, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.355910519 (21h49m25.871013101s) [noise=true , address=104.36.82.2:4741 , piece_id=3NWVL6DQCMDO5PQLSFU3VER7RSHGRAB6YRT32XBFUVLAXGEJQN7A, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.357467009 (21h49m25.872569585s) [noise=true , address=104.36.82.59:4322 , piece_id=NKCSGWGYWOIJWHSQUN2ZYOVZYIRML5G3DXVRJYDTAMHTEN2VES3Q, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.358628018 (21h49m25.873730598s) [noise=true , address=45.91.116.11:4120 , piece_id=274GAU5MCZAZOHB3ESPDSJP23Z5FZB636CWGN7ELS2HYY5OFZ7RA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.359105086 (21h49m25.874207666s) [noise=true , address=104.36.82.45:4907 , piece_id=ZF2SKF5O2WVNHM7CHQ7E4U7HVKQWCJGOZEVIDXO3OU4XHVUFPIGA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.360554866 (21h49m25.875657451s) [noise=true , address=104.36.82.36:4648 , piece_id=3WAWCS3XXWVX3ACYJG2OOS2R34SXOWYSHXVF6AMPFXJFCMYH4SZA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.363015621 (21h49m25.8781182s) [noise=true , address=104.36.82.62:4420 , piece_id=7S6ALI3DNOM7IURBTYY3DLALDQV7R7LLHPQCA4Q6G4HA2SOZ2YMA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.364089235 (21h49m25.87919182s) [noise=true , address=104.36.82.9:4008 , piece_id=EGIGCEJHELTHOJZWPFWNHCP672MIXAHZIQUK2WDHER77VVJ4NZZA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.467747752 (21h49m25.982850337s) [upload=22550]: Next returned nil segment
Aug 8 07:28:51.467767955 (21h49m25.982870543s) [upload=22550]: Waiting for error group managing segments...
Aug 8 07:28:51.473577577 (21h49m25.988680143s) [noise=true , address=45.91.116.7:4013 , piece_id=XG4M7HF2E5YIRJ7PVBEGZC2ADPUPAL2QL5TPUAFIYXZUP3BX2Y7A, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.478188638 (21h49m25.993291216s) [noise=true , address=104.36.82.16:4321 , piece_id=MTRDWQYPHECWBAH4ECUGJL2Y2LAUA2RWHX5J6XMR65NZOACBF6PA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.478342049 (21h49m25.993444627s) [noise=true , address=104.36.82.53:4139 , piece_id=IGOHVQJMZ3NWLCBKVPVHTVWVHDS43DABQPEMJUTB2TXAQPQOURSQ, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.478768394 (21h49m25.993870961s) [noise=true , address=104.36.82.36:4648 , piece_id=3WAWCS3XXWVX3ACYJG2OOS2R34SXOWYSHXVF6AMPFXJFCMYH4SZA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.479336172 (21h49m25.994438735s) [noise=true , address=104.36.82.59:4322 , piece_id=NKCSGWGYWOIJWHSQUN2ZYOVZYIRML5G3DXVRJYDTAMHTEN2VES3Q, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.479515883 (21h49m25.994618457s) [noise=true , address=104.36.82.58:4286 , piece_id=36LFLTD2YWV3K7LM4M2GY3JIKLUDOZVRINQT3VLIFBRR55UCNA4Q, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.480474985 (21h49m25.995577567s) [noise=true , address=104.36.82.2:4741 , piece_id=3NWVL6DQCMDO5PQLSFU3VER7RSHGRAB6YRT32XBFUVLAXGEJQN7A, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.481008628 (21h49m25.996111195s) [noise=true , address=104.36.82.38:4707 , piece_id=WZWQHYDMTUG2FOTZ6JJRVIXXFIPHJ3I2YIRQXMN7B2W7QGKAFY3A, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.481589774 (21h49m25.996692358s) [noise=true , address=104.36.82.60:4363 , piece_id=QMR7CM4ASNAKG5PT2XUPFR4MIBHNFYJ7OZMASJ7Q6R2C225L655A, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.482143054 (21h49m25.99724562s) [noise=true , address=104.36.82.46:4487 , piece_id=JOSIX5WT5A36RXKJNWO5KLBEQOCNQCEZ7A2S4KYIBJHRKN3KT6RQ, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.485172215 (21h49m26.000274779s) [noise=true , address=104.36.82.47:4544 , piece_id=YDG3KOKUS5NVBWMQYJ7JJEO3PAXZKPZPUOUWUDCZ4RAVATW67NKQ, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.486088591 (21h49m26.001191172s) [noise=true , address=104.36.82.44:4880 , piece_id=HFXFL7ULX5OTYJFEOC45DI75DGDIHAQA6ZJIXHO22X4GPSEZN75Q, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.486628345 (21h49m26.001730916s) [noise=true , address=104.36.82.23:4469 , piece_id=CI6CU5IBYK37MBIQW554EW6PPXVYA6ZF2CJPVZCIXMFFLXZ7UO7Q, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.487179958 (21h49m26.002282521s) [noise=true , address=104.36.82.62:4420 , piece_id=7S6ALI3DNOM7IURBTYY3DLALDQV7R7LLHPQCA4Q6G4HA2SOZ2YMA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.487192580 (21h49m26.002295147s) [seg_pos={0 0}, upload=22550]: Segment reached optimal threshold of 14 pieces.
Aug 8 07:28:51.487306760 (21h49m26.002409325s) [noise=true , address=104.36.82.45:4907 , piece_id=ZF2SKF5O2WVNHM7CHQ7E4U7HVKQWCJGOZEVIDXO3OU4XHVUFPIGA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12URx18Ckh7VaAKJ1hdQQucMRu5qiDrAraqaycjYqxETgwZ4uWS): protocol: expected piece hash; context canceled; EOF
Aug 8 07:28:51.487325587 (21h49m26.00242815s) [noise=true , address=104.36.82.9:4008 , piece_id=EGIGCEJHELTHOJZWPFWNHCP672MIXAHZIQUK2WDHER77VVJ4NZZA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.487748790 (21h49m26.002851367s) [noise=true , address=193.149.164.143:4693 , piece_id=RCLLY6ABOIXYK6BBO5EWKTDDJ4MUH6IP2WGIHSTER4TEPSF2FYVA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LHoCE9CnM32tbMNqAB7HpmArSVLTycv9o6kSSFNUJsoDeERkQ): protocol: read tcp 159.223.143.39:37688->193.149.164.143:4693: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 07:28:51.487858041 (21h49m26.002960604s) [noise=true , address=45.91.116.11:4120 , piece_id=274GAU5MCZAZOHB3ESPDSJP23Z5FZB636CWGN7ELS2HYY5OFZ7RA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.487890658 (21h49m26.002993237s) [seg_pos={0 0}, upload=22550]: Done waiting for segment. successful: 16 optimal: 14 errs: <nil>
Aug 8 07:28:51.487971404 (21h49m26.00307399s) [upload=22550]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 8 07:28:51.487976917 (21h49m26.003079481s) [upload=22550]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 8 07:28:51.647177827 (21h49m26.162280406s) [upload=22550]: Done uploading segments...
Aug 8 07:28:51.647194442 (21h49m26.162297008s) [upload=22550]: Upload finished. err: <nil>
Aug 8 07:29:34.528501362 (21h50m9.043603939s) [upload=22562]: Starting upload
Aug 8 07:29:34.528548565 (21h50m9.043651143s) [upload=22562]: Done starting upload
Aug 8 07:29:34.528615374 (21h50m9.043717942s) [upload=22562]: Uploading segments...
Aug 8 07:29:34.528903298 (21h50m9.044005874s) [upload=22562]: Got next segment. Inline: false
Aug 8 07:29:34.528940157 (21h50m9.044042733s) [upload=22562]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 8 07:29:34.528944789 (21h50m9.044047366s) [upload=22562]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 8 07:29:34.536881082 (21h50m9.051983672s) [upload=22562]: Next segment err: metaclient: Too Many Requests
Aug 8 07:29:34.536925024 (21h50m9.052027602s) [upload=22562]: Done uploading segments...
Aug 8 07:29:34.536943164 (21h50m9.052045741s) [upload=22562]: Upload finished. err: metaclient: Too Many Requests
Duration: 390.157931ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....] duration:
54.931258ms target:
https://us1.storj.io/login
FAIL (2025-08-08 07:36:09 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.150.199.48:443: connect: connection refused
Duration: 54.959929ms
core
satellite
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........] duration:
605.619966ms
FAIL (2025-08-08 07:35:55 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
g_pos={0 0}, upload=22550]: Begin upload segment...
Aug 8 07:28:51.351770179 (21h49m25.866872766s) [seg_pos={0 0}, upload=22550]: Done begin upload segment.
Aug 8 07:28:51.351799155 (21h49m25.866901747s) [noise=true , address=104.36.82.23:4469 , piece_id=CI6CU5IBYK37MBIQW554EW6PPXVYA6ZF2CJPVZCIXMFFLXZ7UO7Q, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.351816053 (21h49m25.86691864s) [noise=true , address=193.149.164.143:4693 , piece_id=RCLLY6ABOIXYK6BBO5EWKTDDJ4MUH6IP2WGIHSTER4TEPSF2FYVA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.352227136 (21h49m25.867329713s) [noise=true , address=104.36.82.58:4286 , piece_id=36LFLTD2YWV3K7LM4M2GY3JIKLUDOZVRINQT3VLIFBRR55UCNA4Q, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.352322195 (21h49m25.867424772s) [noise=true , address=104.36.82.44:4880 , piece_id=HFXFL7ULX5OTYJFEOC45DI75DGDIHAQA6ZJIXHO22X4GPSEZN75Q, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.352611348 (21h49m25.867713924s) [noise=true , address=104.36.82.53:4139 , piece_id=IGOHVQJMZ3NWLCBKVPVHTVWVHDS43DABQPEMJUTB2TXAQPQOURSQ, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.352702579 (21h49m25.867805142s) [noise=true , address=45.91.116.7:4013 , piece_id=XG4M7HF2E5YIRJ7PVBEGZC2ADPUPAL2QL5TPUAFIYXZUP3BX2Y7A, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.353022021 (21h49m25.868124597s) [noise=true , address=104.36.82.16:4321 , piece_id=MTRDWQYPHECWBAH4ECUGJL2Y2LAUA2RWHX5J6XMR65NZOACBF6PA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.353464633 (21h49m25.868567211s) [noise=true , address=104.36.82.46:4487 , piece_id=JOSIX5WT5A36RXKJNWO5KLBEQOCNQCEZ7A2S4KYIBJHRKN3KT6RQ, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.354418304 (21h49m25.869520884s) [noise=true , address=104.36.82.38:4707 , piece_id=WZWQHYDMTUG2FOTZ6JJRVIXXFIPHJ3I2YIRQXMN7B2W7QGKAFY3A, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.354959848 (21h49m25.87006243s) [noise=true , address=104.36.82.47:4544 , piece_id=YDG3KOKUS5NVBWMQYJ7JJEO3PAXZKPZPUOUWUDCZ4RAVATW67NKQ, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.355461506 (21h49m25.870564086s) [noise=true , address=104.36.82.60:4363 , piece_id=QMR7CM4ASNAKG5PT2XUPFR4MIBHNFYJ7OZMASJ7Q6R2C225L655A, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.355910519 (21h49m25.871013101s) [noise=true , address=104.36.82.2:4741 , piece_id=3NWVL6DQCMDO5PQLSFU3VER7RSHGRAB6YRT32XBFUVLAXGEJQN7A, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.357467009 (21h49m25.872569585s) [noise=true , address=104.36.82.59:4322 , piece_id=NKCSGWGYWOIJWHSQUN2ZYOVZYIRML5G3DXVRJYDTAMHTEN2VES3Q, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.358628018 (21h49m25.873730598s) [noise=true , address=45.91.116.11:4120 , piece_id=274GAU5MCZAZOHB3ESPDSJP23Z5FZB636CWGN7ELS2HYY5OFZ7RA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.359105086 (21h49m25.874207666s) [noise=true , address=104.36.82.45:4907 , piece_id=ZF2SKF5O2WVNHM7CHQ7E4U7HVKQWCJGOZEVIDXO3OU4XHVUFPIGA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.360554866 (21h49m25.875657451s) [noise=true , address=104.36.82.36:4648 , piece_id=3WAWCS3XXWVX3ACYJG2OOS2R34SXOWYSHXVF6AMPFXJFCMYH4SZA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.363015621 (21h49m25.8781182s) [noise=true , address=104.36.82.62:4420 , piece_id=7S6ALI3DNOM7IURBTYY3DLALDQV7R7LLHPQCA4Q6G4HA2SOZ2YMA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.364089235 (21h49m25.87919182s) [noise=true , address=104.36.82.9:4008 , piece_id=EGIGCEJHELTHOJZWPFWNHCP672MIXAHZIQUK2WDHER77VVJ4NZZA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.467747752 (21h49m25.982850337s) [upload=22550]: Next returned nil segment
Aug 8 07:28:51.467767955 (21h49m25.982870543s) [upload=22550]: Waiting for error group managing segments...
Aug 8 07:28:51.473577577 (21h49m25.988680143s) [noise=true , address=45.91.116.7:4013 , piece_id=XG4M7HF2E5YIRJ7PVBEGZC2ADPUPAL2QL5TPUAFIYXZUP3BX2Y7A, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.478188638 (21h49m25.993291216s) [noise=true , address=104.36.82.16:4321 , piece_id=MTRDWQYPHECWBAH4ECUGJL2Y2LAUA2RWHX5J6XMR65NZOACBF6PA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.478342049 (21h49m25.993444627s) [noise=true , address=104.36.82.53:4139 , piece_id=IGOHVQJMZ3NWLCBKVPVHTVWVHDS43DABQPEMJUTB2TXAQPQOURSQ, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.478768394 (21h49m25.993870961s) [noise=true , address=104.36.82.36:4648 , piece_id=3WAWCS3XXWVX3ACYJG2OOS2R34SXOWYSHXVF6AMPFXJFCMYH4SZA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.479336172 (21h49m25.994438735s) [noise=true , address=104.36.82.59:4322 , piece_id=NKCSGWGYWOIJWHSQUN2ZYOVZYIRML5G3DXVRJYDTAMHTEN2VES3Q, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.479515883 (21h49m25.994618457s) [noise=true , address=104.36.82.58:4286 , piece_id=36LFLTD2YWV3K7LM4M2GY3JIKLUDOZVRINQT3VLIFBRR55UCNA4Q, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.480474985 (21h49m25.995577567s) [noise=true , address=104.36.82.2:4741 , piece_id=3NWVL6DQCMDO5PQLSFU3VER7RSHGRAB6YRT32XBFUVLAXGEJQN7A, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.481008628 (21h49m25.996111195s) [noise=true , address=104.36.82.38:4707 , piece_id=WZWQHYDMTUG2FOTZ6JJRVIXXFIPHJ3I2YIRQXMN7B2W7QGKAFY3A, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.481589774 (21h49m25.996692358s) [noise=true , address=104.36.82.60:4363 , piece_id=QMR7CM4ASNAKG5PT2XUPFR4MIBHNFYJ7OZMASJ7Q6R2C225L655A, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.482143054 (21h49m25.99724562s) [noise=true , address=104.36.82.46:4487 , piece_id=JOSIX5WT5A36RXKJNWO5KLBEQOCNQCEZ7A2S4KYIBJHRKN3KT6RQ, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.485172215 (21h49m26.000274779s) [noise=true , address=104.36.82.47:4544 , piece_id=YDG3KOKUS5NVBWMQYJ7JJEO3PAXZKPZPUOUWUDCZ4RAVATW67NKQ, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.486088591 (21h49m26.001191172s) [noise=true , address=104.36.82.44:4880 , piece_id=HFXFL7ULX5OTYJFEOC45DI75DGDIHAQA6ZJIXHO22X4GPSEZN75Q, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.486628345 (21h49m26.001730916s) [noise=true , address=104.36.82.23:4469 , piece_id=CI6CU5IBYK37MBIQW554EW6PPXVYA6ZF2CJPVZCIXMFFLXZ7UO7Q, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.487179958 (21h49m26.002282521s) [noise=true , address=104.36.82.62:4420 , piece_id=7S6ALI3DNOM7IURBTYY3DLALDQV7R7LLHPQCA4Q6G4HA2SOZ2YMA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.487192580 (21h49m26.002295147s) [seg_pos={0 0}, upload=22550]: Segment reached optimal threshold of 14 pieces.
Aug 8 07:28:51.487306760 (21h49m26.002409325s) [noise=true , address=104.36.82.45:4907 , piece_id=ZF2SKF5O2WVNHM7CHQ7E4U7HVKQWCJGOZEVIDXO3OU4XHVUFPIGA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12URx18Ckh7VaAKJ1hdQQucMRu5qiDrAraqaycjYqxETgwZ4uWS): protocol: expected piece hash; context canceled; EOF
Aug 8 07:28:51.487325587 (21h49m26.00242815s) [noise=true , address=104.36.82.9:4008 , piece_id=EGIGCEJHELTHOJZWPFWNHCP672MIXAHZIQUK2WDHER77VVJ4NZZA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.487748790 (21h49m26.002851367s) [noise=true , address=193.149.164.143:4693 , piece_id=RCLLY6ABOIXYK6BBO5EWKTDDJ4MUH6IP2WGIHSTER4TEPSF2FYVA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LHoCE9CnM32tbMNqAB7HpmArSVLTycv9o6kSSFNUJsoDeERkQ): protocol: read tcp 159.223.143.39:37688->193.149.164.143:4693: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 07:28:51.487858041 (21h49m26.002960604s) [noise=true , address=45.91.116.11:4120 , piece_id=274GAU5MCZAZOHB3ESPDSJP23Z5FZB636CWGN7ELS2HYY5OFZ7RA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.487890658 (21h49m26.002993237s) [seg_pos={0 0}, upload=22550]: Done waiting for segment. successful: 16 optimal: 14 errs: <nil>
Aug 8 07:28:51.487971404 (21h49m26.00307399s) [upload=22550]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 8 07:28:51.487976917 (21h49m26.003079481s) [upload=22550]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 8 07:28:51.647177827 (21h49m26.162280406s) [upload=22550]: Done uploading segments...
Aug 8 07:28:51.647194442 (21h49m26.162297008s) [upload=22550]: Upload finished. err: <nil>
Aug 8 07:29:34.528501362 (21h50m9.043603939s) [upload=22562]: Starting upload
Aug 8 07:29:34.528548565 (21h50m9.043651143s) [upload=22562]: Done starting upload
Aug 8 07:29:34.528615374 (21h50m9.043717942s) [upload=22562]: Uploading segments...
Aug 8 07:29:34.528903298 (21h50m9.044005874s) [upload=22562]: Got next segment. Inline: false
Aug 8 07:29:34.528940157 (21h50m9.044042733s) [upload=22562]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 8 07:29:34.528944789 (21h50m9.044047366s) [upload=22562]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 8 07:29:34.536881082 (21h50m9.051983672s) [upload=22562]: Next segment err: metaclient: Too Many Requests
Aug 8 07:29:34.536925024 (21h50m9.052027602s) [upload=22562]: Done uploading segments...
Aug 8 07:29:34.536943164 (21h50m9.052045741s) [upload=22562]: Upload finished. err: metaclient: Too Many Requests
Duration: 605.653994ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [......X..............................................X......] duration:
177.71033ms
FAIL (2025-08-08 07:35:38 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== nts... Aug 8 07:24:18.405184267 (21h44m52.920286834s) [upload=22475]: Upload finished. err: <nil> Aug 8 07:24:44.148856783 (21h45m18.663959349s) [upload=22481]: Starting upload Aug 8 07:24:44.148908040 (21h45m18.664010617s) [upload=22481]: Done starting upload Aug 8 07:24:44.148978727 (21h45m18.664081294s) [upload=22481]: Uploading segments... Aug 8 07:24:44.149031432 (21h45m18.664134016s) [upload=22481]: Got next segment. Inline: true Aug 8 07:24:44.149047929 (21h45m18.664150495s) [upload=22481]: Waiting for error group managing segments... Aug 8 07:24:44.149231742 (21h45m18.664334308s) [upload=22481]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:24:44.149238484 (21h45m18.664341061s) [upload=22481]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:24:44.149241695 (21h45m18.664344271s) [upload=22481]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:24:44.236175458 (21h45m18.751278039s) [upload=22481]: Done uploading segments... Aug 8 07:24:44.236197562 (21h45m18.75130015s) [upload=22481]: Upload finished. err: <nil> Aug 8 07:25:45.798907959 (21h46m20.31401054s) [upload=22496]: Starting upload Aug 8 07:25:45.798982852 (21h46m20.314085432s) [upload=22496]: Done starting upload Aug 8 07:25:45.799141617 (21h46m20.314244194s) [upload=22496]: Uploading segments... Aug 8 07:25:45.799187034 (21h46m20.314289621s) [upload=22496]: Got next segment. Inline: true Aug 8 07:25:45.799192794 (21h46m20.314295378s) [upload=22496]: Waiting for error group managing segments... Aug 8 07:25:45.799260506 (21h46m20.314363086s) [upload=22496]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:25:45.799265543 (21h46m20.314368123s) [upload=22496]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:25:45.799269614 (21h46m20.314372195s) [upload=22496]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:25:45.856873466 (21h46m20.371976033s) [upload=22496]: Done uploading segments... Aug 8 07:25:45.856894517 (21h46m20.371997083s) [upload=22496]: Upload finished. err: <nil> Aug 8 07:26:45.160306388 (21h47m19.67540897s) [upload=22512]: Starting upload Aug 8 07:26:45.160369840 (21h47m19.675472407s) [upload=22512]: Done starting upload Aug 8 07:26:45.160448308 (21h47m19.675550874s) [upload=22512]: Uploading segments... Aug 8 07:26:45.160512759 (21h47m19.675615339s) [upload=22512]: Got next segment. Inline: true Aug 8 07:26:45.160518821 (21h47m19.675621389s) [upload=22512]: Waiting for error group managing segments... Aug 8 07:26:45.160566486 (21h47m19.675669054s) [upload=22512]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:26:45.160571363 (21h47m19.675673943s) [upload=22512]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:26:45.160575116 (21h47m19.675677698s) [upload=22512]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:26:45.219521344 (21h47m19.73462392s) [upload=22512]: Done uploading segments... Aug 8 07:26:45.219537755 (21h47m19.73464032s) [upload=22512]: Upload finished. err: <nil> Aug 8 07:27:50.008745364 (21h48m24.523847971s) [upload=22529]: Starting upload Aug 8 07:27:50.008828662 (21h48m24.523931252s) [upload=22529]: Done starting upload Aug 8 07:27:50.008913720 (21h48m24.524016289s) [upload=22529]: Uploading segments... Aug 8 07:27:50.008968901 (21h48m24.524071486s) [upload=22529]: Got next segment. Inline: true Aug 8 07:27:50.008973893 (21h48m24.524076469s) [upload=22529]: Waiting for error group managing segments... Aug 8 07:27:50.009023439 (21h48m24.524126015s) [upload=22529]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:27:50.009028404 (21h48m24.524130983s) [upload=22529]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:27:50.009032740 (21h48m24.524135324s) [upload=22529]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:27:50.072222330 (21h48m24.587324909s) [upload=22529]: Done uploading segments... Aug 8 07:27:50.072260086 (21h48m24.587362652s) [upload=22529]: Upload finished. err: <nil> Aug 8 07:28:08.797868182 (21h48m43.312970759s) [upload=22537]: Starting upload Aug 8 07:28:08.797909617 (21h48m43.313012202s) [upload=22537]: Done starting upload Aug 8 07:28:08.797978107 (21h48m43.313080676s) [upload=22537]: Uploading segments... Aug 8 07:28:08.798028549 (21h48m43.313131134s) [upload=22537]: Got next segment. Inline: true Aug 8 07:28:08.798033746 (21h48m43.313136324s) [upload=22537]: Waiting for error group managing segments... Aug 8 07:28:08.798088133 (21h48m43.313190709s) [upload=22537]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:28:08.798092673 (21h48m43.313195249s) [upload=22537]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:28:08.798096028 (21h48m43.313198604s) [upload=22537]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:28:08.856181519 (21h48m43.371284096s) [upload=22537]: Done uploading segments... Aug 8 07:28:08.856219288 (21h48m43.371321856s) [upload=22537]: Upload finished. err: <nil> Aug 8 07:28:26.426406302 (21h49m0.941508883s) [upload=22543]: Starting upload Aug 8 07:28:26.426484520 (21h49m0.94158709s) [upload=22543]: Done starting upload Aug 8 07:28:26.426561912 (21h49m0.941664479s) [upload=22543]: Uploading segments... Aug 8 07:28:26.426613562 (21h49m0.941716147s) [upload=22543]: Got next segment. Inline: true Aug 8 07:28:26.426618334 (21h49m0.94172091s) [upload=22543]: Waiting for error group managing segments... Aug 8 07:28:26.426667358 (21h49m0.941769934s) [upload=22543]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:28:26.426672477 (21h49m0.941775052s) [upload=22543]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:28:26.426676980 (21h49m0.941779557s) [upload=22543]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:28:26.739786156 (21h49m1.254888734s) [upload=22543]: Done uploading segments... Aug 8 07:28:26.739818088 (21h49m1.254920663s) [upload=22543]: Upload finished. err: <nil> Aug 8 07:28:44.768272418 (21h49m19.283374995s) [upload=22547]: Starting upload Aug 8 07:28:44.768352182 (21h49m19.28345476s) [upload=22547]: Done starting upload Aug 8 07:28:44.768463680 (21h49m19.283566246s) [upload=22547]: Uploading segments... Aug 8 07:28:44.768552166 (21h49m19.283654733s) [upload=22547]: Got next segment. Inline: true Aug 8 07:28:44.768558003 (21h49m19.28366057s) [upload=22547]: Waiting for error group managing segments... Aug 8 07:28:44.768617460 (21h49m19.28372004s) [upload=22547]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:28:44.768622292 (21h49m19.283724868s) [upload=22547]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:28:44.768625505 (21h49m19.283728082s) [upload=22547]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:28:44.832641596 (21h49m19.347744163s) [upload=22547]: Done uploading segments... Aug 8 07:28:44.832661990 (21h49m19.347764559s) [upload=22547]: Upload finished. err: <nil> Aug 8 07:29:19.907807846 (21h49m54.422910428s) [upload=22558]: Starting upload Aug 8 07:29:19.907919209 (21h49m54.423021801s) [upload=22558]: Done starting upload Aug 8 07:29:19.908035700 (21h49m54.423138281s) [upload=22558]: Uploading segments... Aug 8 07:29:19.908115743 (21h49m54.423218334s) [upload=22558]: Got next segment. Inline: true Aug 8 07:29:19.908123216 (21h49m54.423225802s) [upload=22558]: Waiting for error group managing segments... Aug 8 07:29:19.908206488 (21h49m54.423309073s) [upload=22558]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:29:19.908213079 (21h49m54.423315659s) [upload=22558]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:29:19.908216828 (21h49m54.423319408s) [upload=22558]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:29:19.968416399 (21h49m54.483518987s) [upload=22558]: Done uploading segments... Aug 8 07:29:19.968437892 (21h49m54.48354047s) [upload=22558]: Upload finished. err: <nil> Aug 8 07:30:04.570811298 (21h50m39.085913875s) [upload=22574]: Starting upload Aug 8 07:30:04.570856561 (21h50m39.085959129s) [upload=22574]: Done starting upload Aug 8 07:30:04.570937395 (21h50m39.086039963s) [upload=22574]: Uploading segments... Aug 8 07:30:04.570993290 (21h50m39.086095879s) [upload=22574]: Got next segment. Inline: true Aug 8 07:30:04.570999714 (21h50m39.086102306s) [upload=22574]: Waiting for error group managing segments... Aug 8 07:30:04.571094735 (21h50m39.086197316s) [upload=22574]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:30:04.571101887 (21h50m39.086204467s) [upload=22574]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:30:04.571105790 (21h50m39.086208372s) [upload=22574]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:30:04.649336948 (21h50m39.164439536s) [upload=22574]: Done uploading segments... Aug 8 07:30:04.649362004 (21h50m39.164464585s) [upload=22574]: Upload finished. err: <nil> Aug 8 07:30:54.851564431 (21h51m29.366666997s) [upload=22587]: Starting upload Aug 8 07:30:54.851618044 (21h51m29.36672062s) [upload=22587]: Done starting upload Aug 8 07:30:54.851712059 (21h51m29.366814635s) [upload=22587]: Uploading segments... Aug 8 07:30:54.851783178 (21h51m29.36688576s) [upload=22587]: Got next segment. Inline: true Aug 8 07:30:54.851789517 (21h51m29.366892093s) [upload=22587]: Waiting for error group managing segments... Aug 8 07:30:54.851854337 (21h51m29.366956924s) [upload=22587]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:30:54.851861334 (21h51m29.366963924s) [upload=22587]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:30:54.851866059 (21h51m29.366968642s) [upload=22587]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:30:54.860593007 (21h51m29.375695595s) [upload=22587]: Done uploading segments... Aug 8 07:30:54.860627519 (21h51m29.375730095s) [upload=22587]: Upload finished. err: metaclient: Too Many Requests
Duration: 177.758034ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [.......X..............................................X.....] duration:
192.891726ms
FAIL (2025-08-08 07:35:06 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== nts... Aug 8 07:24:18.405184267 (21h44m52.920286834s) [upload=22475]: Upload finished. err: <nil> Aug 8 07:24:44.148856783 (21h45m18.663959349s) [upload=22481]: Starting upload Aug 8 07:24:44.148908040 (21h45m18.664010617s) [upload=22481]: Done starting upload Aug 8 07:24:44.148978727 (21h45m18.664081294s) [upload=22481]: Uploading segments... Aug 8 07:24:44.149031432 (21h45m18.664134016s) [upload=22481]: Got next segment. Inline: true Aug 8 07:24:44.149047929 (21h45m18.664150495s) [upload=22481]: Waiting for error group managing segments... Aug 8 07:24:44.149231742 (21h45m18.664334308s) [upload=22481]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:24:44.149238484 (21h45m18.664341061s) [upload=22481]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:24:44.149241695 (21h45m18.664344271s) [upload=22481]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:24:44.236175458 (21h45m18.751278039s) [upload=22481]: Done uploading segments... Aug 8 07:24:44.236197562 (21h45m18.75130015s) [upload=22481]: Upload finished. err: <nil> Aug 8 07:25:45.798907959 (21h46m20.31401054s) [upload=22496]: Starting upload Aug 8 07:25:45.798982852 (21h46m20.314085432s) [upload=22496]: Done starting upload Aug 8 07:25:45.799141617 (21h46m20.314244194s) [upload=22496]: Uploading segments... Aug 8 07:25:45.799187034 (21h46m20.314289621s) [upload=22496]: Got next segment. Inline: true Aug 8 07:25:45.799192794 (21h46m20.314295378s) [upload=22496]: Waiting for error group managing segments... Aug 8 07:25:45.799260506 (21h46m20.314363086s) [upload=22496]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:25:45.799265543 (21h46m20.314368123s) [upload=22496]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:25:45.799269614 (21h46m20.314372195s) [upload=22496]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:25:45.856873466 (21h46m20.371976033s) [upload=22496]: Done uploading segments... Aug 8 07:25:45.856894517 (21h46m20.371997083s) [upload=22496]: Upload finished. err: <nil> Aug 8 07:26:45.160306388 (21h47m19.67540897s) [upload=22512]: Starting upload Aug 8 07:26:45.160369840 (21h47m19.675472407s) [upload=22512]: Done starting upload Aug 8 07:26:45.160448308 (21h47m19.675550874s) [upload=22512]: Uploading segments... Aug 8 07:26:45.160512759 (21h47m19.675615339s) [upload=22512]: Got next segment. Inline: true Aug 8 07:26:45.160518821 (21h47m19.675621389s) [upload=22512]: Waiting for error group managing segments... Aug 8 07:26:45.160566486 (21h47m19.675669054s) [upload=22512]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:26:45.160571363 (21h47m19.675673943s) [upload=22512]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:26:45.160575116 (21h47m19.675677698s) [upload=22512]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:26:45.219521344 (21h47m19.73462392s) [upload=22512]: Done uploading segments... Aug 8 07:26:45.219537755 (21h47m19.73464032s) [upload=22512]: Upload finished. err: <nil> Aug 8 07:27:50.008745364 (21h48m24.523847971s) [upload=22529]: Starting upload Aug 8 07:27:50.008828662 (21h48m24.523931252s) [upload=22529]: Done starting upload Aug 8 07:27:50.008913720 (21h48m24.524016289s) [upload=22529]: Uploading segments... Aug 8 07:27:50.008968901 (21h48m24.524071486s) [upload=22529]: Got next segment. Inline: true Aug 8 07:27:50.008973893 (21h48m24.524076469s) [upload=22529]: Waiting for error group managing segments... Aug 8 07:27:50.009023439 (21h48m24.524126015s) [upload=22529]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:27:50.009028404 (21h48m24.524130983s) [upload=22529]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:27:50.009032740 (21h48m24.524135324s) [upload=22529]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:27:50.072222330 (21h48m24.587324909s) [upload=22529]: Done uploading segments... Aug 8 07:27:50.072260086 (21h48m24.587362652s) [upload=22529]: Upload finished. err: <nil> Aug 8 07:28:08.797868182 (21h48m43.312970759s) [upload=22537]: Starting upload Aug 8 07:28:08.797909617 (21h48m43.313012202s) [upload=22537]: Done starting upload Aug 8 07:28:08.797978107 (21h48m43.313080676s) [upload=22537]: Uploading segments... Aug 8 07:28:08.798028549 (21h48m43.313131134s) [upload=22537]: Got next segment. Inline: true Aug 8 07:28:08.798033746 (21h48m43.313136324s) [upload=22537]: Waiting for error group managing segments... Aug 8 07:28:08.798088133 (21h48m43.313190709s) [upload=22537]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:28:08.798092673 (21h48m43.313195249s) [upload=22537]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:28:08.798096028 (21h48m43.313198604s) [upload=22537]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:28:08.856181519 (21h48m43.371284096s) [upload=22537]: Done uploading segments... Aug 8 07:28:08.856219288 (21h48m43.371321856s) [upload=22537]: Upload finished. err: <nil> Aug 8 07:28:26.426406302 (21h49m0.941508883s) [upload=22543]: Starting upload Aug 8 07:28:26.426484520 (21h49m0.94158709s) [upload=22543]: Done starting upload Aug 8 07:28:26.426561912 (21h49m0.941664479s) [upload=22543]: Uploading segments... Aug 8 07:28:26.426613562 (21h49m0.941716147s) [upload=22543]: Got next segment. Inline: true Aug 8 07:28:26.426618334 (21h49m0.94172091s) [upload=22543]: Waiting for error group managing segments... Aug 8 07:28:26.426667358 (21h49m0.941769934s) [upload=22543]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:28:26.426672477 (21h49m0.941775052s) [upload=22543]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:28:26.426676980 (21h49m0.941779557s) [upload=22543]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:28:26.739786156 (21h49m1.254888734s) [upload=22543]: Done uploading segments... Aug 8 07:28:26.739818088 (21h49m1.254920663s) [upload=22543]: Upload finished. err: <nil> Aug 8 07:28:44.768272418 (21h49m19.283374995s) [upload=22547]: Starting upload Aug 8 07:28:44.768352182 (21h49m19.28345476s) [upload=22547]: Done starting upload Aug 8 07:28:44.768463680 (21h49m19.283566246s) [upload=22547]: Uploading segments... Aug 8 07:28:44.768552166 (21h49m19.283654733s) [upload=22547]: Got next segment. Inline: true Aug 8 07:28:44.768558003 (21h49m19.28366057s) [upload=22547]: Waiting for error group managing segments... Aug 8 07:28:44.768617460 (21h49m19.28372004s) [upload=22547]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:28:44.768622292 (21h49m19.283724868s) [upload=22547]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:28:44.768625505 (21h49m19.283728082s) [upload=22547]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:28:44.832641596 (21h49m19.347744163s) [upload=22547]: Done uploading segments... Aug 8 07:28:44.832661990 (21h49m19.347764559s) [upload=22547]: Upload finished. err: <nil> Aug 8 07:29:19.907807846 (21h49m54.422910428s) [upload=22558]: Starting upload Aug 8 07:29:19.907919209 (21h49m54.423021801s) [upload=22558]: Done starting upload Aug 8 07:29:19.908035700 (21h49m54.423138281s) [upload=22558]: Uploading segments... Aug 8 07:29:19.908115743 (21h49m54.423218334s) [upload=22558]: Got next segment. Inline: true Aug 8 07:29:19.908123216 (21h49m54.423225802s) [upload=22558]: Waiting for error group managing segments... Aug 8 07:29:19.908206488 (21h49m54.423309073s) [upload=22558]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:29:19.908213079 (21h49m54.423315659s) [upload=22558]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:29:19.908216828 (21h49m54.423319408s) [upload=22558]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:29:19.968416399 (21h49m54.483518987s) [upload=22558]: Done uploading segments... Aug 8 07:29:19.968437892 (21h49m54.48354047s) [upload=22558]: Upload finished. err: <nil> Aug 8 07:30:04.570811298 (21h50m39.085913875s) [upload=22574]: Starting upload Aug 8 07:30:04.570856561 (21h50m39.085959129s) [upload=22574]: Done starting upload Aug 8 07:30:04.570937395 (21h50m39.086039963s) [upload=22574]: Uploading segments... Aug 8 07:30:04.570993290 (21h50m39.086095879s) [upload=22574]: Got next segment. Inline: true Aug 8 07:30:04.570999714 (21h50m39.086102306s) [upload=22574]: Waiting for error group managing segments... Aug 8 07:30:04.571094735 (21h50m39.086197316s) [upload=22574]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:30:04.571101887 (21h50m39.086204467s) [upload=22574]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:30:04.571105790 (21h50m39.086208372s) [upload=22574]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:30:04.649336948 (21h50m39.164439536s) [upload=22574]: Done uploading segments... Aug 8 07:30:04.649362004 (21h50m39.164464585s) [upload=22574]: Upload finished. err: <nil> Aug 8 07:30:54.851564431 (21h51m29.366666997s) [upload=22587]: Starting upload Aug 8 07:30:54.851618044 (21h51m29.36672062s) [upload=22587]: Done starting upload Aug 8 07:30:54.851712059 (21h51m29.366814635s) [upload=22587]: Uploading segments... Aug 8 07:30:54.851783178 (21h51m29.36688576s) [upload=22587]: Got next segment. Inline: true Aug 8 07:30:54.851789517 (21h51m29.366892093s) [upload=22587]: Waiting for error group managing segments... Aug 8 07:30:54.851854337 (21h51m29.366956924s) [upload=22587]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:30:54.851861334 (21h51m29.366963924s) [upload=22587]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:30:54.851866059 (21h51m29.366968642s) [upload=22587]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:30:54.860593007 (21h51m29.375695595s) [upload=22587]: Done uploading segments... Aug 8 07:30:54.860627519 (21h51m29.375730095s) [upload=22587]: Upload finished. err: metaclient: Too Many Requests
Duration: 192.920197ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...] duration:
32.862089ms target:
https://us1.storj.io/login
FAIL (2025-08-08 07:34:54 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.150.199.48:443: connect: connection refused
Duration: 32.885552ms
core
satellite
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......] duration:
423.049081ms
FAIL (2025-08-08 07:34:51 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
g_pos={0 0}, upload=22550]: Begin upload segment...
Aug 8 07:28:51.351770179 (21h49m25.866872766s) [seg_pos={0 0}, upload=22550]: Done begin upload segment.
Aug 8 07:28:51.351799155 (21h49m25.866901747s) [noise=true , address=104.36.82.23:4469 , piece_id=CI6CU5IBYK37MBIQW554EW6PPXVYA6ZF2CJPVZCIXMFFLXZ7UO7Q, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.351816053 (21h49m25.86691864s) [noise=true , address=193.149.164.143:4693 , piece_id=RCLLY6ABOIXYK6BBO5EWKTDDJ4MUH6IP2WGIHSTER4TEPSF2FYVA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.352227136 (21h49m25.867329713s) [noise=true , address=104.36.82.58:4286 , piece_id=36LFLTD2YWV3K7LM4M2GY3JIKLUDOZVRINQT3VLIFBRR55UCNA4Q, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.352322195 (21h49m25.867424772s) [noise=true , address=104.36.82.44:4880 , piece_id=HFXFL7ULX5OTYJFEOC45DI75DGDIHAQA6ZJIXHO22X4GPSEZN75Q, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.352611348 (21h49m25.867713924s) [noise=true , address=104.36.82.53:4139 , piece_id=IGOHVQJMZ3NWLCBKVPVHTVWVHDS43DABQPEMJUTB2TXAQPQOURSQ, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.352702579 (21h49m25.867805142s) [noise=true , address=45.91.116.7:4013 , piece_id=XG4M7HF2E5YIRJ7PVBEGZC2ADPUPAL2QL5TPUAFIYXZUP3BX2Y7A, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.353022021 (21h49m25.868124597s) [noise=true , address=104.36.82.16:4321 , piece_id=MTRDWQYPHECWBAH4ECUGJL2Y2LAUA2RWHX5J6XMR65NZOACBF6PA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.353464633 (21h49m25.868567211s) [noise=true , address=104.36.82.46:4487 , piece_id=JOSIX5WT5A36RXKJNWO5KLBEQOCNQCEZ7A2S4KYIBJHRKN3KT6RQ, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.354418304 (21h49m25.869520884s) [noise=true , address=104.36.82.38:4707 , piece_id=WZWQHYDMTUG2FOTZ6JJRVIXXFIPHJ3I2YIRQXMN7B2W7QGKAFY3A, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.354959848 (21h49m25.87006243s) [noise=true , address=104.36.82.47:4544 , piece_id=YDG3KOKUS5NVBWMQYJ7JJEO3PAXZKPZPUOUWUDCZ4RAVATW67NKQ, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.355461506 (21h49m25.870564086s) [noise=true , address=104.36.82.60:4363 , piece_id=QMR7CM4ASNAKG5PT2XUPFR4MIBHNFYJ7OZMASJ7Q6R2C225L655A, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.355910519 (21h49m25.871013101s) [noise=true , address=104.36.82.2:4741 , piece_id=3NWVL6DQCMDO5PQLSFU3VER7RSHGRAB6YRT32XBFUVLAXGEJQN7A, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.357467009 (21h49m25.872569585s) [noise=true , address=104.36.82.59:4322 , piece_id=NKCSGWGYWOIJWHSQUN2ZYOVZYIRML5G3DXVRJYDTAMHTEN2VES3Q, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.358628018 (21h49m25.873730598s) [noise=true , address=45.91.116.11:4120 , piece_id=274GAU5MCZAZOHB3ESPDSJP23Z5FZB636CWGN7ELS2HYY5OFZ7RA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.359105086 (21h49m25.874207666s) [noise=true , address=104.36.82.45:4907 , piece_id=ZF2SKF5O2WVNHM7CHQ7E4U7HVKQWCJGOZEVIDXO3OU4XHVUFPIGA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.360554866 (21h49m25.875657451s) [noise=true , address=104.36.82.36:4648 , piece_id=3WAWCS3XXWVX3ACYJG2OOS2R34SXOWYSHXVF6AMPFXJFCMYH4SZA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.363015621 (21h49m25.8781182s) [noise=true , address=104.36.82.62:4420 , piece_id=7S6ALI3DNOM7IURBTYY3DLALDQV7R7LLHPQCA4Q6G4HA2SOZ2YMA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.364089235 (21h49m25.87919182s) [noise=true , address=104.36.82.9:4008 , piece_id=EGIGCEJHELTHOJZWPFWNHCP672MIXAHZIQUK2WDHER77VVJ4NZZA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.467747752 (21h49m25.982850337s) [upload=22550]: Next returned nil segment
Aug 8 07:28:51.467767955 (21h49m25.982870543s) [upload=22550]: Waiting for error group managing segments...
Aug 8 07:28:51.473577577 (21h49m25.988680143s) [noise=true , address=45.91.116.7:4013 , piece_id=XG4M7HF2E5YIRJ7PVBEGZC2ADPUPAL2QL5TPUAFIYXZUP3BX2Y7A, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.478188638 (21h49m25.993291216s) [noise=true , address=104.36.82.16:4321 , piece_id=MTRDWQYPHECWBAH4ECUGJL2Y2LAUA2RWHX5J6XMR65NZOACBF6PA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.478342049 (21h49m25.993444627s) [noise=true , address=104.36.82.53:4139 , piece_id=IGOHVQJMZ3NWLCBKVPVHTVWVHDS43DABQPEMJUTB2TXAQPQOURSQ, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.478768394 (21h49m25.993870961s) [noise=true , address=104.36.82.36:4648 , piece_id=3WAWCS3XXWVX3ACYJG2OOS2R34SXOWYSHXVF6AMPFXJFCMYH4SZA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.479336172 (21h49m25.994438735s) [noise=true , address=104.36.82.59:4322 , piece_id=NKCSGWGYWOIJWHSQUN2ZYOVZYIRML5G3DXVRJYDTAMHTEN2VES3Q, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.479515883 (21h49m25.994618457s) [noise=true , address=104.36.82.58:4286 , piece_id=36LFLTD2YWV3K7LM4M2GY3JIKLUDOZVRINQT3VLIFBRR55UCNA4Q, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.480474985 (21h49m25.995577567s) [noise=true , address=104.36.82.2:4741 , piece_id=3NWVL6DQCMDO5PQLSFU3VER7RSHGRAB6YRT32XBFUVLAXGEJQN7A, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.481008628 (21h49m25.996111195s) [noise=true , address=104.36.82.38:4707 , piece_id=WZWQHYDMTUG2FOTZ6JJRVIXXFIPHJ3I2YIRQXMN7B2W7QGKAFY3A, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.481589774 (21h49m25.996692358s) [noise=true , address=104.36.82.60:4363 , piece_id=QMR7CM4ASNAKG5PT2XUPFR4MIBHNFYJ7OZMASJ7Q6R2C225L655A, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.482143054 (21h49m25.99724562s) [noise=true , address=104.36.82.46:4487 , piece_id=JOSIX5WT5A36RXKJNWO5KLBEQOCNQCEZ7A2S4KYIBJHRKN3KT6RQ, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.485172215 (21h49m26.000274779s) [noise=true , address=104.36.82.47:4544 , piece_id=YDG3KOKUS5NVBWMQYJ7JJEO3PAXZKPZPUOUWUDCZ4RAVATW67NKQ, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.486088591 (21h49m26.001191172s) [noise=true , address=104.36.82.44:4880 , piece_id=HFXFL7ULX5OTYJFEOC45DI75DGDIHAQA6ZJIXHO22X4GPSEZN75Q, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.486628345 (21h49m26.001730916s) [noise=true , address=104.36.82.23:4469 , piece_id=CI6CU5IBYK37MBIQW554EW6PPXVYA6ZF2CJPVZCIXMFFLXZ7UO7Q, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.487179958 (21h49m26.002282521s) [noise=true , address=104.36.82.62:4420 , piece_id=7S6ALI3DNOM7IURBTYY3DLALDQV7R7LLHPQCA4Q6G4HA2SOZ2YMA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.487192580 (21h49m26.002295147s) [seg_pos={0 0}, upload=22550]: Segment reached optimal threshold of 14 pieces.
Aug 8 07:28:51.487306760 (21h49m26.002409325s) [noise=true , address=104.36.82.45:4907 , piece_id=ZF2SKF5O2WVNHM7CHQ7E4U7HVKQWCJGOZEVIDXO3OU4XHVUFPIGA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12URx18Ckh7VaAKJ1hdQQucMRu5qiDrAraqaycjYqxETgwZ4uWS): protocol: expected piece hash; context canceled; EOF
Aug 8 07:28:51.487325587 (21h49m26.00242815s) [noise=true , address=104.36.82.9:4008 , piece_id=EGIGCEJHELTHOJZWPFWNHCP672MIXAHZIQUK2WDHER77VVJ4NZZA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.487748790 (21h49m26.002851367s) [noise=true , address=193.149.164.143:4693 , piece_id=RCLLY6ABOIXYK6BBO5EWKTDDJ4MUH6IP2WGIHSTER4TEPSF2FYVA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LHoCE9CnM32tbMNqAB7HpmArSVLTycv9o6kSSFNUJsoDeERkQ): protocol: read tcp 159.223.143.39:37688->193.149.164.143:4693: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 07:28:51.487858041 (21h49m26.002960604s) [noise=true , address=45.91.116.11:4120 , piece_id=274GAU5MCZAZOHB3ESPDSJP23Z5FZB636CWGN7ELS2HYY5OFZ7RA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.487890658 (21h49m26.002993237s) [seg_pos={0 0}, upload=22550]: Done waiting for segment. successful: 16 optimal: 14 errs: <nil>
Aug 8 07:28:51.487971404 (21h49m26.00307399s) [upload=22550]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 8 07:28:51.487976917 (21h49m26.003079481s) [upload=22550]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 8 07:28:51.647177827 (21h49m26.162280406s) [upload=22550]: Done uploading segments...
Aug 8 07:28:51.647194442 (21h49m26.162297008s) [upload=22550]: Upload finished. err: <nil>
Aug 8 07:29:34.528501362 (21h50m9.043603939s) [upload=22562]: Starting upload
Aug 8 07:29:34.528548565 (21h50m9.043651143s) [upload=22562]: Done starting upload
Aug 8 07:29:34.528615374 (21h50m9.043717942s) [upload=22562]: Uploading segments...
Aug 8 07:29:34.528903298 (21h50m9.044005874s) [upload=22562]: Got next segment. Inline: false
Aug 8 07:29:34.528940157 (21h50m9.044042733s) [upload=22562]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 8 07:29:34.528944789 (21h50m9.044047366s) [upload=22562]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 8 07:29:34.536881082 (21h50m9.051983672s) [upload=22562]: Next segment err: metaclient: Too Many Requests
Aug 8 07:29:34.536925024 (21h50m9.052027602s) [upload=22562]: Done uploading segments...
Aug 8 07:29:34.536943164 (21h50m9.052045741s) [upload=22562]: Upload finished. err: metaclient: Too Many Requests
Duration: 423.095447ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......] duration:
444.601522ms
FAIL (2025-08-08 07:34:32 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
g_pos={0 0}, upload=22550]: Begin upload segment...
Aug 8 07:28:51.351770179 (21h49m25.866872766s) [seg_pos={0 0}, upload=22550]: Done begin upload segment.
Aug 8 07:28:51.351799155 (21h49m25.866901747s) [noise=true , address=104.36.82.23:4469 , piece_id=CI6CU5IBYK37MBIQW554EW6PPXVYA6ZF2CJPVZCIXMFFLXZ7UO7Q, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.351816053 (21h49m25.86691864s) [noise=true , address=193.149.164.143:4693 , piece_id=RCLLY6ABOIXYK6BBO5EWKTDDJ4MUH6IP2WGIHSTER4TEPSF2FYVA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.352227136 (21h49m25.867329713s) [noise=true , address=104.36.82.58:4286 , piece_id=36LFLTD2YWV3K7LM4M2GY3JIKLUDOZVRINQT3VLIFBRR55UCNA4Q, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.352322195 (21h49m25.867424772s) [noise=true , address=104.36.82.44:4880 , piece_id=HFXFL7ULX5OTYJFEOC45DI75DGDIHAQA6ZJIXHO22X4GPSEZN75Q, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.352611348 (21h49m25.867713924s) [noise=true , address=104.36.82.53:4139 , piece_id=IGOHVQJMZ3NWLCBKVPVHTVWVHDS43DABQPEMJUTB2TXAQPQOURSQ, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.352702579 (21h49m25.867805142s) [noise=true , address=45.91.116.7:4013 , piece_id=XG4M7HF2E5YIRJ7PVBEGZC2ADPUPAL2QL5TPUAFIYXZUP3BX2Y7A, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.353022021 (21h49m25.868124597s) [noise=true , address=104.36.82.16:4321 , piece_id=MTRDWQYPHECWBAH4ECUGJL2Y2LAUA2RWHX5J6XMR65NZOACBF6PA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.353464633 (21h49m25.868567211s) [noise=true , address=104.36.82.46:4487 , piece_id=JOSIX5WT5A36RXKJNWO5KLBEQOCNQCEZ7A2S4KYIBJHRKN3KT6RQ, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.354418304 (21h49m25.869520884s) [noise=true , address=104.36.82.38:4707 , piece_id=WZWQHYDMTUG2FOTZ6JJRVIXXFIPHJ3I2YIRQXMN7B2W7QGKAFY3A, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.354959848 (21h49m25.87006243s) [noise=true , address=104.36.82.47:4544 , piece_id=YDG3KOKUS5NVBWMQYJ7JJEO3PAXZKPZPUOUWUDCZ4RAVATW67NKQ, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.355461506 (21h49m25.870564086s) [noise=true , address=104.36.82.60:4363 , piece_id=QMR7CM4ASNAKG5PT2XUPFR4MIBHNFYJ7OZMASJ7Q6R2C225L655A, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.355910519 (21h49m25.871013101s) [noise=true , address=104.36.82.2:4741 , piece_id=3NWVL6DQCMDO5PQLSFU3VER7RSHGRAB6YRT32XBFUVLAXGEJQN7A, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.357467009 (21h49m25.872569585s) [noise=true , address=104.36.82.59:4322 , piece_id=NKCSGWGYWOIJWHSQUN2ZYOVZYIRML5G3DXVRJYDTAMHTEN2VES3Q, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.358628018 (21h49m25.873730598s) [noise=true , address=45.91.116.11:4120 , piece_id=274GAU5MCZAZOHB3ESPDSJP23Z5FZB636CWGN7ELS2HYY5OFZ7RA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.359105086 (21h49m25.874207666s) [noise=true , address=104.36.82.45:4907 , piece_id=ZF2SKF5O2WVNHM7CHQ7E4U7HVKQWCJGOZEVIDXO3OU4XHVUFPIGA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.360554866 (21h49m25.875657451s) [noise=true , address=104.36.82.36:4648 , piece_id=3WAWCS3XXWVX3ACYJG2OOS2R34SXOWYSHXVF6AMPFXJFCMYH4SZA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.363015621 (21h49m25.8781182s) [noise=true , address=104.36.82.62:4420 , piece_id=7S6ALI3DNOM7IURBTYY3DLALDQV7R7LLHPQCA4Q6G4HA2SOZ2YMA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.364089235 (21h49m25.87919182s) [noise=true , address=104.36.82.9:4008 , piece_id=EGIGCEJHELTHOJZWPFWNHCP672MIXAHZIQUK2WDHER77VVJ4NZZA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.467747752 (21h49m25.982850337s) [upload=22550]: Next returned nil segment
Aug 8 07:28:51.467767955 (21h49m25.982870543s) [upload=22550]: Waiting for error group managing segments...
Aug 8 07:28:51.473577577 (21h49m25.988680143s) [noise=true , address=45.91.116.7:4013 , piece_id=XG4M7HF2E5YIRJ7PVBEGZC2ADPUPAL2QL5TPUAFIYXZUP3BX2Y7A, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.478188638 (21h49m25.993291216s) [noise=true , address=104.36.82.16:4321 , piece_id=MTRDWQYPHECWBAH4ECUGJL2Y2LAUA2RWHX5J6XMR65NZOACBF6PA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.478342049 (21h49m25.993444627s) [noise=true , address=104.36.82.53:4139 , piece_id=IGOHVQJMZ3NWLCBKVPVHTVWVHDS43DABQPEMJUTB2TXAQPQOURSQ, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.478768394 (21h49m25.993870961s) [noise=true , address=104.36.82.36:4648 , piece_id=3WAWCS3XXWVX3ACYJG2OOS2R34SXOWYSHXVF6AMPFXJFCMYH4SZA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.479336172 (21h49m25.994438735s) [noise=true , address=104.36.82.59:4322 , piece_id=NKCSGWGYWOIJWHSQUN2ZYOVZYIRML5G3DXVRJYDTAMHTEN2VES3Q, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.479515883 (21h49m25.994618457s) [noise=true , address=104.36.82.58:4286 , piece_id=36LFLTD2YWV3K7LM4M2GY3JIKLUDOZVRINQT3VLIFBRR55UCNA4Q, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.480474985 (21h49m25.995577567s) [noise=true , address=104.36.82.2:4741 , piece_id=3NWVL6DQCMDO5PQLSFU3VER7RSHGRAB6YRT32XBFUVLAXGEJQN7A, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.481008628 (21h49m25.996111195s) [noise=true , address=104.36.82.38:4707 , piece_id=WZWQHYDMTUG2FOTZ6JJRVIXXFIPHJ3I2YIRQXMN7B2W7QGKAFY3A, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.481589774 (21h49m25.996692358s) [noise=true , address=104.36.82.60:4363 , piece_id=QMR7CM4ASNAKG5PT2XUPFR4MIBHNFYJ7OZMASJ7Q6R2C225L655A, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.482143054 (21h49m25.99724562s) [noise=true , address=104.36.82.46:4487 , piece_id=JOSIX5WT5A36RXKJNWO5KLBEQOCNQCEZ7A2S4KYIBJHRKN3KT6RQ, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.485172215 (21h49m26.000274779s) [noise=true , address=104.36.82.47:4544 , piece_id=YDG3KOKUS5NVBWMQYJ7JJEO3PAXZKPZPUOUWUDCZ4RAVATW67NKQ, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.486088591 (21h49m26.001191172s) [noise=true , address=104.36.82.44:4880 , piece_id=HFXFL7ULX5OTYJFEOC45DI75DGDIHAQA6ZJIXHO22X4GPSEZN75Q, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.486628345 (21h49m26.001730916s) [noise=true , address=104.36.82.23:4469 , piece_id=CI6CU5IBYK37MBIQW554EW6PPXVYA6ZF2CJPVZCIXMFFLXZ7UO7Q, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.487179958 (21h49m26.002282521s) [noise=true , address=104.36.82.62:4420 , piece_id=7S6ALI3DNOM7IURBTYY3DLALDQV7R7LLHPQCA4Q6G4HA2SOZ2YMA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.487192580 (21h49m26.002295147s) [seg_pos={0 0}, upload=22550]: Segment reached optimal threshold of 14 pieces.
Aug 8 07:28:51.487306760 (21h49m26.002409325s) [noise=true , address=104.36.82.45:4907 , piece_id=ZF2SKF5O2WVNHM7CHQ7E4U7HVKQWCJGOZEVIDXO3OU4XHVUFPIGA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12URx18Ckh7VaAKJ1hdQQucMRu5qiDrAraqaycjYqxETgwZ4uWS): protocol: expected piece hash; context canceled; EOF
Aug 8 07:28:51.487325587 (21h49m26.00242815s) [noise=true , address=104.36.82.9:4008 , piece_id=EGIGCEJHELTHOJZWPFWNHCP672MIXAHZIQUK2WDHER77VVJ4NZZA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.487748790 (21h49m26.002851367s) [noise=true , address=193.149.164.143:4693 , piece_id=RCLLY6ABOIXYK6BBO5EWKTDDJ4MUH6IP2WGIHSTER4TEPSF2FYVA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LHoCE9CnM32tbMNqAB7HpmArSVLTycv9o6kSSFNUJsoDeERkQ): protocol: read tcp 159.223.143.39:37688->193.149.164.143:4693: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 07:28:51.487858041 (21h49m26.002960604s) [noise=true , address=45.91.116.11:4120 , piece_id=274GAU5MCZAZOHB3ESPDSJP23Z5FZB636CWGN7ELS2HYY5OFZ7RA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.487890658 (21h49m26.002993237s) [seg_pos={0 0}, upload=22550]: Done waiting for segment. successful: 16 optimal: 14 errs: <nil>
Aug 8 07:28:51.487971404 (21h49m26.00307399s) [upload=22550]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 8 07:28:51.487976917 (21h49m26.003079481s) [upload=22550]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 8 07:28:51.647177827 (21h49m26.162280406s) [upload=22550]: Done uploading segments...
Aug 8 07:28:51.647194442 (21h49m26.162297008s) [upload=22550]: Upload finished. err: <nil>
Aug 8 07:29:34.528501362 (21h50m9.043603939s) [upload=22562]: Starting upload
Aug 8 07:29:34.528548565 (21h50m9.043651143s) [upload=22562]: Done starting upload
Aug 8 07:29:34.528615374 (21h50m9.043717942s) [upload=22562]: Uploading segments...
Aug 8 07:29:34.528903298 (21h50m9.044005874s) [upload=22562]: Got next segment. Inline: false
Aug 8 07:29:34.528940157 (21h50m9.044042733s) [upload=22562]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 8 07:29:34.528944789 (21h50m9.044047366s) [upload=22562]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 8 07:29:34.536881082 (21h50m9.051983672s) [upload=22562]: Next segment err: metaclient: Too Many Requests
Aug 8 07:29:34.536925024 (21h50m9.052027602s) [upload=22562]: Done uploading segments...
Aug 8 07:29:34.536943164 (21h50m9.052045741s) [upload=22562]: Upload finished. err: metaclient: Too Many Requests
Duration: 444.627065ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [........X..............................................X....] duration:
199.913785ms
FAIL (2025-08-08 07:33:59 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== nts... Aug 8 07:24:18.405184267 (21h44m52.920286834s) [upload=22475]: Upload finished. err: <nil> Aug 8 07:24:44.148856783 (21h45m18.663959349s) [upload=22481]: Starting upload Aug 8 07:24:44.148908040 (21h45m18.664010617s) [upload=22481]: Done starting upload Aug 8 07:24:44.148978727 (21h45m18.664081294s) [upload=22481]: Uploading segments... Aug 8 07:24:44.149031432 (21h45m18.664134016s) [upload=22481]: Got next segment. Inline: true Aug 8 07:24:44.149047929 (21h45m18.664150495s) [upload=22481]: Waiting for error group managing segments... Aug 8 07:24:44.149231742 (21h45m18.664334308s) [upload=22481]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:24:44.149238484 (21h45m18.664341061s) [upload=22481]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:24:44.149241695 (21h45m18.664344271s) [upload=22481]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:24:44.236175458 (21h45m18.751278039s) [upload=22481]: Done uploading segments... Aug 8 07:24:44.236197562 (21h45m18.75130015s) [upload=22481]: Upload finished. err: <nil> Aug 8 07:25:45.798907959 (21h46m20.31401054s) [upload=22496]: Starting upload Aug 8 07:25:45.798982852 (21h46m20.314085432s) [upload=22496]: Done starting upload Aug 8 07:25:45.799141617 (21h46m20.314244194s) [upload=22496]: Uploading segments... Aug 8 07:25:45.799187034 (21h46m20.314289621s) [upload=22496]: Got next segment. Inline: true Aug 8 07:25:45.799192794 (21h46m20.314295378s) [upload=22496]: Waiting for error group managing segments... Aug 8 07:25:45.799260506 (21h46m20.314363086s) [upload=22496]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:25:45.799265543 (21h46m20.314368123s) [upload=22496]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:25:45.799269614 (21h46m20.314372195s) [upload=22496]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:25:45.856873466 (21h46m20.371976033s) [upload=22496]: Done uploading segments... Aug 8 07:25:45.856894517 (21h46m20.371997083s) [upload=22496]: Upload finished. err: <nil> Aug 8 07:26:45.160306388 (21h47m19.67540897s) [upload=22512]: Starting upload Aug 8 07:26:45.160369840 (21h47m19.675472407s) [upload=22512]: Done starting upload Aug 8 07:26:45.160448308 (21h47m19.675550874s) [upload=22512]: Uploading segments... Aug 8 07:26:45.160512759 (21h47m19.675615339s) [upload=22512]: Got next segment. Inline: true Aug 8 07:26:45.160518821 (21h47m19.675621389s) [upload=22512]: Waiting for error group managing segments... Aug 8 07:26:45.160566486 (21h47m19.675669054s) [upload=22512]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:26:45.160571363 (21h47m19.675673943s) [upload=22512]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:26:45.160575116 (21h47m19.675677698s) [upload=22512]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:26:45.219521344 (21h47m19.73462392s) [upload=22512]: Done uploading segments... Aug 8 07:26:45.219537755 (21h47m19.73464032s) [upload=22512]: Upload finished. err: <nil> Aug 8 07:27:50.008745364 (21h48m24.523847971s) [upload=22529]: Starting upload Aug 8 07:27:50.008828662 (21h48m24.523931252s) [upload=22529]: Done starting upload Aug 8 07:27:50.008913720 (21h48m24.524016289s) [upload=22529]: Uploading segments... Aug 8 07:27:50.008968901 (21h48m24.524071486s) [upload=22529]: Got next segment. Inline: true Aug 8 07:27:50.008973893 (21h48m24.524076469s) [upload=22529]: Waiting for error group managing segments... Aug 8 07:27:50.009023439 (21h48m24.524126015s) [upload=22529]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:27:50.009028404 (21h48m24.524130983s) [upload=22529]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:27:50.009032740 (21h48m24.524135324s) [upload=22529]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:27:50.072222330 (21h48m24.587324909s) [upload=22529]: Done uploading segments... Aug 8 07:27:50.072260086 (21h48m24.587362652s) [upload=22529]: Upload finished. err: <nil> Aug 8 07:28:08.797868182 (21h48m43.312970759s) [upload=22537]: Starting upload Aug 8 07:28:08.797909617 (21h48m43.313012202s) [upload=22537]: Done starting upload Aug 8 07:28:08.797978107 (21h48m43.313080676s) [upload=22537]: Uploading segments... Aug 8 07:28:08.798028549 (21h48m43.313131134s) [upload=22537]: Got next segment. Inline: true Aug 8 07:28:08.798033746 (21h48m43.313136324s) [upload=22537]: Waiting for error group managing segments... Aug 8 07:28:08.798088133 (21h48m43.313190709s) [upload=22537]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:28:08.798092673 (21h48m43.313195249s) [upload=22537]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:28:08.798096028 (21h48m43.313198604s) [upload=22537]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:28:08.856181519 (21h48m43.371284096s) [upload=22537]: Done uploading segments... Aug 8 07:28:08.856219288 (21h48m43.371321856s) [upload=22537]: Upload finished. err: <nil> Aug 8 07:28:26.426406302 (21h49m0.941508883s) [upload=22543]: Starting upload Aug 8 07:28:26.426484520 (21h49m0.94158709s) [upload=22543]: Done starting upload Aug 8 07:28:26.426561912 (21h49m0.941664479s) [upload=22543]: Uploading segments... Aug 8 07:28:26.426613562 (21h49m0.941716147s) [upload=22543]: Got next segment. Inline: true Aug 8 07:28:26.426618334 (21h49m0.94172091s) [upload=22543]: Waiting for error group managing segments... Aug 8 07:28:26.426667358 (21h49m0.941769934s) [upload=22543]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:28:26.426672477 (21h49m0.941775052s) [upload=22543]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:28:26.426676980 (21h49m0.941779557s) [upload=22543]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:28:26.739786156 (21h49m1.254888734s) [upload=22543]: Done uploading segments... Aug 8 07:28:26.739818088 (21h49m1.254920663s) [upload=22543]: Upload finished. err: <nil> Aug 8 07:28:44.768272418 (21h49m19.283374995s) [upload=22547]: Starting upload Aug 8 07:28:44.768352182 (21h49m19.28345476s) [upload=22547]: Done starting upload Aug 8 07:28:44.768463680 (21h49m19.283566246s) [upload=22547]: Uploading segments... Aug 8 07:28:44.768552166 (21h49m19.283654733s) [upload=22547]: Got next segment. Inline: true Aug 8 07:28:44.768558003 (21h49m19.28366057s) [upload=22547]: Waiting for error group managing segments... Aug 8 07:28:44.768617460 (21h49m19.28372004s) [upload=22547]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:28:44.768622292 (21h49m19.283724868s) [upload=22547]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:28:44.768625505 (21h49m19.283728082s) [upload=22547]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:28:44.832641596 (21h49m19.347744163s) [upload=22547]: Done uploading segments... Aug 8 07:28:44.832661990 (21h49m19.347764559s) [upload=22547]: Upload finished. err: <nil> Aug 8 07:29:19.907807846 (21h49m54.422910428s) [upload=22558]: Starting upload Aug 8 07:29:19.907919209 (21h49m54.423021801s) [upload=22558]: Done starting upload Aug 8 07:29:19.908035700 (21h49m54.423138281s) [upload=22558]: Uploading segments... Aug 8 07:29:19.908115743 (21h49m54.423218334s) [upload=22558]: Got next segment. Inline: true Aug 8 07:29:19.908123216 (21h49m54.423225802s) [upload=22558]: Waiting for error group managing segments... Aug 8 07:29:19.908206488 (21h49m54.423309073s) [upload=22558]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:29:19.908213079 (21h49m54.423315659s) [upload=22558]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:29:19.908216828 (21h49m54.423319408s) [upload=22558]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:29:19.968416399 (21h49m54.483518987s) [upload=22558]: Done uploading segments... Aug 8 07:29:19.968437892 (21h49m54.48354047s) [upload=22558]: Upload finished. err: <nil> Aug 8 07:30:04.570811298 (21h50m39.085913875s) [upload=22574]: Starting upload Aug 8 07:30:04.570856561 (21h50m39.085959129s) [upload=22574]: Done starting upload Aug 8 07:30:04.570937395 (21h50m39.086039963s) [upload=22574]: Uploading segments... Aug 8 07:30:04.570993290 (21h50m39.086095879s) [upload=22574]: Got next segment. Inline: true Aug 8 07:30:04.570999714 (21h50m39.086102306s) [upload=22574]: Waiting for error group managing segments... Aug 8 07:30:04.571094735 (21h50m39.086197316s) [upload=22574]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:30:04.571101887 (21h50m39.086204467s) [upload=22574]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:30:04.571105790 (21h50m39.086208372s) [upload=22574]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:30:04.649336948 (21h50m39.164439536s) [upload=22574]: Done uploading segments... Aug 8 07:30:04.649362004 (21h50m39.164464585s) [upload=22574]: Upload finished. err: <nil> Aug 8 07:30:54.851564431 (21h51m29.366666997s) [upload=22587]: Starting upload Aug 8 07:30:54.851618044 (21h51m29.36672062s) [upload=22587]: Done starting upload Aug 8 07:30:54.851712059 (21h51m29.366814635s) [upload=22587]: Uploading segments... Aug 8 07:30:54.851783178 (21h51m29.36688576s) [upload=22587]: Got next segment. Inline: true Aug 8 07:30:54.851789517 (21h51m29.366892093s) [upload=22587]: Waiting for error group managing segments... Aug 8 07:30:54.851854337 (21h51m29.366956924s) [upload=22587]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:30:54.851861334 (21h51m29.366963924s) [upload=22587]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:30:54.851866059 (21h51m29.366968642s) [upload=22587]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:30:54.860593007 (21h51m29.375695595s) [upload=22587]: Done uploading segments... Aug 8 07:30:54.860627519 (21h51m29.375730095s) [upload=22587]: Upload finished. err: metaclient: Too Many Requests
Duration: 199.94153ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..] duration:
31.9539ms target:
https://us1.storj.io/login
FAIL (2025-08-08 07:33:50 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.150.199.48:443: connect: connection refused
Duration: 31.982417ms
core
satellite
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....] duration:
435.53174ms
FAIL (2025-08-08 07:33:48 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
g_pos={0 0}, upload=22550]: Begin upload segment...
Aug 8 07:28:51.351770179 (21h49m25.866872766s) [seg_pos={0 0}, upload=22550]: Done begin upload segment.
Aug 8 07:28:51.351799155 (21h49m25.866901747s) [noise=true , address=104.36.82.23:4469 , piece_id=CI6CU5IBYK37MBIQW554EW6PPXVYA6ZF2CJPVZCIXMFFLXZ7UO7Q, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.351816053 (21h49m25.86691864s) [noise=true , address=193.149.164.143:4693 , piece_id=RCLLY6ABOIXYK6BBO5EWKTDDJ4MUH6IP2WGIHSTER4TEPSF2FYVA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.352227136 (21h49m25.867329713s) [noise=true , address=104.36.82.58:4286 , piece_id=36LFLTD2YWV3K7LM4M2GY3JIKLUDOZVRINQT3VLIFBRR55UCNA4Q, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.352322195 (21h49m25.867424772s) [noise=true , address=104.36.82.44:4880 , piece_id=HFXFL7ULX5OTYJFEOC45DI75DGDIHAQA6ZJIXHO22X4GPSEZN75Q, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.352611348 (21h49m25.867713924s) [noise=true , address=104.36.82.53:4139 , piece_id=IGOHVQJMZ3NWLCBKVPVHTVWVHDS43DABQPEMJUTB2TXAQPQOURSQ, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.352702579 (21h49m25.867805142s) [noise=true , address=45.91.116.7:4013 , piece_id=XG4M7HF2E5YIRJ7PVBEGZC2ADPUPAL2QL5TPUAFIYXZUP3BX2Y7A, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.353022021 (21h49m25.868124597s) [noise=true , address=104.36.82.16:4321 , piece_id=MTRDWQYPHECWBAH4ECUGJL2Y2LAUA2RWHX5J6XMR65NZOACBF6PA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.353464633 (21h49m25.868567211s) [noise=true , address=104.36.82.46:4487 , piece_id=JOSIX5WT5A36RXKJNWO5KLBEQOCNQCEZ7A2S4KYIBJHRKN3KT6RQ, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.354418304 (21h49m25.869520884s) [noise=true , address=104.36.82.38:4707 , piece_id=WZWQHYDMTUG2FOTZ6JJRVIXXFIPHJ3I2YIRQXMN7B2W7QGKAFY3A, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.354959848 (21h49m25.87006243s) [noise=true , address=104.36.82.47:4544 , piece_id=YDG3KOKUS5NVBWMQYJ7JJEO3PAXZKPZPUOUWUDCZ4RAVATW67NKQ, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.355461506 (21h49m25.870564086s) [noise=true , address=104.36.82.60:4363 , piece_id=QMR7CM4ASNAKG5PT2XUPFR4MIBHNFYJ7OZMASJ7Q6R2C225L655A, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.355910519 (21h49m25.871013101s) [noise=true , address=104.36.82.2:4741 , piece_id=3NWVL6DQCMDO5PQLSFU3VER7RSHGRAB6YRT32XBFUVLAXGEJQN7A, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.357467009 (21h49m25.872569585s) [noise=true , address=104.36.82.59:4322 , piece_id=NKCSGWGYWOIJWHSQUN2ZYOVZYIRML5G3DXVRJYDTAMHTEN2VES3Q, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.358628018 (21h49m25.873730598s) [noise=true , address=45.91.116.11:4120 , piece_id=274GAU5MCZAZOHB3ESPDSJP23Z5FZB636CWGN7ELS2HYY5OFZ7RA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.359105086 (21h49m25.874207666s) [noise=true , address=104.36.82.45:4907 , piece_id=ZF2SKF5O2WVNHM7CHQ7E4U7HVKQWCJGOZEVIDXO3OU4XHVUFPIGA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.360554866 (21h49m25.875657451s) [noise=true , address=104.36.82.36:4648 , piece_id=3WAWCS3XXWVX3ACYJG2OOS2R34SXOWYSHXVF6AMPFXJFCMYH4SZA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.363015621 (21h49m25.8781182s) [noise=true , address=104.36.82.62:4420 , piece_id=7S6ALI3DNOM7IURBTYY3DLALDQV7R7LLHPQCA4Q6G4HA2SOZ2YMA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.364089235 (21h49m25.87919182s) [noise=true , address=104.36.82.9:4008 , piece_id=EGIGCEJHELTHOJZWPFWNHCP672MIXAHZIQUK2WDHER77VVJ4NZZA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.467747752 (21h49m25.982850337s) [upload=22550]: Next returned nil segment
Aug 8 07:28:51.467767955 (21h49m25.982870543s) [upload=22550]: Waiting for error group managing segments...
Aug 8 07:28:51.473577577 (21h49m25.988680143s) [noise=true , address=45.91.116.7:4013 , piece_id=XG4M7HF2E5YIRJ7PVBEGZC2ADPUPAL2QL5TPUAFIYXZUP3BX2Y7A, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.478188638 (21h49m25.993291216s) [noise=true , address=104.36.82.16:4321 , piece_id=MTRDWQYPHECWBAH4ECUGJL2Y2LAUA2RWHX5J6XMR65NZOACBF6PA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.478342049 (21h49m25.993444627s) [noise=true , address=104.36.82.53:4139 , piece_id=IGOHVQJMZ3NWLCBKVPVHTVWVHDS43DABQPEMJUTB2TXAQPQOURSQ, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.478768394 (21h49m25.993870961s) [noise=true , address=104.36.82.36:4648 , piece_id=3WAWCS3XXWVX3ACYJG2OOS2R34SXOWYSHXVF6AMPFXJFCMYH4SZA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.479336172 (21h49m25.994438735s) [noise=true , address=104.36.82.59:4322 , piece_id=NKCSGWGYWOIJWHSQUN2ZYOVZYIRML5G3DXVRJYDTAMHTEN2VES3Q, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.479515883 (21h49m25.994618457s) [noise=true , address=104.36.82.58:4286 , piece_id=36LFLTD2YWV3K7LM4M2GY3JIKLUDOZVRINQT3VLIFBRR55UCNA4Q, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.480474985 (21h49m25.995577567s) [noise=true , address=104.36.82.2:4741 , piece_id=3NWVL6DQCMDO5PQLSFU3VER7RSHGRAB6YRT32XBFUVLAXGEJQN7A, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.481008628 (21h49m25.996111195s) [noise=true , address=104.36.82.38:4707 , piece_id=WZWQHYDMTUG2FOTZ6JJRVIXXFIPHJ3I2YIRQXMN7B2W7QGKAFY3A, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.481589774 (21h49m25.996692358s) [noise=true , address=104.36.82.60:4363 , piece_id=QMR7CM4ASNAKG5PT2XUPFR4MIBHNFYJ7OZMASJ7Q6R2C225L655A, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.482143054 (21h49m25.99724562s) [noise=true , address=104.36.82.46:4487 , piece_id=JOSIX5WT5A36RXKJNWO5KLBEQOCNQCEZ7A2S4KYIBJHRKN3KT6RQ, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.485172215 (21h49m26.000274779s) [noise=true , address=104.36.82.47:4544 , piece_id=YDG3KOKUS5NVBWMQYJ7JJEO3PAXZKPZPUOUWUDCZ4RAVATW67NKQ, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.486088591 (21h49m26.001191172s) [noise=true , address=104.36.82.44:4880 , piece_id=HFXFL7ULX5OTYJFEOC45DI75DGDIHAQA6ZJIXHO22X4GPSEZN75Q, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.486628345 (21h49m26.001730916s) [noise=true , address=104.36.82.23:4469 , piece_id=CI6CU5IBYK37MBIQW554EW6PPXVYA6ZF2CJPVZCIXMFFLXZ7UO7Q, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.487179958 (21h49m26.002282521s) [noise=true , address=104.36.82.62:4420 , piece_id=7S6ALI3DNOM7IURBTYY3DLALDQV7R7LLHPQCA4Q6G4HA2SOZ2YMA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.487192580 (21h49m26.002295147s) [seg_pos={0 0}, upload=22550]: Segment reached optimal threshold of 14 pieces.
Aug 8 07:28:51.487306760 (21h49m26.002409325s) [noise=true , address=104.36.82.45:4907 , piece_id=ZF2SKF5O2WVNHM7CHQ7E4U7HVKQWCJGOZEVIDXO3OU4XHVUFPIGA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12URx18Ckh7VaAKJ1hdQQucMRu5qiDrAraqaycjYqxETgwZ4uWS): protocol: expected piece hash; context canceled; EOF
Aug 8 07:28:51.487325587 (21h49m26.00242815s) [noise=true , address=104.36.82.9:4008 , piece_id=EGIGCEJHELTHOJZWPFWNHCP672MIXAHZIQUK2WDHER77VVJ4NZZA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.487748790 (21h49m26.002851367s) [noise=true , address=193.149.164.143:4693 , piece_id=RCLLY6ABOIXYK6BBO5EWKTDDJ4MUH6IP2WGIHSTER4TEPSF2FYVA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LHoCE9CnM32tbMNqAB7HpmArSVLTycv9o6kSSFNUJsoDeERkQ): protocol: read tcp 159.223.143.39:37688->193.149.164.143:4693: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 07:28:51.487858041 (21h49m26.002960604s) [noise=true , address=45.91.116.11:4120 , piece_id=274GAU5MCZAZOHB3ESPDSJP23Z5FZB636CWGN7ELS2HYY5OFZ7RA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.487890658 (21h49m26.002993237s) [seg_pos={0 0}, upload=22550]: Done waiting for segment. successful: 16 optimal: 14 errs: <nil>
Aug 8 07:28:51.487971404 (21h49m26.00307399s) [upload=22550]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 8 07:28:51.487976917 (21h49m26.003079481s) [upload=22550]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 8 07:28:51.647177827 (21h49m26.162280406s) [upload=22550]: Done uploading segments...
Aug 8 07:28:51.647194442 (21h49m26.162297008s) [upload=22550]: Upload finished. err: <nil>
Aug 8 07:29:34.528501362 (21h50m9.043603939s) [upload=22562]: Starting upload
Aug 8 07:29:34.528548565 (21h50m9.043651143s) [upload=22562]: Done starting upload
Aug 8 07:29:34.528615374 (21h50m9.043717942s) [upload=22562]: Uploading segments...
Aug 8 07:29:34.528903298 (21h50m9.044005874s) [upload=22562]: Got next segment. Inline: false
Aug 8 07:29:34.528940157 (21h50m9.044042733s) [upload=22562]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 8 07:29:34.528944789 (21h50m9.044047366s) [upload=22562]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 8 07:29:34.536881082 (21h50m9.051983672s) [upload=22562]: Next segment err: metaclient: Too Many Requests
Aug 8 07:29:34.536925024 (21h50m9.052027602s) [upload=22562]: Done uploading segments...
Aug 8 07:29:34.536943164 (21h50m9.052045741s) [upload=22562]: Upload finished. err: metaclient: Too Many Requests
Duration: 435.56586ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....] duration:
398.012971ms
FAIL (2025-08-08 07:33:21 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
g_pos={0 0}, upload=22550]: Begin upload segment...
Aug 8 07:28:51.351770179 (21h49m25.866872766s) [seg_pos={0 0}, upload=22550]: Done begin upload segment.
Aug 8 07:28:51.351799155 (21h49m25.866901747s) [noise=true , address=104.36.82.23:4469 , piece_id=CI6CU5IBYK37MBIQW554EW6PPXVYA6ZF2CJPVZCIXMFFLXZ7UO7Q, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.351816053 (21h49m25.86691864s) [noise=true , address=193.149.164.143:4693 , piece_id=RCLLY6ABOIXYK6BBO5EWKTDDJ4MUH6IP2WGIHSTER4TEPSF2FYVA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.352227136 (21h49m25.867329713s) [noise=true , address=104.36.82.58:4286 , piece_id=36LFLTD2YWV3K7LM4M2GY3JIKLUDOZVRINQT3VLIFBRR55UCNA4Q, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.352322195 (21h49m25.867424772s) [noise=true , address=104.36.82.44:4880 , piece_id=HFXFL7ULX5OTYJFEOC45DI75DGDIHAQA6ZJIXHO22X4GPSEZN75Q, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.352611348 (21h49m25.867713924s) [noise=true , address=104.36.82.53:4139 , piece_id=IGOHVQJMZ3NWLCBKVPVHTVWVHDS43DABQPEMJUTB2TXAQPQOURSQ, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.352702579 (21h49m25.867805142s) [noise=true , address=45.91.116.7:4013 , piece_id=XG4M7HF2E5YIRJ7PVBEGZC2ADPUPAL2QL5TPUAFIYXZUP3BX2Y7A, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.353022021 (21h49m25.868124597s) [noise=true , address=104.36.82.16:4321 , piece_id=MTRDWQYPHECWBAH4ECUGJL2Y2LAUA2RWHX5J6XMR65NZOACBF6PA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.353464633 (21h49m25.868567211s) [noise=true , address=104.36.82.46:4487 , piece_id=JOSIX5WT5A36RXKJNWO5KLBEQOCNQCEZ7A2S4KYIBJHRKN3KT6RQ, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.354418304 (21h49m25.869520884s) [noise=true , address=104.36.82.38:4707 , piece_id=WZWQHYDMTUG2FOTZ6JJRVIXXFIPHJ3I2YIRQXMN7B2W7QGKAFY3A, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.354959848 (21h49m25.87006243s) [noise=true , address=104.36.82.47:4544 , piece_id=YDG3KOKUS5NVBWMQYJ7JJEO3PAXZKPZPUOUWUDCZ4RAVATW67NKQ, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.355461506 (21h49m25.870564086s) [noise=true , address=104.36.82.60:4363 , piece_id=QMR7CM4ASNAKG5PT2XUPFR4MIBHNFYJ7OZMASJ7Q6R2C225L655A, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.355910519 (21h49m25.871013101s) [noise=true , address=104.36.82.2:4741 , piece_id=3NWVL6DQCMDO5PQLSFU3VER7RSHGRAB6YRT32XBFUVLAXGEJQN7A, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.357467009 (21h49m25.872569585s) [noise=true , address=104.36.82.59:4322 , piece_id=NKCSGWGYWOIJWHSQUN2ZYOVZYIRML5G3DXVRJYDTAMHTEN2VES3Q, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.358628018 (21h49m25.873730598s) [noise=true , address=45.91.116.11:4120 , piece_id=274GAU5MCZAZOHB3ESPDSJP23Z5FZB636CWGN7ELS2HYY5OFZ7RA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.359105086 (21h49m25.874207666s) [noise=true , address=104.36.82.45:4907 , piece_id=ZF2SKF5O2WVNHM7CHQ7E4U7HVKQWCJGOZEVIDXO3OU4XHVUFPIGA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.360554866 (21h49m25.875657451s) [noise=true , address=104.36.82.36:4648 , piece_id=3WAWCS3XXWVX3ACYJG2OOS2R34SXOWYSHXVF6AMPFXJFCMYH4SZA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.363015621 (21h49m25.8781182s) [noise=true , address=104.36.82.62:4420 , piece_id=7S6ALI3DNOM7IURBTYY3DLALDQV7R7LLHPQCA4Q6G4HA2SOZ2YMA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.364089235 (21h49m25.87919182s) [noise=true , address=104.36.82.9:4008 , piece_id=EGIGCEJHELTHOJZWPFWNHCP672MIXAHZIQUK2WDHER77VVJ4NZZA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.467747752 (21h49m25.982850337s) [upload=22550]: Next returned nil segment
Aug 8 07:28:51.467767955 (21h49m25.982870543s) [upload=22550]: Waiting for error group managing segments...
Aug 8 07:28:51.473577577 (21h49m25.988680143s) [noise=true , address=45.91.116.7:4013 , piece_id=XG4M7HF2E5YIRJ7PVBEGZC2ADPUPAL2QL5TPUAFIYXZUP3BX2Y7A, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.478188638 (21h49m25.993291216s) [noise=true , address=104.36.82.16:4321 , piece_id=MTRDWQYPHECWBAH4ECUGJL2Y2LAUA2RWHX5J6XMR65NZOACBF6PA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.478342049 (21h49m25.993444627s) [noise=true , address=104.36.82.53:4139 , piece_id=IGOHVQJMZ3NWLCBKVPVHTVWVHDS43DABQPEMJUTB2TXAQPQOURSQ, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.478768394 (21h49m25.993870961s) [noise=true , address=104.36.82.36:4648 , piece_id=3WAWCS3XXWVX3ACYJG2OOS2R34SXOWYSHXVF6AMPFXJFCMYH4SZA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.479336172 (21h49m25.994438735s) [noise=true , address=104.36.82.59:4322 , piece_id=NKCSGWGYWOIJWHSQUN2ZYOVZYIRML5G3DXVRJYDTAMHTEN2VES3Q, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.479515883 (21h49m25.994618457s) [noise=true , address=104.36.82.58:4286 , piece_id=36LFLTD2YWV3K7LM4M2GY3JIKLUDOZVRINQT3VLIFBRR55UCNA4Q, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.480474985 (21h49m25.995577567s) [noise=true , address=104.36.82.2:4741 , piece_id=3NWVL6DQCMDO5PQLSFU3VER7RSHGRAB6YRT32XBFUVLAXGEJQN7A, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.481008628 (21h49m25.996111195s) [noise=true , address=104.36.82.38:4707 , piece_id=WZWQHYDMTUG2FOTZ6JJRVIXXFIPHJ3I2YIRQXMN7B2W7QGKAFY3A, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.481589774 (21h49m25.996692358s) [noise=true , address=104.36.82.60:4363 , piece_id=QMR7CM4ASNAKG5PT2XUPFR4MIBHNFYJ7OZMASJ7Q6R2C225L655A, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.482143054 (21h49m25.99724562s) [noise=true , address=104.36.82.46:4487 , piece_id=JOSIX5WT5A36RXKJNWO5KLBEQOCNQCEZ7A2S4KYIBJHRKN3KT6RQ, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.485172215 (21h49m26.000274779s) [noise=true , address=104.36.82.47:4544 , piece_id=YDG3KOKUS5NVBWMQYJ7JJEO3PAXZKPZPUOUWUDCZ4RAVATW67NKQ, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.486088591 (21h49m26.001191172s) [noise=true , address=104.36.82.44:4880 , piece_id=HFXFL7ULX5OTYJFEOC45DI75DGDIHAQA6ZJIXHO22X4GPSEZN75Q, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.486628345 (21h49m26.001730916s) [noise=true , address=104.36.82.23:4469 , piece_id=CI6CU5IBYK37MBIQW554EW6PPXVYA6ZF2CJPVZCIXMFFLXZ7UO7Q, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.487179958 (21h49m26.002282521s) [noise=true , address=104.36.82.62:4420 , piece_id=7S6ALI3DNOM7IURBTYY3DLALDQV7R7LLHPQCA4Q6G4HA2SOZ2YMA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.487192580 (21h49m26.002295147s) [seg_pos={0 0}, upload=22550]: Segment reached optimal threshold of 14 pieces.
Aug 8 07:28:51.487306760 (21h49m26.002409325s) [noise=true , address=104.36.82.45:4907 , piece_id=ZF2SKF5O2WVNHM7CHQ7E4U7HVKQWCJGOZEVIDXO3OU4XHVUFPIGA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12URx18Ckh7VaAKJ1hdQQucMRu5qiDrAraqaycjYqxETgwZ4uWS): protocol: expected piece hash; context canceled; EOF
Aug 8 07:28:51.487325587 (21h49m26.00242815s) [noise=true , address=104.36.82.9:4008 , piece_id=EGIGCEJHELTHOJZWPFWNHCP672MIXAHZIQUK2WDHER77VVJ4NZZA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.487748790 (21h49m26.002851367s) [noise=true , address=193.149.164.143:4693 , piece_id=RCLLY6ABOIXYK6BBO5EWKTDDJ4MUH6IP2WGIHSTER4TEPSF2FYVA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LHoCE9CnM32tbMNqAB7HpmArSVLTycv9o6kSSFNUJsoDeERkQ): protocol: read tcp 159.223.143.39:37688->193.149.164.143:4693: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 07:28:51.487858041 (21h49m26.002960604s) [noise=true , address=45.91.116.11:4120 , piece_id=274GAU5MCZAZOHB3ESPDSJP23Z5FZB636CWGN7ELS2HYY5OFZ7RA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.487890658 (21h49m26.002993237s) [seg_pos={0 0}, upload=22550]: Done waiting for segment. successful: 16 optimal: 14 errs: <nil>
Aug 8 07:28:51.487971404 (21h49m26.00307399s) [upload=22550]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 8 07:28:51.487976917 (21h49m26.003079481s) [upload=22550]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 8 07:28:51.647177827 (21h49m26.162280406s) [upload=22550]: Done uploading segments...
Aug 8 07:28:51.647194442 (21h49m26.162297008s) [upload=22550]: Upload finished. err: <nil>
Aug 8 07:29:34.528501362 (21h50m9.043603939s) [upload=22562]: Starting upload
Aug 8 07:29:34.528548565 (21h50m9.043651143s) [upload=22562]: Done starting upload
Aug 8 07:29:34.528615374 (21h50m9.043717942s) [upload=22562]: Uploading segments...
Aug 8 07:29:34.528903298 (21h50m9.044005874s) [upload=22562]: Got next segment. Inline: false
Aug 8 07:29:34.528940157 (21h50m9.044042733s) [upload=22562]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 8 07:29:34.528944789 (21h50m9.044047366s) [upload=22562]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 8 07:29:34.536881082 (21h50m9.051983672s) [upload=22562]: Next segment err: metaclient: Too Many Requests
Aug 8 07:29:34.536925024 (21h50m9.052027602s) [upload=22562]: Done uploading segments...
Aug 8 07:29:34.536943164 (21h50m9.052045741s) [upload=22562]: Upload finished. err: metaclient: Too Many Requests
Duration: 398.041184ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [.........X..............................................X...] duration:
217.877833ms
FAIL (2025-08-08 07:32:52 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== nts... Aug 8 07:24:18.405184267 (21h44m52.920286834s) [upload=22475]: Upload finished. err: <nil> Aug 8 07:24:44.148856783 (21h45m18.663959349s) [upload=22481]: Starting upload Aug 8 07:24:44.148908040 (21h45m18.664010617s) [upload=22481]: Done starting upload Aug 8 07:24:44.148978727 (21h45m18.664081294s) [upload=22481]: Uploading segments... Aug 8 07:24:44.149031432 (21h45m18.664134016s) [upload=22481]: Got next segment. Inline: true Aug 8 07:24:44.149047929 (21h45m18.664150495s) [upload=22481]: Waiting for error group managing segments... Aug 8 07:24:44.149231742 (21h45m18.664334308s) [upload=22481]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:24:44.149238484 (21h45m18.664341061s) [upload=22481]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:24:44.149241695 (21h45m18.664344271s) [upload=22481]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:24:44.236175458 (21h45m18.751278039s) [upload=22481]: Done uploading segments... Aug 8 07:24:44.236197562 (21h45m18.75130015s) [upload=22481]: Upload finished. err: <nil> Aug 8 07:25:45.798907959 (21h46m20.31401054s) [upload=22496]: Starting upload Aug 8 07:25:45.798982852 (21h46m20.314085432s) [upload=22496]: Done starting upload Aug 8 07:25:45.799141617 (21h46m20.314244194s) [upload=22496]: Uploading segments... Aug 8 07:25:45.799187034 (21h46m20.314289621s) [upload=22496]: Got next segment. Inline: true Aug 8 07:25:45.799192794 (21h46m20.314295378s) [upload=22496]: Waiting for error group managing segments... Aug 8 07:25:45.799260506 (21h46m20.314363086s) [upload=22496]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:25:45.799265543 (21h46m20.314368123s) [upload=22496]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:25:45.799269614 (21h46m20.314372195s) [upload=22496]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:25:45.856873466 (21h46m20.371976033s) [upload=22496]: Done uploading segments... Aug 8 07:25:45.856894517 (21h46m20.371997083s) [upload=22496]: Upload finished. err: <nil> Aug 8 07:26:45.160306388 (21h47m19.67540897s) [upload=22512]: Starting upload Aug 8 07:26:45.160369840 (21h47m19.675472407s) [upload=22512]: Done starting upload Aug 8 07:26:45.160448308 (21h47m19.675550874s) [upload=22512]: Uploading segments... Aug 8 07:26:45.160512759 (21h47m19.675615339s) [upload=22512]: Got next segment. Inline: true Aug 8 07:26:45.160518821 (21h47m19.675621389s) [upload=22512]: Waiting for error group managing segments... Aug 8 07:26:45.160566486 (21h47m19.675669054s) [upload=22512]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:26:45.160571363 (21h47m19.675673943s) [upload=22512]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:26:45.160575116 (21h47m19.675677698s) [upload=22512]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:26:45.219521344 (21h47m19.73462392s) [upload=22512]: Done uploading segments... Aug 8 07:26:45.219537755 (21h47m19.73464032s) [upload=22512]: Upload finished. err: <nil> Aug 8 07:27:50.008745364 (21h48m24.523847971s) [upload=22529]: Starting upload Aug 8 07:27:50.008828662 (21h48m24.523931252s) [upload=22529]: Done starting upload Aug 8 07:27:50.008913720 (21h48m24.524016289s) [upload=22529]: Uploading segments... Aug 8 07:27:50.008968901 (21h48m24.524071486s) [upload=22529]: Got next segment. Inline: true Aug 8 07:27:50.008973893 (21h48m24.524076469s) [upload=22529]: Waiting for error group managing segments... Aug 8 07:27:50.009023439 (21h48m24.524126015s) [upload=22529]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:27:50.009028404 (21h48m24.524130983s) [upload=22529]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:27:50.009032740 (21h48m24.524135324s) [upload=22529]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:27:50.072222330 (21h48m24.587324909s) [upload=22529]: Done uploading segments... Aug 8 07:27:50.072260086 (21h48m24.587362652s) [upload=22529]: Upload finished. err: <nil> Aug 8 07:28:08.797868182 (21h48m43.312970759s) [upload=22537]: Starting upload Aug 8 07:28:08.797909617 (21h48m43.313012202s) [upload=22537]: Done starting upload Aug 8 07:28:08.797978107 (21h48m43.313080676s) [upload=22537]: Uploading segments... Aug 8 07:28:08.798028549 (21h48m43.313131134s) [upload=22537]: Got next segment. Inline: true Aug 8 07:28:08.798033746 (21h48m43.313136324s) [upload=22537]: Waiting for error group managing segments... Aug 8 07:28:08.798088133 (21h48m43.313190709s) [upload=22537]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:28:08.798092673 (21h48m43.313195249s) [upload=22537]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:28:08.798096028 (21h48m43.313198604s) [upload=22537]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:28:08.856181519 (21h48m43.371284096s) [upload=22537]: Done uploading segments... Aug 8 07:28:08.856219288 (21h48m43.371321856s) [upload=22537]: Upload finished. err: <nil> Aug 8 07:28:26.426406302 (21h49m0.941508883s) [upload=22543]: Starting upload Aug 8 07:28:26.426484520 (21h49m0.94158709s) [upload=22543]: Done starting upload Aug 8 07:28:26.426561912 (21h49m0.941664479s) [upload=22543]: Uploading segments... Aug 8 07:28:26.426613562 (21h49m0.941716147s) [upload=22543]: Got next segment. Inline: true Aug 8 07:28:26.426618334 (21h49m0.94172091s) [upload=22543]: Waiting for error group managing segments... Aug 8 07:28:26.426667358 (21h49m0.941769934s) [upload=22543]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:28:26.426672477 (21h49m0.941775052s) [upload=22543]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:28:26.426676980 (21h49m0.941779557s) [upload=22543]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:28:26.739786156 (21h49m1.254888734s) [upload=22543]: Done uploading segments... Aug 8 07:28:26.739818088 (21h49m1.254920663s) [upload=22543]: Upload finished. err: <nil> Aug 8 07:28:44.768272418 (21h49m19.283374995s) [upload=22547]: Starting upload Aug 8 07:28:44.768352182 (21h49m19.28345476s) [upload=22547]: Done starting upload Aug 8 07:28:44.768463680 (21h49m19.283566246s) [upload=22547]: Uploading segments... Aug 8 07:28:44.768552166 (21h49m19.283654733s) [upload=22547]: Got next segment. Inline: true Aug 8 07:28:44.768558003 (21h49m19.28366057s) [upload=22547]: Waiting for error group managing segments... Aug 8 07:28:44.768617460 (21h49m19.28372004s) [upload=22547]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:28:44.768622292 (21h49m19.283724868s) [upload=22547]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:28:44.768625505 (21h49m19.283728082s) [upload=22547]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:28:44.832641596 (21h49m19.347744163s) [upload=22547]: Done uploading segments... Aug 8 07:28:44.832661990 (21h49m19.347764559s) [upload=22547]: Upload finished. err: <nil> Aug 8 07:29:19.907807846 (21h49m54.422910428s) [upload=22558]: Starting upload Aug 8 07:29:19.907919209 (21h49m54.423021801s) [upload=22558]: Done starting upload Aug 8 07:29:19.908035700 (21h49m54.423138281s) [upload=22558]: Uploading segments... Aug 8 07:29:19.908115743 (21h49m54.423218334s) [upload=22558]: Got next segment. Inline: true Aug 8 07:29:19.908123216 (21h49m54.423225802s) [upload=22558]: Waiting for error group managing segments... Aug 8 07:29:19.908206488 (21h49m54.423309073s) [upload=22558]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:29:19.908213079 (21h49m54.423315659s) [upload=22558]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:29:19.908216828 (21h49m54.423319408s) [upload=22558]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:29:19.968416399 (21h49m54.483518987s) [upload=22558]: Done uploading segments... Aug 8 07:29:19.968437892 (21h49m54.48354047s) [upload=22558]: Upload finished. err: <nil> Aug 8 07:30:04.570811298 (21h50m39.085913875s) [upload=22574]: Starting upload Aug 8 07:30:04.570856561 (21h50m39.085959129s) [upload=22574]: Done starting upload Aug 8 07:30:04.570937395 (21h50m39.086039963s) [upload=22574]: Uploading segments... Aug 8 07:30:04.570993290 (21h50m39.086095879s) [upload=22574]: Got next segment. Inline: true Aug 8 07:30:04.570999714 (21h50m39.086102306s) [upload=22574]: Waiting for error group managing segments... Aug 8 07:30:04.571094735 (21h50m39.086197316s) [upload=22574]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:30:04.571101887 (21h50m39.086204467s) [upload=22574]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:30:04.571105790 (21h50m39.086208372s) [upload=22574]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:30:04.649336948 (21h50m39.164439536s) [upload=22574]: Done uploading segments... Aug 8 07:30:04.649362004 (21h50m39.164464585s) [upload=22574]: Upload finished. err: <nil> Aug 8 07:30:54.851564431 (21h51m29.366666997s) [upload=22587]: Starting upload Aug 8 07:30:54.851618044 (21h51m29.36672062s) [upload=22587]: Done starting upload Aug 8 07:30:54.851712059 (21h51m29.366814635s) [upload=22587]: Uploading segments... Aug 8 07:30:54.851783178 (21h51m29.36688576s) [upload=22587]: Got next segment. Inline: true Aug 8 07:30:54.851789517 (21h51m29.366892093s) [upload=22587]: Waiting for error group managing segments... Aug 8 07:30:54.851854337 (21h51m29.366956924s) [upload=22587]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:30:54.851861334 (21h51m29.366963924s) [upload=22587]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:30:54.851866059 (21h51m29.366968642s) [upload=22587]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:30:54.860593007 (21h51m29.375695595s) [upload=22587]: Done uploading segments... Aug 8 07:30:54.860627519 (21h51m29.375730095s) [upload=22587]: Upload finished. err: metaclient: Too Many Requests
Duration: 217.901449ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...] duration:
386.869293ms
FAIL (2025-08-08 07:32:41 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
g_pos={0 0}, upload=22550]: Begin upload segment...
Aug 8 07:28:51.351770179 (21h49m25.866872766s) [seg_pos={0 0}, upload=22550]: Done begin upload segment.
Aug 8 07:28:51.351799155 (21h49m25.866901747s) [noise=true , address=104.36.82.23:4469 , piece_id=CI6CU5IBYK37MBIQW554EW6PPXVYA6ZF2CJPVZCIXMFFLXZ7UO7Q, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.351816053 (21h49m25.86691864s) [noise=true , address=193.149.164.143:4693 , piece_id=RCLLY6ABOIXYK6BBO5EWKTDDJ4MUH6IP2WGIHSTER4TEPSF2FYVA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.352227136 (21h49m25.867329713s) [noise=true , address=104.36.82.58:4286 , piece_id=36LFLTD2YWV3K7LM4M2GY3JIKLUDOZVRINQT3VLIFBRR55UCNA4Q, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.352322195 (21h49m25.867424772s) [noise=true , address=104.36.82.44:4880 , piece_id=HFXFL7ULX5OTYJFEOC45DI75DGDIHAQA6ZJIXHO22X4GPSEZN75Q, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.352611348 (21h49m25.867713924s) [noise=true , address=104.36.82.53:4139 , piece_id=IGOHVQJMZ3NWLCBKVPVHTVWVHDS43DABQPEMJUTB2TXAQPQOURSQ, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.352702579 (21h49m25.867805142s) [noise=true , address=45.91.116.7:4013 , piece_id=XG4M7HF2E5YIRJ7PVBEGZC2ADPUPAL2QL5TPUAFIYXZUP3BX2Y7A, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.353022021 (21h49m25.868124597s) [noise=true , address=104.36.82.16:4321 , piece_id=MTRDWQYPHECWBAH4ECUGJL2Y2LAUA2RWHX5J6XMR65NZOACBF6PA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.353464633 (21h49m25.868567211s) [noise=true , address=104.36.82.46:4487 , piece_id=JOSIX5WT5A36RXKJNWO5KLBEQOCNQCEZ7A2S4KYIBJHRKN3KT6RQ, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.354418304 (21h49m25.869520884s) [noise=true , address=104.36.82.38:4707 , piece_id=WZWQHYDMTUG2FOTZ6JJRVIXXFIPHJ3I2YIRQXMN7B2W7QGKAFY3A, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.354959848 (21h49m25.87006243s) [noise=true , address=104.36.82.47:4544 , piece_id=YDG3KOKUS5NVBWMQYJ7JJEO3PAXZKPZPUOUWUDCZ4RAVATW67NKQ, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.355461506 (21h49m25.870564086s) [noise=true , address=104.36.82.60:4363 , piece_id=QMR7CM4ASNAKG5PT2XUPFR4MIBHNFYJ7OZMASJ7Q6R2C225L655A, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.355910519 (21h49m25.871013101s) [noise=true , address=104.36.82.2:4741 , piece_id=3NWVL6DQCMDO5PQLSFU3VER7RSHGRAB6YRT32XBFUVLAXGEJQN7A, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.357467009 (21h49m25.872569585s) [noise=true , address=104.36.82.59:4322 , piece_id=NKCSGWGYWOIJWHSQUN2ZYOVZYIRML5G3DXVRJYDTAMHTEN2VES3Q, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.358628018 (21h49m25.873730598s) [noise=true , address=45.91.116.11:4120 , piece_id=274GAU5MCZAZOHB3ESPDSJP23Z5FZB636CWGN7ELS2HYY5OFZ7RA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.359105086 (21h49m25.874207666s) [noise=true , address=104.36.82.45:4907 , piece_id=ZF2SKF5O2WVNHM7CHQ7E4U7HVKQWCJGOZEVIDXO3OU4XHVUFPIGA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.360554866 (21h49m25.875657451s) [noise=true , address=104.36.82.36:4648 , piece_id=3WAWCS3XXWVX3ACYJG2OOS2R34SXOWYSHXVF6AMPFXJFCMYH4SZA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.363015621 (21h49m25.8781182s) [noise=true , address=104.36.82.62:4420 , piece_id=7S6ALI3DNOM7IURBTYY3DLALDQV7R7LLHPQCA4Q6G4HA2SOZ2YMA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.364089235 (21h49m25.87919182s) [noise=true , address=104.36.82.9:4008 , piece_id=EGIGCEJHELTHOJZWPFWNHCP672MIXAHZIQUK2WDHER77VVJ4NZZA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.467747752 (21h49m25.982850337s) [upload=22550]: Next returned nil segment
Aug 8 07:28:51.467767955 (21h49m25.982870543s) [upload=22550]: Waiting for error group managing segments...
Aug 8 07:28:51.473577577 (21h49m25.988680143s) [noise=true , address=45.91.116.7:4013 , piece_id=XG4M7HF2E5YIRJ7PVBEGZC2ADPUPAL2QL5TPUAFIYXZUP3BX2Y7A, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.478188638 (21h49m25.993291216s) [noise=true , address=104.36.82.16:4321 , piece_id=MTRDWQYPHECWBAH4ECUGJL2Y2LAUA2RWHX5J6XMR65NZOACBF6PA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.478342049 (21h49m25.993444627s) [noise=true , address=104.36.82.53:4139 , piece_id=IGOHVQJMZ3NWLCBKVPVHTVWVHDS43DABQPEMJUTB2TXAQPQOURSQ, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.478768394 (21h49m25.993870961s) [noise=true , address=104.36.82.36:4648 , piece_id=3WAWCS3XXWVX3ACYJG2OOS2R34SXOWYSHXVF6AMPFXJFCMYH4SZA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.479336172 (21h49m25.994438735s) [noise=true , address=104.36.82.59:4322 , piece_id=NKCSGWGYWOIJWHSQUN2ZYOVZYIRML5G3DXVRJYDTAMHTEN2VES3Q, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.479515883 (21h49m25.994618457s) [noise=true , address=104.36.82.58:4286 , piece_id=36LFLTD2YWV3K7LM4M2GY3JIKLUDOZVRINQT3VLIFBRR55UCNA4Q, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.480474985 (21h49m25.995577567s) [noise=true , address=104.36.82.2:4741 , piece_id=3NWVL6DQCMDO5PQLSFU3VER7RSHGRAB6YRT32XBFUVLAXGEJQN7A, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.481008628 (21h49m25.996111195s) [noise=true , address=104.36.82.38:4707 , piece_id=WZWQHYDMTUG2FOTZ6JJRVIXXFIPHJ3I2YIRQXMN7B2W7QGKAFY3A, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.481589774 (21h49m25.996692358s) [noise=true , address=104.36.82.60:4363 , piece_id=QMR7CM4ASNAKG5PT2XUPFR4MIBHNFYJ7OZMASJ7Q6R2C225L655A, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.482143054 (21h49m25.99724562s) [noise=true , address=104.36.82.46:4487 , piece_id=JOSIX5WT5A36RXKJNWO5KLBEQOCNQCEZ7A2S4KYIBJHRKN3KT6RQ, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.485172215 (21h49m26.000274779s) [noise=true , address=104.36.82.47:4544 , piece_id=YDG3KOKUS5NVBWMQYJ7JJEO3PAXZKPZPUOUWUDCZ4RAVATW67NKQ, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.486088591 (21h49m26.001191172s) [noise=true , address=104.36.82.44:4880 , piece_id=HFXFL7ULX5OTYJFEOC45DI75DGDIHAQA6ZJIXHO22X4GPSEZN75Q, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.486628345 (21h49m26.001730916s) [noise=true , address=104.36.82.23:4469 , piece_id=CI6CU5IBYK37MBIQW554EW6PPXVYA6ZF2CJPVZCIXMFFLXZ7UO7Q, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.487179958 (21h49m26.002282521s) [noise=true , address=104.36.82.62:4420 , piece_id=7S6ALI3DNOM7IURBTYY3DLALDQV7R7LLHPQCA4Q6G4HA2SOZ2YMA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.487192580 (21h49m26.002295147s) [seg_pos={0 0}, upload=22550]: Segment reached optimal threshold of 14 pieces.
Aug 8 07:28:51.487306760 (21h49m26.002409325s) [noise=true , address=104.36.82.45:4907 , piece_id=ZF2SKF5O2WVNHM7CHQ7E4U7HVKQWCJGOZEVIDXO3OU4XHVUFPIGA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12URx18Ckh7VaAKJ1hdQQucMRu5qiDrAraqaycjYqxETgwZ4uWS): protocol: expected piece hash; context canceled; EOF
Aug 8 07:28:51.487325587 (21h49m26.00242815s) [noise=true , address=104.36.82.9:4008 , piece_id=EGIGCEJHELTHOJZWPFWNHCP672MIXAHZIQUK2WDHER77VVJ4NZZA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.487748790 (21h49m26.002851367s) [noise=true , address=193.149.164.143:4693 , piece_id=RCLLY6ABOIXYK6BBO5EWKTDDJ4MUH6IP2WGIHSTER4TEPSF2FYVA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LHoCE9CnM32tbMNqAB7HpmArSVLTycv9o6kSSFNUJsoDeERkQ): protocol: read tcp 159.223.143.39:37688->193.149.164.143:4693: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 07:28:51.487858041 (21h49m26.002960604s) [noise=true , address=45.91.116.11:4120 , piece_id=274GAU5MCZAZOHB3ESPDSJP23Z5FZB636CWGN7ELS2HYY5OFZ7RA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.487890658 (21h49m26.002993237s) [seg_pos={0 0}, upload=22550]: Done waiting for segment. successful: 16 optimal: 14 errs: <nil>
Aug 8 07:28:51.487971404 (21h49m26.00307399s) [upload=22550]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 8 07:28:51.487976917 (21h49m26.003079481s) [upload=22550]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 8 07:28:51.647177827 (21h49m26.162280406s) [upload=22550]: Done uploading segments...
Aug 8 07:28:51.647194442 (21h49m26.162297008s) [upload=22550]: Upload finished. err: <nil>
Aug 8 07:29:34.528501362 (21h50m9.043603939s) [upload=22562]: Starting upload
Aug 8 07:29:34.528548565 (21h50m9.043651143s) [upload=22562]: Done starting upload
Aug 8 07:29:34.528615374 (21h50m9.043717942s) [upload=22562]: Uploading segments...
Aug 8 07:29:34.528903298 (21h50m9.044005874s) [upload=22562]: Got next segment. Inline: false
Aug 8 07:29:34.528940157 (21h50m9.044042733s) [upload=22562]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 8 07:29:34.528944789 (21h50m9.044047366s) [upload=22562]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 8 07:29:34.536881082 (21h50m9.051983672s) [upload=22562]: Next segment err: metaclient: Too Many Requests
Aug 8 07:29:34.536925024 (21h50m9.052027602s) [upload=22562]: Done uploading segments...
Aug 8 07:29:34.536943164 (21h50m9.052045741s) [upload=22562]: Upload finished. err: metaclient: Too Many Requests
Duration: 386.89698ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.] duration:
31.029447ms target:
https://us1.storj.io/login
FAIL (2025-08-08 07:32:40 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.150.199.48:443: connect: connection refused
Duration: 31.06104ms
core
satellite
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [..........X..............................................X..] duration:
553.316382ms
FAIL (2025-08-08 07:32:18 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== nts... Aug 8 07:24:18.405184267 (21h44m52.920286834s) [upload=22475]: Upload finished. err: <nil> Aug 8 07:24:44.148856783 (21h45m18.663959349s) [upload=22481]: Starting upload Aug 8 07:24:44.148908040 (21h45m18.664010617s) [upload=22481]: Done starting upload Aug 8 07:24:44.148978727 (21h45m18.664081294s) [upload=22481]: Uploading segments... Aug 8 07:24:44.149031432 (21h45m18.664134016s) [upload=22481]: Got next segment. Inline: true Aug 8 07:24:44.149047929 (21h45m18.664150495s) [upload=22481]: Waiting for error group managing segments... Aug 8 07:24:44.149231742 (21h45m18.664334308s) [upload=22481]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:24:44.149238484 (21h45m18.664341061s) [upload=22481]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:24:44.149241695 (21h45m18.664344271s) [upload=22481]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:24:44.236175458 (21h45m18.751278039s) [upload=22481]: Done uploading segments... Aug 8 07:24:44.236197562 (21h45m18.75130015s) [upload=22481]: Upload finished. err: <nil> Aug 8 07:25:45.798907959 (21h46m20.31401054s) [upload=22496]: Starting upload Aug 8 07:25:45.798982852 (21h46m20.314085432s) [upload=22496]: Done starting upload Aug 8 07:25:45.799141617 (21h46m20.314244194s) [upload=22496]: Uploading segments... Aug 8 07:25:45.799187034 (21h46m20.314289621s) [upload=22496]: Got next segment. Inline: true Aug 8 07:25:45.799192794 (21h46m20.314295378s) [upload=22496]: Waiting for error group managing segments... Aug 8 07:25:45.799260506 (21h46m20.314363086s) [upload=22496]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:25:45.799265543 (21h46m20.314368123s) [upload=22496]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:25:45.799269614 (21h46m20.314372195s) [upload=22496]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:25:45.856873466 (21h46m20.371976033s) [upload=22496]: Done uploading segments... Aug 8 07:25:45.856894517 (21h46m20.371997083s) [upload=22496]: Upload finished. err: <nil> Aug 8 07:26:45.160306388 (21h47m19.67540897s) [upload=22512]: Starting upload Aug 8 07:26:45.160369840 (21h47m19.675472407s) [upload=22512]: Done starting upload Aug 8 07:26:45.160448308 (21h47m19.675550874s) [upload=22512]: Uploading segments... Aug 8 07:26:45.160512759 (21h47m19.675615339s) [upload=22512]: Got next segment. Inline: true Aug 8 07:26:45.160518821 (21h47m19.675621389s) [upload=22512]: Waiting for error group managing segments... Aug 8 07:26:45.160566486 (21h47m19.675669054s) [upload=22512]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:26:45.160571363 (21h47m19.675673943s) [upload=22512]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:26:45.160575116 (21h47m19.675677698s) [upload=22512]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:26:45.219521344 (21h47m19.73462392s) [upload=22512]: Done uploading segments... Aug 8 07:26:45.219537755 (21h47m19.73464032s) [upload=22512]: Upload finished. err: <nil> Aug 8 07:27:50.008745364 (21h48m24.523847971s) [upload=22529]: Starting upload Aug 8 07:27:50.008828662 (21h48m24.523931252s) [upload=22529]: Done starting upload Aug 8 07:27:50.008913720 (21h48m24.524016289s) [upload=22529]: Uploading segments... Aug 8 07:27:50.008968901 (21h48m24.524071486s) [upload=22529]: Got next segment. Inline: true Aug 8 07:27:50.008973893 (21h48m24.524076469s) [upload=22529]: Waiting for error group managing segments... Aug 8 07:27:50.009023439 (21h48m24.524126015s) [upload=22529]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:27:50.009028404 (21h48m24.524130983s) [upload=22529]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:27:50.009032740 (21h48m24.524135324s) [upload=22529]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:27:50.072222330 (21h48m24.587324909s) [upload=22529]: Done uploading segments... Aug 8 07:27:50.072260086 (21h48m24.587362652s) [upload=22529]: Upload finished. err: <nil> Aug 8 07:28:08.797868182 (21h48m43.312970759s) [upload=22537]: Starting upload Aug 8 07:28:08.797909617 (21h48m43.313012202s) [upload=22537]: Done starting upload Aug 8 07:28:08.797978107 (21h48m43.313080676s) [upload=22537]: Uploading segments... Aug 8 07:28:08.798028549 (21h48m43.313131134s) [upload=22537]: Got next segment. Inline: true Aug 8 07:28:08.798033746 (21h48m43.313136324s) [upload=22537]: Waiting for error group managing segments... Aug 8 07:28:08.798088133 (21h48m43.313190709s) [upload=22537]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:28:08.798092673 (21h48m43.313195249s) [upload=22537]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:28:08.798096028 (21h48m43.313198604s) [upload=22537]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:28:08.856181519 (21h48m43.371284096s) [upload=22537]: Done uploading segments... Aug 8 07:28:08.856219288 (21h48m43.371321856s) [upload=22537]: Upload finished. err: <nil> Aug 8 07:28:26.426406302 (21h49m0.941508883s) [upload=22543]: Starting upload Aug 8 07:28:26.426484520 (21h49m0.94158709s) [upload=22543]: Done starting upload Aug 8 07:28:26.426561912 (21h49m0.941664479s) [upload=22543]: Uploading segments... Aug 8 07:28:26.426613562 (21h49m0.941716147s) [upload=22543]: Got next segment. Inline: true Aug 8 07:28:26.426618334 (21h49m0.94172091s) [upload=22543]: Waiting for error group managing segments... Aug 8 07:28:26.426667358 (21h49m0.941769934s) [upload=22543]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:28:26.426672477 (21h49m0.941775052s) [upload=22543]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:28:26.426676980 (21h49m0.941779557s) [upload=22543]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:28:26.739786156 (21h49m1.254888734s) [upload=22543]: Done uploading segments... Aug 8 07:28:26.739818088 (21h49m1.254920663s) [upload=22543]: Upload finished. err: <nil> Aug 8 07:28:44.768272418 (21h49m19.283374995s) [upload=22547]: Starting upload Aug 8 07:28:44.768352182 (21h49m19.28345476s) [upload=22547]: Done starting upload Aug 8 07:28:44.768463680 (21h49m19.283566246s) [upload=22547]: Uploading segments... Aug 8 07:28:44.768552166 (21h49m19.283654733s) [upload=22547]: Got next segment. Inline: true Aug 8 07:28:44.768558003 (21h49m19.28366057s) [upload=22547]: Waiting for error group managing segments... Aug 8 07:28:44.768617460 (21h49m19.28372004s) [upload=22547]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:28:44.768622292 (21h49m19.283724868s) [upload=22547]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:28:44.768625505 (21h49m19.283728082s) [upload=22547]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:28:44.832641596 (21h49m19.347744163s) [upload=22547]: Done uploading segments... Aug 8 07:28:44.832661990 (21h49m19.347764559s) [upload=22547]: Upload finished. err: <nil> Aug 8 07:29:19.907807846 (21h49m54.422910428s) [upload=22558]: Starting upload Aug 8 07:29:19.907919209 (21h49m54.423021801s) [upload=22558]: Done starting upload Aug 8 07:29:19.908035700 (21h49m54.423138281s) [upload=22558]: Uploading segments... Aug 8 07:29:19.908115743 (21h49m54.423218334s) [upload=22558]: Got next segment. Inline: true Aug 8 07:29:19.908123216 (21h49m54.423225802s) [upload=22558]: Waiting for error group managing segments... Aug 8 07:29:19.908206488 (21h49m54.423309073s) [upload=22558]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:29:19.908213079 (21h49m54.423315659s) [upload=22558]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:29:19.908216828 (21h49m54.423319408s) [upload=22558]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:29:19.968416399 (21h49m54.483518987s) [upload=22558]: Done uploading segments... Aug 8 07:29:19.968437892 (21h49m54.48354047s) [upload=22558]: Upload finished. err: <nil> Aug 8 07:30:04.570811298 (21h50m39.085913875s) [upload=22574]: Starting upload Aug 8 07:30:04.570856561 (21h50m39.085959129s) [upload=22574]: Done starting upload Aug 8 07:30:04.570937395 (21h50m39.086039963s) [upload=22574]: Uploading segments... Aug 8 07:30:04.570993290 (21h50m39.086095879s) [upload=22574]: Got next segment. Inline: true Aug 8 07:30:04.570999714 (21h50m39.086102306s) [upload=22574]: Waiting for error group managing segments... Aug 8 07:30:04.571094735 (21h50m39.086197316s) [upload=22574]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:30:04.571101887 (21h50m39.086204467s) [upload=22574]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:30:04.571105790 (21h50m39.086208372s) [upload=22574]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:30:04.649336948 (21h50m39.164439536s) [upload=22574]: Done uploading segments... Aug 8 07:30:04.649362004 (21h50m39.164464585s) [upload=22574]: Upload finished. err: <nil> Aug 8 07:30:54.851564431 (21h51m29.366666997s) [upload=22587]: Starting upload Aug 8 07:30:54.851618044 (21h51m29.36672062s) [upload=22587]: Done starting upload Aug 8 07:30:54.851712059 (21h51m29.366814635s) [upload=22587]: Uploading segments... Aug 8 07:30:54.851783178 (21h51m29.36688576s) [upload=22587]: Got next segment. Inline: true Aug 8 07:30:54.851789517 (21h51m29.366892093s) [upload=22587]: Waiting for error group managing segments... Aug 8 07:30:54.851854337 (21h51m29.366956924s) [upload=22587]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:30:54.851861334 (21h51m29.366963924s) [upload=22587]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:30:54.851866059 (21h51m29.366968642s) [upload=22587]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:30:54.860593007 (21h51m29.375695595s) [upload=22587]: Done uploading segments... Aug 8 07:30:54.860627519 (21h51m29.375730095s) [upload=22587]: Upload finished. err: metaclient: Too Many Requests
Duration: 553.350247ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..] duration:
795.025458ms
FAIL (2025-08-08 07:31:54 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
g_pos={0 0}, upload=22550]: Begin upload segment...
Aug 8 07:28:51.351770179 (21h49m25.866872766s) [seg_pos={0 0}, upload=22550]: Done begin upload segment.
Aug 8 07:28:51.351799155 (21h49m25.866901747s) [noise=true , address=104.36.82.23:4469 , piece_id=CI6CU5IBYK37MBIQW554EW6PPXVYA6ZF2CJPVZCIXMFFLXZ7UO7Q, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.351816053 (21h49m25.86691864s) [noise=true , address=193.149.164.143:4693 , piece_id=RCLLY6ABOIXYK6BBO5EWKTDDJ4MUH6IP2WGIHSTER4TEPSF2FYVA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.352227136 (21h49m25.867329713s) [noise=true , address=104.36.82.58:4286 , piece_id=36LFLTD2YWV3K7LM4M2GY3JIKLUDOZVRINQT3VLIFBRR55UCNA4Q, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.352322195 (21h49m25.867424772s) [noise=true , address=104.36.82.44:4880 , piece_id=HFXFL7ULX5OTYJFEOC45DI75DGDIHAQA6ZJIXHO22X4GPSEZN75Q, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.352611348 (21h49m25.867713924s) [noise=true , address=104.36.82.53:4139 , piece_id=IGOHVQJMZ3NWLCBKVPVHTVWVHDS43DABQPEMJUTB2TXAQPQOURSQ, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.352702579 (21h49m25.867805142s) [noise=true , address=45.91.116.7:4013 , piece_id=XG4M7HF2E5YIRJ7PVBEGZC2ADPUPAL2QL5TPUAFIYXZUP3BX2Y7A, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.353022021 (21h49m25.868124597s) [noise=true , address=104.36.82.16:4321 , piece_id=MTRDWQYPHECWBAH4ECUGJL2Y2LAUA2RWHX5J6XMR65NZOACBF6PA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.353464633 (21h49m25.868567211s) [noise=true , address=104.36.82.46:4487 , piece_id=JOSIX5WT5A36RXKJNWO5KLBEQOCNQCEZ7A2S4KYIBJHRKN3KT6RQ, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.354418304 (21h49m25.869520884s) [noise=true , address=104.36.82.38:4707 , piece_id=WZWQHYDMTUG2FOTZ6JJRVIXXFIPHJ3I2YIRQXMN7B2W7QGKAFY3A, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.354959848 (21h49m25.87006243s) [noise=true , address=104.36.82.47:4544 , piece_id=YDG3KOKUS5NVBWMQYJ7JJEO3PAXZKPZPUOUWUDCZ4RAVATW67NKQ, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.355461506 (21h49m25.870564086s) [noise=true , address=104.36.82.60:4363 , piece_id=QMR7CM4ASNAKG5PT2XUPFR4MIBHNFYJ7OZMASJ7Q6R2C225L655A, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.355910519 (21h49m25.871013101s) [noise=true , address=104.36.82.2:4741 , piece_id=3NWVL6DQCMDO5PQLSFU3VER7RSHGRAB6YRT32XBFUVLAXGEJQN7A, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.357467009 (21h49m25.872569585s) [noise=true , address=104.36.82.59:4322 , piece_id=NKCSGWGYWOIJWHSQUN2ZYOVZYIRML5G3DXVRJYDTAMHTEN2VES3Q, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.358628018 (21h49m25.873730598s) [noise=true , address=45.91.116.11:4120 , piece_id=274GAU5MCZAZOHB3ESPDSJP23Z5FZB636CWGN7ELS2HYY5OFZ7RA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.359105086 (21h49m25.874207666s) [noise=true , address=104.36.82.45:4907 , piece_id=ZF2SKF5O2WVNHM7CHQ7E4U7HVKQWCJGOZEVIDXO3OU4XHVUFPIGA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.360554866 (21h49m25.875657451s) [noise=true , address=104.36.82.36:4648 , piece_id=3WAWCS3XXWVX3ACYJG2OOS2R34SXOWYSHXVF6AMPFXJFCMYH4SZA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.363015621 (21h49m25.8781182s) [noise=true , address=104.36.82.62:4420 , piece_id=7S6ALI3DNOM7IURBTYY3DLALDQV7R7LLHPQCA4Q6G4HA2SOZ2YMA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.364089235 (21h49m25.87919182s) [noise=true , address=104.36.82.9:4008 , piece_id=EGIGCEJHELTHOJZWPFWNHCP672MIXAHZIQUK2WDHER77VVJ4NZZA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.467747752 (21h49m25.982850337s) [upload=22550]: Next returned nil segment
Aug 8 07:28:51.467767955 (21h49m25.982870543s) [upload=22550]: Waiting for error group managing segments...
Aug 8 07:28:51.473577577 (21h49m25.988680143s) [noise=true , address=45.91.116.7:4013 , piece_id=XG4M7HF2E5YIRJ7PVBEGZC2ADPUPAL2QL5TPUAFIYXZUP3BX2Y7A, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.478188638 (21h49m25.993291216s) [noise=true , address=104.36.82.16:4321 , piece_id=MTRDWQYPHECWBAH4ECUGJL2Y2LAUA2RWHX5J6XMR65NZOACBF6PA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.478342049 (21h49m25.993444627s) [noise=true , address=104.36.82.53:4139 , piece_id=IGOHVQJMZ3NWLCBKVPVHTVWVHDS43DABQPEMJUTB2TXAQPQOURSQ, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.478768394 (21h49m25.993870961s) [noise=true , address=104.36.82.36:4648 , piece_id=3WAWCS3XXWVX3ACYJG2OOS2R34SXOWYSHXVF6AMPFXJFCMYH4SZA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.479336172 (21h49m25.994438735s) [noise=true , address=104.36.82.59:4322 , piece_id=NKCSGWGYWOIJWHSQUN2ZYOVZYIRML5G3DXVRJYDTAMHTEN2VES3Q, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.479515883 (21h49m25.994618457s) [noise=true , address=104.36.82.58:4286 , piece_id=36LFLTD2YWV3K7LM4M2GY3JIKLUDOZVRINQT3VLIFBRR55UCNA4Q, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.480474985 (21h49m25.995577567s) [noise=true , address=104.36.82.2:4741 , piece_id=3NWVL6DQCMDO5PQLSFU3VER7RSHGRAB6YRT32XBFUVLAXGEJQN7A, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.481008628 (21h49m25.996111195s) [noise=true , address=104.36.82.38:4707 , piece_id=WZWQHYDMTUG2FOTZ6JJRVIXXFIPHJ3I2YIRQXMN7B2W7QGKAFY3A, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.481589774 (21h49m25.996692358s) [noise=true , address=104.36.82.60:4363 , piece_id=QMR7CM4ASNAKG5PT2XUPFR4MIBHNFYJ7OZMASJ7Q6R2C225L655A, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.482143054 (21h49m25.99724562s) [noise=true , address=104.36.82.46:4487 , piece_id=JOSIX5WT5A36RXKJNWO5KLBEQOCNQCEZ7A2S4KYIBJHRKN3KT6RQ, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.485172215 (21h49m26.000274779s) [noise=true , address=104.36.82.47:4544 , piece_id=YDG3KOKUS5NVBWMQYJ7JJEO3PAXZKPZPUOUWUDCZ4RAVATW67NKQ, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.486088591 (21h49m26.001191172s) [noise=true , address=104.36.82.44:4880 , piece_id=HFXFL7ULX5OTYJFEOC45DI75DGDIHAQA6ZJIXHO22X4GPSEZN75Q, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.486628345 (21h49m26.001730916s) [noise=true , address=104.36.82.23:4469 , piece_id=CI6CU5IBYK37MBIQW554EW6PPXVYA6ZF2CJPVZCIXMFFLXZ7UO7Q, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.487179958 (21h49m26.002282521s) [noise=true , address=104.36.82.62:4420 , piece_id=7S6ALI3DNOM7IURBTYY3DLALDQV7R7LLHPQCA4Q6G4HA2SOZ2YMA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.487192580 (21h49m26.002295147s) [seg_pos={0 0}, upload=22550]: Segment reached optimal threshold of 14 pieces.
Aug 8 07:28:51.487306760 (21h49m26.002409325s) [noise=true , address=104.36.82.45:4907 , piece_id=ZF2SKF5O2WVNHM7CHQ7E4U7HVKQWCJGOZEVIDXO3OU4XHVUFPIGA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12URx18Ckh7VaAKJ1hdQQucMRu5qiDrAraqaycjYqxETgwZ4uWS): protocol: expected piece hash; context canceled; EOF
Aug 8 07:28:51.487325587 (21h49m26.00242815s) [noise=true , address=104.36.82.9:4008 , piece_id=EGIGCEJHELTHOJZWPFWNHCP672MIXAHZIQUK2WDHER77VVJ4NZZA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.487748790 (21h49m26.002851367s) [noise=true , address=193.149.164.143:4693 , piece_id=RCLLY6ABOIXYK6BBO5EWKTDDJ4MUH6IP2WGIHSTER4TEPSF2FYVA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LHoCE9CnM32tbMNqAB7HpmArSVLTycv9o6kSSFNUJsoDeERkQ): protocol: read tcp 159.223.143.39:37688->193.149.164.143:4693: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 07:28:51.487858041 (21h49m26.002960604s) [noise=true , address=45.91.116.11:4120 , piece_id=274GAU5MCZAZOHB3ESPDSJP23Z5FZB636CWGN7ELS2HYY5OFZ7RA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.487890658 (21h49m26.002993237s) [seg_pos={0 0}, upload=22550]: Done waiting for segment. successful: 16 optimal: 14 errs: <nil>
Aug 8 07:28:51.487971404 (21h49m26.00307399s) [upload=22550]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 8 07:28:51.487976917 (21h49m26.003079481s) [upload=22550]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 8 07:28:51.647177827 (21h49m26.162280406s) [upload=22550]: Done uploading segments...
Aug 8 07:28:51.647194442 (21h49m26.162297008s) [upload=22550]: Upload finished. err: <nil>
Aug 8 07:29:34.528501362 (21h50m9.043603939s) [upload=22562]: Starting upload
Aug 8 07:29:34.528548565 (21h50m9.043651143s) [upload=22562]: Done starting upload
Aug 8 07:29:34.528615374 (21h50m9.043717942s) [upload=22562]: Uploading segments...
Aug 8 07:29:34.528903298 (21h50m9.044005874s) [upload=22562]: Got next segment. Inline: false
Aug 8 07:29:34.528940157 (21h50m9.044042733s) [upload=22562]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 8 07:29:34.528944789 (21h50m9.044047366s) [upload=22562]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 8 07:29:34.536881082 (21h50m9.051983672s) [upload=22562]: Next segment err: metaclient: Too Many Requests
Aug 8 07:29:34.536925024 (21h50m9.052027602s) [upload=22562]: Done uploading segments...
Aug 8 07:29:34.536943164 (21h50m9.052045741s) [upload=22562]: Upload finished. err: metaclient: Too Many Requests
Duration: 795.058817ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
11.353808ms target:
https://us1.storj.io/login
FAIL (2025-08-08 07:31:39 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.150.199.48:443: connect: connection refused
Duration: 11.377757ms
core
satellite
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [...........X..............................................X.] duration:
225.389763ms
FAIL (2025-08-08 07:31:26 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== nts... Aug 8 07:24:18.405184267 (21h44m52.920286834s) [upload=22475]: Upload finished. err: <nil> Aug 8 07:24:44.148856783 (21h45m18.663959349s) [upload=22481]: Starting upload Aug 8 07:24:44.148908040 (21h45m18.664010617s) [upload=22481]: Done starting upload Aug 8 07:24:44.148978727 (21h45m18.664081294s) [upload=22481]: Uploading segments... Aug 8 07:24:44.149031432 (21h45m18.664134016s) [upload=22481]: Got next segment. Inline: true Aug 8 07:24:44.149047929 (21h45m18.664150495s) [upload=22481]: Waiting for error group managing segments... Aug 8 07:24:44.149231742 (21h45m18.664334308s) [upload=22481]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:24:44.149238484 (21h45m18.664341061s) [upload=22481]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:24:44.149241695 (21h45m18.664344271s) [upload=22481]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:24:44.236175458 (21h45m18.751278039s) [upload=22481]: Done uploading segments... Aug 8 07:24:44.236197562 (21h45m18.75130015s) [upload=22481]: Upload finished. err: <nil> Aug 8 07:25:45.798907959 (21h46m20.31401054s) [upload=22496]: Starting upload Aug 8 07:25:45.798982852 (21h46m20.314085432s) [upload=22496]: Done starting upload Aug 8 07:25:45.799141617 (21h46m20.314244194s) [upload=22496]: Uploading segments... Aug 8 07:25:45.799187034 (21h46m20.314289621s) [upload=22496]: Got next segment. Inline: true Aug 8 07:25:45.799192794 (21h46m20.314295378s) [upload=22496]: Waiting for error group managing segments... Aug 8 07:25:45.799260506 (21h46m20.314363086s) [upload=22496]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:25:45.799265543 (21h46m20.314368123s) [upload=22496]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:25:45.799269614 (21h46m20.314372195s) [upload=22496]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:25:45.856873466 (21h46m20.371976033s) [upload=22496]: Done uploading segments... Aug 8 07:25:45.856894517 (21h46m20.371997083s) [upload=22496]: Upload finished. err: <nil> Aug 8 07:26:45.160306388 (21h47m19.67540897s) [upload=22512]: Starting upload Aug 8 07:26:45.160369840 (21h47m19.675472407s) [upload=22512]: Done starting upload Aug 8 07:26:45.160448308 (21h47m19.675550874s) [upload=22512]: Uploading segments... Aug 8 07:26:45.160512759 (21h47m19.675615339s) [upload=22512]: Got next segment. Inline: true Aug 8 07:26:45.160518821 (21h47m19.675621389s) [upload=22512]: Waiting for error group managing segments... Aug 8 07:26:45.160566486 (21h47m19.675669054s) [upload=22512]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:26:45.160571363 (21h47m19.675673943s) [upload=22512]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:26:45.160575116 (21h47m19.675677698s) [upload=22512]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:26:45.219521344 (21h47m19.73462392s) [upload=22512]: Done uploading segments... Aug 8 07:26:45.219537755 (21h47m19.73464032s) [upload=22512]: Upload finished. err: <nil> Aug 8 07:27:50.008745364 (21h48m24.523847971s) [upload=22529]: Starting upload Aug 8 07:27:50.008828662 (21h48m24.523931252s) [upload=22529]: Done starting upload Aug 8 07:27:50.008913720 (21h48m24.524016289s) [upload=22529]: Uploading segments... Aug 8 07:27:50.008968901 (21h48m24.524071486s) [upload=22529]: Got next segment. Inline: true Aug 8 07:27:50.008973893 (21h48m24.524076469s) [upload=22529]: Waiting for error group managing segments... Aug 8 07:27:50.009023439 (21h48m24.524126015s) [upload=22529]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:27:50.009028404 (21h48m24.524130983s) [upload=22529]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:27:50.009032740 (21h48m24.524135324s) [upload=22529]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:27:50.072222330 (21h48m24.587324909s) [upload=22529]: Done uploading segments... Aug 8 07:27:50.072260086 (21h48m24.587362652s) [upload=22529]: Upload finished. err: <nil> Aug 8 07:28:08.797868182 (21h48m43.312970759s) [upload=22537]: Starting upload Aug 8 07:28:08.797909617 (21h48m43.313012202s) [upload=22537]: Done starting upload Aug 8 07:28:08.797978107 (21h48m43.313080676s) [upload=22537]: Uploading segments... Aug 8 07:28:08.798028549 (21h48m43.313131134s) [upload=22537]: Got next segment. Inline: true Aug 8 07:28:08.798033746 (21h48m43.313136324s) [upload=22537]: Waiting for error group managing segments... Aug 8 07:28:08.798088133 (21h48m43.313190709s) [upload=22537]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:28:08.798092673 (21h48m43.313195249s) [upload=22537]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:28:08.798096028 (21h48m43.313198604s) [upload=22537]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:28:08.856181519 (21h48m43.371284096s) [upload=22537]: Done uploading segments... Aug 8 07:28:08.856219288 (21h48m43.371321856s) [upload=22537]: Upload finished. err: <nil> Aug 8 07:28:26.426406302 (21h49m0.941508883s) [upload=22543]: Starting upload Aug 8 07:28:26.426484520 (21h49m0.94158709s) [upload=22543]: Done starting upload Aug 8 07:28:26.426561912 (21h49m0.941664479s) [upload=22543]: Uploading segments... Aug 8 07:28:26.426613562 (21h49m0.941716147s) [upload=22543]: Got next segment. Inline: true Aug 8 07:28:26.426618334 (21h49m0.94172091s) [upload=22543]: Waiting for error group managing segments... Aug 8 07:28:26.426667358 (21h49m0.941769934s) [upload=22543]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:28:26.426672477 (21h49m0.941775052s) [upload=22543]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:28:26.426676980 (21h49m0.941779557s) [upload=22543]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:28:26.739786156 (21h49m1.254888734s) [upload=22543]: Done uploading segments... Aug 8 07:28:26.739818088 (21h49m1.254920663s) [upload=22543]: Upload finished. err: <nil> Aug 8 07:28:44.768272418 (21h49m19.283374995s) [upload=22547]: Starting upload Aug 8 07:28:44.768352182 (21h49m19.28345476s) [upload=22547]: Done starting upload Aug 8 07:28:44.768463680 (21h49m19.283566246s) [upload=22547]: Uploading segments... Aug 8 07:28:44.768552166 (21h49m19.283654733s) [upload=22547]: Got next segment. Inline: true Aug 8 07:28:44.768558003 (21h49m19.28366057s) [upload=22547]: Waiting for error group managing segments... Aug 8 07:28:44.768617460 (21h49m19.28372004s) [upload=22547]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:28:44.768622292 (21h49m19.283724868s) [upload=22547]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:28:44.768625505 (21h49m19.283728082s) [upload=22547]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:28:44.832641596 (21h49m19.347744163s) [upload=22547]: Done uploading segments... Aug 8 07:28:44.832661990 (21h49m19.347764559s) [upload=22547]: Upload finished. err: <nil> Aug 8 07:29:19.907807846 (21h49m54.422910428s) [upload=22558]: Starting upload Aug 8 07:29:19.907919209 (21h49m54.423021801s) [upload=22558]: Done starting upload Aug 8 07:29:19.908035700 (21h49m54.423138281s) [upload=22558]: Uploading segments... Aug 8 07:29:19.908115743 (21h49m54.423218334s) [upload=22558]: Got next segment. Inline: true Aug 8 07:29:19.908123216 (21h49m54.423225802s) [upload=22558]: Waiting for error group managing segments... Aug 8 07:29:19.908206488 (21h49m54.423309073s) [upload=22558]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:29:19.908213079 (21h49m54.423315659s) [upload=22558]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:29:19.908216828 (21h49m54.423319408s) [upload=22558]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:29:19.968416399 (21h49m54.483518987s) [upload=22558]: Done uploading segments... Aug 8 07:29:19.968437892 (21h49m54.48354047s) [upload=22558]: Upload finished. err: <nil> Aug 8 07:30:04.570811298 (21h50m39.085913875s) [upload=22574]: Starting upload Aug 8 07:30:04.570856561 (21h50m39.085959129s) [upload=22574]: Done starting upload Aug 8 07:30:04.570937395 (21h50m39.086039963s) [upload=22574]: Uploading segments... Aug 8 07:30:04.570993290 (21h50m39.086095879s) [upload=22574]: Got next segment. Inline: true Aug 8 07:30:04.570999714 (21h50m39.086102306s) [upload=22574]: Waiting for error group managing segments... Aug 8 07:30:04.571094735 (21h50m39.086197316s) [upload=22574]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:30:04.571101887 (21h50m39.086204467s) [upload=22574]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:30:04.571105790 (21h50m39.086208372s) [upload=22574]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:30:04.649336948 (21h50m39.164439536s) [upload=22574]: Done uploading segments... Aug 8 07:30:04.649362004 (21h50m39.164464585s) [upload=22574]: Upload finished. err: <nil> Aug 8 07:30:54.851564431 (21h51m29.366666997s) [upload=22587]: Starting upload Aug 8 07:30:54.851618044 (21h51m29.36672062s) [upload=22587]: Done starting upload Aug 8 07:30:54.851712059 (21h51m29.366814635s) [upload=22587]: Uploading segments... Aug 8 07:30:54.851783178 (21h51m29.36688576s) [upload=22587]: Got next segment. Inline: true Aug 8 07:30:54.851789517 (21h51m29.366892093s) [upload=22587]: Waiting for error group managing segments... Aug 8 07:30:54.851854337 (21h51m29.366956924s) [upload=22587]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:30:54.851861334 (21h51m29.366963924s) [upload=22587]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:30:54.851866059 (21h51m29.366968642s) [upload=22587]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:30:54.860593007 (21h51m29.375695595s) [upload=22587]: Done uploading segments... Aug 8 07:30:54.860627519 (21h51m29.375730095s) [upload=22587]: Upload finished. err: metaclient: Too Many Requests
Duration: 225.410126ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [............X..............................................X] duration:
39.299996ms
FAIL (2025-08-08 07:30:54 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== nts... Aug 8 07:24:18.405184267 (21h44m52.920286834s) [upload=22475]: Upload finished. err: <nil> Aug 8 07:24:44.148856783 (21h45m18.663959349s) [upload=22481]: Starting upload Aug 8 07:24:44.148908040 (21h45m18.664010617s) [upload=22481]: Done starting upload Aug 8 07:24:44.148978727 (21h45m18.664081294s) [upload=22481]: Uploading segments... Aug 8 07:24:44.149031432 (21h45m18.664134016s) [upload=22481]: Got next segment. Inline: true Aug 8 07:24:44.149047929 (21h45m18.664150495s) [upload=22481]: Waiting for error group managing segments... Aug 8 07:24:44.149231742 (21h45m18.664334308s) [upload=22481]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:24:44.149238484 (21h45m18.664341061s) [upload=22481]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:24:44.149241695 (21h45m18.664344271s) [upload=22481]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:24:44.236175458 (21h45m18.751278039s) [upload=22481]: Done uploading segments... Aug 8 07:24:44.236197562 (21h45m18.75130015s) [upload=22481]: Upload finished. err: <nil> Aug 8 07:25:45.798907959 (21h46m20.31401054s) [upload=22496]: Starting upload Aug 8 07:25:45.798982852 (21h46m20.314085432s) [upload=22496]: Done starting upload Aug 8 07:25:45.799141617 (21h46m20.314244194s) [upload=22496]: Uploading segments... Aug 8 07:25:45.799187034 (21h46m20.314289621s) [upload=22496]: Got next segment. Inline: true Aug 8 07:25:45.799192794 (21h46m20.314295378s) [upload=22496]: Waiting for error group managing segments... Aug 8 07:25:45.799260506 (21h46m20.314363086s) [upload=22496]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:25:45.799265543 (21h46m20.314368123s) [upload=22496]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:25:45.799269614 (21h46m20.314372195s) [upload=22496]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:25:45.856873466 (21h46m20.371976033s) [upload=22496]: Done uploading segments... Aug 8 07:25:45.856894517 (21h46m20.371997083s) [upload=22496]: Upload finished. err: <nil> Aug 8 07:26:45.160306388 (21h47m19.67540897s) [upload=22512]: Starting upload Aug 8 07:26:45.160369840 (21h47m19.675472407s) [upload=22512]: Done starting upload Aug 8 07:26:45.160448308 (21h47m19.675550874s) [upload=22512]: Uploading segments... Aug 8 07:26:45.160512759 (21h47m19.675615339s) [upload=22512]: Got next segment. Inline: true Aug 8 07:26:45.160518821 (21h47m19.675621389s) [upload=22512]: Waiting for error group managing segments... Aug 8 07:26:45.160566486 (21h47m19.675669054s) [upload=22512]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:26:45.160571363 (21h47m19.675673943s) [upload=22512]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:26:45.160575116 (21h47m19.675677698s) [upload=22512]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:26:45.219521344 (21h47m19.73462392s) [upload=22512]: Done uploading segments... Aug 8 07:26:45.219537755 (21h47m19.73464032s) [upload=22512]: Upload finished. err: <nil> Aug 8 07:27:50.008745364 (21h48m24.523847971s) [upload=22529]: Starting upload Aug 8 07:27:50.008828662 (21h48m24.523931252s) [upload=22529]: Done starting upload Aug 8 07:27:50.008913720 (21h48m24.524016289s) [upload=22529]: Uploading segments... Aug 8 07:27:50.008968901 (21h48m24.524071486s) [upload=22529]: Got next segment. Inline: true Aug 8 07:27:50.008973893 (21h48m24.524076469s) [upload=22529]: Waiting for error group managing segments... Aug 8 07:27:50.009023439 (21h48m24.524126015s) [upload=22529]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:27:50.009028404 (21h48m24.524130983s) [upload=22529]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:27:50.009032740 (21h48m24.524135324s) [upload=22529]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:27:50.072222330 (21h48m24.587324909s) [upload=22529]: Done uploading segments... Aug 8 07:27:50.072260086 (21h48m24.587362652s) [upload=22529]: Upload finished. err: <nil> Aug 8 07:28:08.797868182 (21h48m43.312970759s) [upload=22537]: Starting upload Aug 8 07:28:08.797909617 (21h48m43.313012202s) [upload=22537]: Done starting upload Aug 8 07:28:08.797978107 (21h48m43.313080676s) [upload=22537]: Uploading segments... Aug 8 07:28:08.798028549 (21h48m43.313131134s) [upload=22537]: Got next segment. Inline: true Aug 8 07:28:08.798033746 (21h48m43.313136324s) [upload=22537]: Waiting for error group managing segments... Aug 8 07:28:08.798088133 (21h48m43.313190709s) [upload=22537]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:28:08.798092673 (21h48m43.313195249s) [upload=22537]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:28:08.798096028 (21h48m43.313198604s) [upload=22537]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:28:08.856181519 (21h48m43.371284096s) [upload=22537]: Done uploading segments... Aug 8 07:28:08.856219288 (21h48m43.371321856s) [upload=22537]: Upload finished. err: <nil> Aug 8 07:28:26.426406302 (21h49m0.941508883s) [upload=22543]: Starting upload Aug 8 07:28:26.426484520 (21h49m0.94158709s) [upload=22543]: Done starting upload Aug 8 07:28:26.426561912 (21h49m0.941664479s) [upload=22543]: Uploading segments... Aug 8 07:28:26.426613562 (21h49m0.941716147s) [upload=22543]: Got next segment. Inline: true Aug 8 07:28:26.426618334 (21h49m0.94172091s) [upload=22543]: Waiting for error group managing segments... Aug 8 07:28:26.426667358 (21h49m0.941769934s) [upload=22543]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:28:26.426672477 (21h49m0.941775052s) [upload=22543]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:28:26.426676980 (21h49m0.941779557s) [upload=22543]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:28:26.739786156 (21h49m1.254888734s) [upload=22543]: Done uploading segments... Aug 8 07:28:26.739818088 (21h49m1.254920663s) [upload=22543]: Upload finished. err: <nil> Aug 8 07:28:44.768272418 (21h49m19.283374995s) [upload=22547]: Starting upload Aug 8 07:28:44.768352182 (21h49m19.28345476s) [upload=22547]: Done starting upload Aug 8 07:28:44.768463680 (21h49m19.283566246s) [upload=22547]: Uploading segments... Aug 8 07:28:44.768552166 (21h49m19.283654733s) [upload=22547]: Got next segment. Inline: true Aug 8 07:28:44.768558003 (21h49m19.28366057s) [upload=22547]: Waiting for error group managing segments... Aug 8 07:28:44.768617460 (21h49m19.28372004s) [upload=22547]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:28:44.768622292 (21h49m19.283724868s) [upload=22547]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:28:44.768625505 (21h49m19.283728082s) [upload=22547]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:28:44.832641596 (21h49m19.347744163s) [upload=22547]: Done uploading segments... Aug 8 07:28:44.832661990 (21h49m19.347764559s) [upload=22547]: Upload finished. err: <nil> Aug 8 07:29:19.907807846 (21h49m54.422910428s) [upload=22558]: Starting upload Aug 8 07:29:19.907919209 (21h49m54.423021801s) [upload=22558]: Done starting upload Aug 8 07:29:19.908035700 (21h49m54.423138281s) [upload=22558]: Uploading segments... Aug 8 07:29:19.908115743 (21h49m54.423218334s) [upload=22558]: Got next segment. Inline: true Aug 8 07:29:19.908123216 (21h49m54.423225802s) [upload=22558]: Waiting for error group managing segments... Aug 8 07:29:19.908206488 (21h49m54.423309073s) [upload=22558]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:29:19.908213079 (21h49m54.423315659s) [upload=22558]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:29:19.908216828 (21h49m54.423319408s) [upload=22558]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:29:19.968416399 (21h49m54.483518987s) [upload=22558]: Done uploading segments... Aug 8 07:29:19.968437892 (21h49m54.48354047s) [upload=22558]: Upload finished. err: <nil> Aug 8 07:30:04.570811298 (21h50m39.085913875s) [upload=22574]: Starting upload Aug 8 07:30:04.570856561 (21h50m39.085959129s) [upload=22574]: Done starting upload Aug 8 07:30:04.570937395 (21h50m39.086039963s) [upload=22574]: Uploading segments... Aug 8 07:30:04.570993290 (21h50m39.086095879s) [upload=22574]: Got next segment. Inline: true Aug 8 07:30:04.570999714 (21h50m39.086102306s) [upload=22574]: Waiting for error group managing segments... Aug 8 07:30:04.571094735 (21h50m39.086197316s) [upload=22574]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:30:04.571101887 (21h50m39.086204467s) [upload=22574]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:30:04.571105790 (21h50m39.086208372s) [upload=22574]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:30:04.649336948 (21h50m39.164439536s) [upload=22574]: Done uploading segments... Aug 8 07:30:04.649362004 (21h50m39.164464585s) [upload=22574]: Upload finished. err: <nil> Aug 8 07:30:54.851564431 (21h51m29.366666997s) [upload=22587]: Starting upload Aug 8 07:30:54.851618044 (21h51m29.36672062s) [upload=22587]: Done starting upload Aug 8 07:30:54.851712059 (21h51m29.366814635s) [upload=22587]: Uploading segments... Aug 8 07:30:54.851783178 (21h51m29.36688576s) [upload=22587]: Got next segment. Inline: true Aug 8 07:30:54.851789517 (21h51m29.366892093s) [upload=22587]: Waiting for error group managing segments... Aug 8 07:30:54.851854337 (21h51m29.366956924s) [upload=22587]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:30:54.851861334 (21h51m29.366963924s) [upload=22587]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:30:54.851866059 (21h51m29.366968642s) [upload=22587]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:30:54.860593007 (21h51m29.375695595s) [upload=22587]: Done uploading segments... Aug 8 07:30:54.860627519 (21h51m29.375730095s) [upload=22587]: Upload finished. err: metaclient: Too Many Requests
Duration: 39.324251ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.] duration:
452.030215ms
FAIL (2025-08-08 07:30:44 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
g_pos={0 0}, upload=22550]: Begin upload segment...
Aug 8 07:28:51.351770179 (21h49m25.866872766s) [seg_pos={0 0}, upload=22550]: Done begin upload segment.
Aug 8 07:28:51.351799155 (21h49m25.866901747s) [noise=true , address=104.36.82.23:4469 , piece_id=CI6CU5IBYK37MBIQW554EW6PPXVYA6ZF2CJPVZCIXMFFLXZ7UO7Q, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.351816053 (21h49m25.86691864s) [noise=true , address=193.149.164.143:4693 , piece_id=RCLLY6ABOIXYK6BBO5EWKTDDJ4MUH6IP2WGIHSTER4TEPSF2FYVA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.352227136 (21h49m25.867329713s) [noise=true , address=104.36.82.58:4286 , piece_id=36LFLTD2YWV3K7LM4M2GY3JIKLUDOZVRINQT3VLIFBRR55UCNA4Q, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.352322195 (21h49m25.867424772s) [noise=true , address=104.36.82.44:4880 , piece_id=HFXFL7ULX5OTYJFEOC45DI75DGDIHAQA6ZJIXHO22X4GPSEZN75Q, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.352611348 (21h49m25.867713924s) [noise=true , address=104.36.82.53:4139 , piece_id=IGOHVQJMZ3NWLCBKVPVHTVWVHDS43DABQPEMJUTB2TXAQPQOURSQ, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.352702579 (21h49m25.867805142s) [noise=true , address=45.91.116.7:4013 , piece_id=XG4M7HF2E5YIRJ7PVBEGZC2ADPUPAL2QL5TPUAFIYXZUP3BX2Y7A, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.353022021 (21h49m25.868124597s) [noise=true , address=104.36.82.16:4321 , piece_id=MTRDWQYPHECWBAH4ECUGJL2Y2LAUA2RWHX5J6XMR65NZOACBF6PA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.353464633 (21h49m25.868567211s) [noise=true , address=104.36.82.46:4487 , piece_id=JOSIX5WT5A36RXKJNWO5KLBEQOCNQCEZ7A2S4KYIBJHRKN3KT6RQ, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.354418304 (21h49m25.869520884s) [noise=true , address=104.36.82.38:4707 , piece_id=WZWQHYDMTUG2FOTZ6JJRVIXXFIPHJ3I2YIRQXMN7B2W7QGKAFY3A, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.354959848 (21h49m25.87006243s) [noise=true , address=104.36.82.47:4544 , piece_id=YDG3KOKUS5NVBWMQYJ7JJEO3PAXZKPZPUOUWUDCZ4RAVATW67NKQ, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.355461506 (21h49m25.870564086s) [noise=true , address=104.36.82.60:4363 , piece_id=QMR7CM4ASNAKG5PT2XUPFR4MIBHNFYJ7OZMASJ7Q6R2C225L655A, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.355910519 (21h49m25.871013101s) [noise=true , address=104.36.82.2:4741 , piece_id=3NWVL6DQCMDO5PQLSFU3VER7RSHGRAB6YRT32XBFUVLAXGEJQN7A, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.357467009 (21h49m25.872569585s) [noise=true , address=104.36.82.59:4322 , piece_id=NKCSGWGYWOIJWHSQUN2ZYOVZYIRML5G3DXVRJYDTAMHTEN2VES3Q, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.358628018 (21h49m25.873730598s) [noise=true , address=45.91.116.11:4120 , piece_id=274GAU5MCZAZOHB3ESPDSJP23Z5FZB636CWGN7ELS2HYY5OFZ7RA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.359105086 (21h49m25.874207666s) [noise=true , address=104.36.82.45:4907 , piece_id=ZF2SKF5O2WVNHM7CHQ7E4U7HVKQWCJGOZEVIDXO3OU4XHVUFPIGA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.360554866 (21h49m25.875657451s) [noise=true , address=104.36.82.36:4648 , piece_id=3WAWCS3XXWVX3ACYJG2OOS2R34SXOWYSHXVF6AMPFXJFCMYH4SZA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.363015621 (21h49m25.8781182s) [noise=true , address=104.36.82.62:4420 , piece_id=7S6ALI3DNOM7IURBTYY3DLALDQV7R7LLHPQCA4Q6G4HA2SOZ2YMA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.364089235 (21h49m25.87919182s) [noise=true , address=104.36.82.9:4008 , piece_id=EGIGCEJHELTHOJZWPFWNHCP672MIXAHZIQUK2WDHER77VVJ4NZZA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.467747752 (21h49m25.982850337s) [upload=22550]: Next returned nil segment
Aug 8 07:28:51.467767955 (21h49m25.982870543s) [upload=22550]: Waiting for error group managing segments...
Aug 8 07:28:51.473577577 (21h49m25.988680143s) [noise=true , address=45.91.116.7:4013 , piece_id=XG4M7HF2E5YIRJ7PVBEGZC2ADPUPAL2QL5TPUAFIYXZUP3BX2Y7A, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.478188638 (21h49m25.993291216s) [noise=true , address=104.36.82.16:4321 , piece_id=MTRDWQYPHECWBAH4ECUGJL2Y2LAUA2RWHX5J6XMR65NZOACBF6PA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.478342049 (21h49m25.993444627s) [noise=true , address=104.36.82.53:4139 , piece_id=IGOHVQJMZ3NWLCBKVPVHTVWVHDS43DABQPEMJUTB2TXAQPQOURSQ, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.478768394 (21h49m25.993870961s) [noise=true , address=104.36.82.36:4648 , piece_id=3WAWCS3XXWVX3ACYJG2OOS2R34SXOWYSHXVF6AMPFXJFCMYH4SZA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.479336172 (21h49m25.994438735s) [noise=true , address=104.36.82.59:4322 , piece_id=NKCSGWGYWOIJWHSQUN2ZYOVZYIRML5G3DXVRJYDTAMHTEN2VES3Q, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.479515883 (21h49m25.994618457s) [noise=true , address=104.36.82.58:4286 , piece_id=36LFLTD2YWV3K7LM4M2GY3JIKLUDOZVRINQT3VLIFBRR55UCNA4Q, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.480474985 (21h49m25.995577567s) [noise=true , address=104.36.82.2:4741 , piece_id=3NWVL6DQCMDO5PQLSFU3VER7RSHGRAB6YRT32XBFUVLAXGEJQN7A, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.481008628 (21h49m25.996111195s) [noise=true , address=104.36.82.38:4707 , piece_id=WZWQHYDMTUG2FOTZ6JJRVIXXFIPHJ3I2YIRQXMN7B2W7QGKAFY3A, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.481589774 (21h49m25.996692358s) [noise=true , address=104.36.82.60:4363 , piece_id=QMR7CM4ASNAKG5PT2XUPFR4MIBHNFYJ7OZMASJ7Q6R2C225L655A, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.482143054 (21h49m25.99724562s) [noise=true , address=104.36.82.46:4487 , piece_id=JOSIX5WT5A36RXKJNWO5KLBEQOCNQCEZ7A2S4KYIBJHRKN3KT6RQ, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.485172215 (21h49m26.000274779s) [noise=true , address=104.36.82.47:4544 , piece_id=YDG3KOKUS5NVBWMQYJ7JJEO3PAXZKPZPUOUWUDCZ4RAVATW67NKQ, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.486088591 (21h49m26.001191172s) [noise=true , address=104.36.82.44:4880 , piece_id=HFXFL7ULX5OTYJFEOC45DI75DGDIHAQA6ZJIXHO22X4GPSEZN75Q, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.486628345 (21h49m26.001730916s) [noise=true , address=104.36.82.23:4469 , piece_id=CI6CU5IBYK37MBIQW554EW6PPXVYA6ZF2CJPVZCIXMFFLXZ7UO7Q, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.487179958 (21h49m26.002282521s) [noise=true , address=104.36.82.62:4420 , piece_id=7S6ALI3DNOM7IURBTYY3DLALDQV7R7LLHPQCA4Q6G4HA2SOZ2YMA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.487192580 (21h49m26.002295147s) [seg_pos={0 0}, upload=22550]: Segment reached optimal threshold of 14 pieces.
Aug 8 07:28:51.487306760 (21h49m26.002409325s) [noise=true , address=104.36.82.45:4907 , piece_id=ZF2SKF5O2WVNHM7CHQ7E4U7HVKQWCJGOZEVIDXO3OU4XHVUFPIGA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12URx18Ckh7VaAKJ1hdQQucMRu5qiDrAraqaycjYqxETgwZ4uWS): protocol: expected piece hash; context canceled; EOF
Aug 8 07:28:51.487325587 (21h49m26.00242815s) [noise=true , address=104.36.82.9:4008 , piece_id=EGIGCEJHELTHOJZWPFWNHCP672MIXAHZIQUK2WDHER77VVJ4NZZA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.487748790 (21h49m26.002851367s) [noise=true , address=193.149.164.143:4693 , piece_id=RCLLY6ABOIXYK6BBO5EWKTDDJ4MUH6IP2WGIHSTER4TEPSF2FYVA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LHoCE9CnM32tbMNqAB7HpmArSVLTycv9o6kSSFNUJsoDeERkQ): protocol: read tcp 159.223.143.39:37688->193.149.164.143:4693: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 07:28:51.487858041 (21h49m26.002960604s) [noise=true , address=45.91.116.11:4120 , piece_id=274GAU5MCZAZOHB3ESPDSJP23Z5FZB636CWGN7ELS2HYY5OFZ7RA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.487890658 (21h49m26.002993237s) [seg_pos={0 0}, upload=22550]: Done waiting for segment. successful: 16 optimal: 14 errs: <nil>
Aug 8 07:28:51.487971404 (21h49m26.00307399s) [upload=22550]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 8 07:28:51.487976917 (21h49m26.003079481s) [upload=22550]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 8 07:28:51.647177827 (21h49m26.162280406s) [upload=22550]: Done uploading segments...
Aug 8 07:28:51.647194442 (21h49m26.162297008s) [upload=22550]: Upload finished. err: <nil>
Aug 8 07:29:34.528501362 (21h50m9.043603939s) [upload=22562]: Starting upload
Aug 8 07:29:34.528548565 (21h50m9.043651143s) [upload=22562]: Done starting upload
Aug 8 07:29:34.528615374 (21h50m9.043717942s) [upload=22562]: Uploading segments...
Aug 8 07:29:34.528903298 (21h50m9.044005874s) [upload=22562]: Got next segment. Inline: false
Aug 8 07:29:34.528940157 (21h50m9.044042733s) [upload=22562]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 8 07:29:34.528944789 (21h50m9.044047366s) [upload=22562]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 8 07:29:34.536881082 (21h50m9.051983672s) [upload=22562]: Next segment err: metaclient: Too Many Requests
Aug 8 07:29:34.536925024 (21h50m9.052027602s) [upload=22562]: Done uploading segments...
Aug 8 07:29:34.536943164 (21h50m9.052045741s) [upload=22562]: Upload finished. err: metaclient: Too Many Requests
Duration: 452.061676ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [.............X..............................................] duration:
212.568985ms
FAIL (2025-08-08 07:30:04 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:1024 (size:1024) ======LOGS====== d=21870]: Done uploading segments... Aug 8 06:48:53.275183069 (21h9m27.790285647s) [upload=21870]: Upload finished. err: <nil> Aug 8 06:49:08.545547742 (21h9m43.060650311s) [upload=21875]: Starting upload Aug 8 06:49:08.545594244 (21h9m43.06069682s) [upload=21875]: Done starting upload Aug 8 06:49:08.545651807 (21h9m43.060754385s) [upload=21875]: Uploading segments... Aug 8 06:49:08.545688834 (21h9m43.060791416s) [upload=21875]: Got next segment. Inline: true Aug 8 06:49:08.545693597 (21h9m43.060796163s) [upload=21875]: Waiting for error group managing segments... Aug 8 06:49:08.545736511 (21h9m43.060839081s) [upload=21875]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:49:08.545742589 (21h9m43.060845161s) [upload=21875]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:49:08.545747536 (21h9m43.060850111s) [upload=21875]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:49:08.608446156 (21h9m43.123548745s) [upload=21875]: Done uploading segments... Aug 8 06:49:08.608468645 (21h9m43.123571226s) [upload=21875]: Upload finished. err: <nil> Aug 8 06:49:53.145025965 (21h10m27.660128534s) [upload=21886]: Starting upload Aug 8 06:49:53.145086326 (21h10m27.660188904s) [upload=21886]: Done starting upload Aug 8 06:49:53.145166436 (21h10m27.660269004s) [upload=21886]: Uploading segments... Aug 8 06:49:53.145207405 (21h10m27.66030999s) [upload=21886]: Got next segment. Inline: true Aug 8 06:49:53.145212962 (21h10m27.660315539s) [upload=21886]: Waiting for error group managing segments... Aug 8 06:49:53.145265613 (21h10m27.660368179s) [upload=21886]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:49:53.145270200 (21h10m27.660372767s) [upload=21886]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:49:53.145274109 (21h10m27.660376686s) [upload=21886]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:49:53.205847742 (21h10m27.720950321s) [upload=21886]: Done uploading segments... Aug 8 06:49:53.205867747 (21h10m27.720970327s) [upload=21886]: Upload finished. err: <nil> Aug 8 06:50:45.962226830 (21h11m20.477329397s) [upload=21901]: Starting upload Aug 8 06:50:45.962960747 (21h11m20.47806339s) [upload=21901]: Done starting upload Aug 8 06:50:45.963081429 (21h11m20.478184007s) [upload=21901]: Uploading segments... Aug 8 06:50:45.963344944 (21h11m20.478447529s) [upload=21901]: Got next segment. Inline: true Aug 8 06:50:45.963354188 (21h11m20.478456768s) [upload=21901]: Waiting for error group managing segments... Aug 8 06:50:45.963408259 (21h11m20.478510836s) [upload=21901]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:50:45.963413437 (21h11m20.478516016s) [upload=21901]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:50:45.963418686 (21h11m20.478521263s) [upload=21901]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:50:46.026766649 (21h11m20.541869227s) [upload=21901]: Done uploading segments... Aug 8 06:50:46.026795865 (21h11m20.541898441s) [upload=21901]: Upload finished. err: <nil> Aug 8 06:51:58.480831248 (21h12m32.995933826s) [upload=21921]: Starting upload Aug 8 06:51:58.480895665 (21h12m32.995998248s) [upload=21921]: Done starting upload Aug 8 06:51:58.481023418 (21h12m32.996125988s) [upload=21921]: Uploading segments... Aug 8 06:51:58.481086289 (21h12m32.996188877s) [upload=21921]: Got next segment. Inline: true Aug 8 06:51:58.481091513 (21h12m32.99619409s) [upload=21921]: Waiting for error group managing segments... Aug 8 06:51:58.481149620 (21h12m32.996252197s) [upload=21921]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:51:58.481154789 (21h12m32.996257366s) [upload=21921]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:51:58.481158546 (21h12m32.996261124s) [upload=21921]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:51:58.722055331 (21h12m33.237157908s) [upload=21921]: Done uploading segments... Aug 8 06:51:58.722073114 (21h12m33.23717568s) [upload=21921]: Upload finished. err: <nil> Aug 8 06:52:27.059254149 (21h13m1.57435673s) [upload=21928]: Starting upload Aug 8 06:52:27.059321812 (21h13m1.574424392s) [upload=21928]: Done starting upload Aug 8 06:52:27.059401946 (21h13m1.574504512s) [upload=21928]: Uploading segments... Aug 8 06:52:27.059448476 (21h13m1.574551053s) [upload=21928]: Got next segment. Inline: true Aug 8 06:52:27.059454258 (21h13m1.574556833s) [upload=21928]: Waiting for error group managing segments... Aug 8 06:52:27.059501468 (21h13m1.574604035s) [upload=21928]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:52:27.059506838 (21h13m1.574609416s) [upload=21928]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:52:27.059510827 (21h13m1.574613404s) [upload=21928]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:52:27.120385847 (21h13m1.635488418s) [upload=21928]: Done uploading segments... Aug 8 06:52:27.120403445 (21h13m1.635506012s) [upload=21928]: Upload finished. err: <nil> Aug 8 06:52:45.219137365 (21h13m19.734239945s) [upload=21933]: Starting upload Aug 8 06:52:45.219214556 (21h13m19.734317133s) [upload=21933]: Done starting upload Aug 8 06:52:45.219284289 (21h13m19.734386877s) [upload=21933]: Uploading segments... Aug 8 06:52:45.219335736 (21h13m19.73443834s) [upload=21933]: Got next segment. Inline: true Aug 8 06:52:45.219342793 (21h13m19.734445378s) [upload=21933]: Waiting for error group managing segments... Aug 8 06:52:45.219397449 (21h13m19.734500028s) [upload=21933]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:52:45.219402775 (21h13m19.734505359s) [upload=21933]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:52:45.219407364 (21h13m19.734509958s) [upload=21933]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:52:45.282551475 (21h13m19.79765406s) [upload=21933]: Done uploading segments... Aug 8 06:52:45.282572182 (21h13m19.797674765s) [upload=21933]: Upload finished. err: <nil> Aug 8 06:53:36.462986837 (21h14m10.978089404s) [upload=21951]: Starting upload Aug 8 06:53:36.463070050 (21h14m10.978172618s) [upload=21951]: Done starting upload Aug 8 06:53:36.463162828 (21h14m10.978265396s) [upload=21951]: Uploading segments... Aug 8 06:53:36.463220839 (21h14m10.978323421s) [upload=21951]: Got next segment. Inline: true Aug 8 06:53:36.463227617 (21h14m10.978330198s) [upload=21951]: Waiting for error group managing segments... Aug 8 06:53:36.463290301 (21h14m10.978392878s) [upload=21951]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:53:36.463295593 (21h14m10.978398162s) [upload=21951]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:53:36.463299324 (21h14m10.978401891s) [upload=21951]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:53:36.745725330 (21h14m11.260827913s) [upload=21951]: Done uploading segments... Aug 8 06:53:36.745748524 (21h14m11.260851096s) [upload=21951]: Upload finished. err: <nil> Aug 8 06:54:42.211460075 (21h15m16.726562642s) [upload=21971]: Starting upload Aug 8 06:54:42.211544762 (21h15m16.726647342s) [upload=21971]: Done starting upload Aug 8 06:54:42.211670011 (21h15m16.726772576s) [upload=21971]: Uploading segments... Aug 8 06:54:42.211709612 (21h15m16.726812197s) [upload=21971]: Got next segment. Inline: true Aug 8 06:54:42.211714530 (21h15m16.726817106s) [upload=21971]: Waiting for error group managing segments... Aug 8 06:54:42.211773234 (21h15m16.726875811s) [upload=21971]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:54:42.211777907 (21h15m16.726880484s) [upload=21971]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:54:42.211781406 (21h15m16.726883982s) [upload=21971]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:54:42.285628155 (21h15m16.800730732s) [upload=21971]: Done uploading segments... Aug 8 06:54:42.285645434 (21h15m16.800748001s) [upload=21971]: Upload finished. err: <nil> Aug 8 06:55:07.196164830 (21h15m41.711267417s) [upload=21978]: Starting upload Aug 8 06:55:07.196239324 (21h15m41.711341924s) [upload=21978]: Done starting upload Aug 8 06:55:07.196316739 (21h15m41.711419326s) [upload=21978]: Uploading segments... Aug 8 06:55:07.196366345 (21h15m41.711468934s) [upload=21978]: Got next segment. Inline: true Aug 8 06:55:07.196372407 (21h15m41.711474978s) [upload=21978]: Waiting for error group managing segments... Aug 8 06:55:07.196426504 (21h15m41.711529089s) [upload=21978]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:55:07.196433124 (21h15m41.711535688s) [upload=21978]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:55:07.196438529 (21h15m41.711541093s) [upload=21978]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:55:07.258380171 (21h15m41.773482756s) [upload=21978]: Done uploading segments... Aug 8 06:55:07.258413055 (21h15m41.773515634s) [upload=21978]: Upload finished. err: <nil> Aug 8 06:56:17.251734653 (21h16m51.766837222s) [upload=21994]: Starting upload Aug 8 06:56:17.251784283 (21h16m51.766886851s) [upload=21994]: Done starting upload Aug 8 06:56:17.251882450 (21h16m51.766985019s) [upload=21994]: Uploading segments... Aug 8 06:56:17.251922005 (21h16m51.767024587s) [upload=21994]: Got next segment. Inline: true Aug 8 06:56:17.251926655 (21h16m51.767029231s) [upload=21994]: Waiting for error group managing segments... Aug 8 06:56:17.251978681 (21h16m51.767081246s) [upload=21994]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:56:17.251983158 (21h16m51.767085734s) [upload=21994]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:56:17.251986429 (21h16m51.767089005s) [upload=21994]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:56:17.318329259 (21h16m51.833431843s) [upload=21994]: Done uploading segments... Aug 8 06:56:17.318350310 (21h16m51.833452887s) [upload=21994]: Upload finished. err: <nil>
Duration: 212.598888ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
39.630402ms
FAIL (2025-08-08 07:29:34 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
g_pos={0 0}, upload=22550]: Begin upload segment...
Aug 8 07:28:51.351770179 (21h49m25.866872766s) [seg_pos={0 0}, upload=22550]: Done begin upload segment.
Aug 8 07:28:51.351799155 (21h49m25.866901747s) [noise=true , address=104.36.82.23:4469 , piece_id=CI6CU5IBYK37MBIQW554EW6PPXVYA6ZF2CJPVZCIXMFFLXZ7UO7Q, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.351816053 (21h49m25.86691864s) [noise=true , address=193.149.164.143:4693 , piece_id=RCLLY6ABOIXYK6BBO5EWKTDDJ4MUH6IP2WGIHSTER4TEPSF2FYVA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.352227136 (21h49m25.867329713s) [noise=true , address=104.36.82.58:4286 , piece_id=36LFLTD2YWV3K7LM4M2GY3JIKLUDOZVRINQT3VLIFBRR55UCNA4Q, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.352322195 (21h49m25.867424772s) [noise=true , address=104.36.82.44:4880 , piece_id=HFXFL7ULX5OTYJFEOC45DI75DGDIHAQA6ZJIXHO22X4GPSEZN75Q, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.352611348 (21h49m25.867713924s) [noise=true , address=104.36.82.53:4139 , piece_id=IGOHVQJMZ3NWLCBKVPVHTVWVHDS43DABQPEMJUTB2TXAQPQOURSQ, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.352702579 (21h49m25.867805142s) [noise=true , address=45.91.116.7:4013 , piece_id=XG4M7HF2E5YIRJ7PVBEGZC2ADPUPAL2QL5TPUAFIYXZUP3BX2Y7A, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.353022021 (21h49m25.868124597s) [noise=true , address=104.36.82.16:4321 , piece_id=MTRDWQYPHECWBAH4ECUGJL2Y2LAUA2RWHX5J6XMR65NZOACBF6PA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.353464633 (21h49m25.868567211s) [noise=true , address=104.36.82.46:4487 , piece_id=JOSIX5WT5A36RXKJNWO5KLBEQOCNQCEZ7A2S4KYIBJHRKN3KT6RQ, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.354418304 (21h49m25.869520884s) [noise=true , address=104.36.82.38:4707 , piece_id=WZWQHYDMTUG2FOTZ6JJRVIXXFIPHJ3I2YIRQXMN7B2W7QGKAFY3A, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.354959848 (21h49m25.87006243s) [noise=true , address=104.36.82.47:4544 , piece_id=YDG3KOKUS5NVBWMQYJ7JJEO3PAXZKPZPUOUWUDCZ4RAVATW67NKQ, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.355461506 (21h49m25.870564086s) [noise=true , address=104.36.82.60:4363 , piece_id=QMR7CM4ASNAKG5PT2XUPFR4MIBHNFYJ7OZMASJ7Q6R2C225L655A, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.355910519 (21h49m25.871013101s) [noise=true , address=104.36.82.2:4741 , piece_id=3NWVL6DQCMDO5PQLSFU3VER7RSHGRAB6YRT32XBFUVLAXGEJQN7A, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.357467009 (21h49m25.872569585s) [noise=true , address=104.36.82.59:4322 , piece_id=NKCSGWGYWOIJWHSQUN2ZYOVZYIRML5G3DXVRJYDTAMHTEN2VES3Q, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.358628018 (21h49m25.873730598s) [noise=true , address=45.91.116.11:4120 , piece_id=274GAU5MCZAZOHB3ESPDSJP23Z5FZB636CWGN7ELS2HYY5OFZ7RA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.359105086 (21h49m25.874207666s) [noise=true , address=104.36.82.45:4907 , piece_id=ZF2SKF5O2WVNHM7CHQ7E4U7HVKQWCJGOZEVIDXO3OU4XHVUFPIGA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.360554866 (21h49m25.875657451s) [noise=true , address=104.36.82.36:4648 , piece_id=3WAWCS3XXWVX3ACYJG2OOS2R34SXOWYSHXVF6AMPFXJFCMYH4SZA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.363015621 (21h49m25.8781182s) [noise=true , address=104.36.82.62:4420 , piece_id=7S6ALI3DNOM7IURBTYY3DLALDQV7R7LLHPQCA4Q6G4HA2SOZ2YMA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.364089235 (21h49m25.87919182s) [noise=true , address=104.36.82.9:4008 , piece_id=EGIGCEJHELTHOJZWPFWNHCP672MIXAHZIQUK2WDHER77VVJ4NZZA, seg_pos={0 0}, upload=22550]: Uploading piece...
Aug 8 07:28:51.467747752 (21h49m25.982850337s) [upload=22550]: Next returned nil segment
Aug 8 07:28:51.467767955 (21h49m25.982870543s) [upload=22550]: Waiting for error group managing segments...
Aug 8 07:28:51.473577577 (21h49m25.988680143s) [noise=true , address=45.91.116.7:4013 , piece_id=XG4M7HF2E5YIRJ7PVBEGZC2ADPUPAL2QL5TPUAFIYXZUP3BX2Y7A, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.478188638 (21h49m25.993291216s) [noise=true , address=104.36.82.16:4321 , piece_id=MTRDWQYPHECWBAH4ECUGJL2Y2LAUA2RWHX5J6XMR65NZOACBF6PA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.478342049 (21h49m25.993444627s) [noise=true , address=104.36.82.53:4139 , piece_id=IGOHVQJMZ3NWLCBKVPVHTVWVHDS43DABQPEMJUTB2TXAQPQOURSQ, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.478768394 (21h49m25.993870961s) [noise=true , address=104.36.82.36:4648 , piece_id=3WAWCS3XXWVX3ACYJG2OOS2R34SXOWYSHXVF6AMPFXJFCMYH4SZA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.479336172 (21h49m25.994438735s) [noise=true , address=104.36.82.59:4322 , piece_id=NKCSGWGYWOIJWHSQUN2ZYOVZYIRML5G3DXVRJYDTAMHTEN2VES3Q, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.479515883 (21h49m25.994618457s) [noise=true , address=104.36.82.58:4286 , piece_id=36LFLTD2YWV3K7LM4M2GY3JIKLUDOZVRINQT3VLIFBRR55UCNA4Q, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.480474985 (21h49m25.995577567s) [noise=true , address=104.36.82.2:4741 , piece_id=3NWVL6DQCMDO5PQLSFU3VER7RSHGRAB6YRT32XBFUVLAXGEJQN7A, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.481008628 (21h49m25.996111195s) [noise=true , address=104.36.82.38:4707 , piece_id=WZWQHYDMTUG2FOTZ6JJRVIXXFIPHJ3I2YIRQXMN7B2W7QGKAFY3A, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.481589774 (21h49m25.996692358s) [noise=true , address=104.36.82.60:4363 , piece_id=QMR7CM4ASNAKG5PT2XUPFR4MIBHNFYJ7OZMASJ7Q6R2C225L655A, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.482143054 (21h49m25.99724562s) [noise=true , address=104.36.82.46:4487 , piece_id=JOSIX5WT5A36RXKJNWO5KLBEQOCNQCEZ7A2S4KYIBJHRKN3KT6RQ, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.485172215 (21h49m26.000274779s) [noise=true , address=104.36.82.47:4544 , piece_id=YDG3KOKUS5NVBWMQYJ7JJEO3PAXZKPZPUOUWUDCZ4RAVATW67NKQ, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.486088591 (21h49m26.001191172s) [noise=true , address=104.36.82.44:4880 , piece_id=HFXFL7ULX5OTYJFEOC45DI75DGDIHAQA6ZJIXHO22X4GPSEZN75Q, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.486628345 (21h49m26.001730916s) [noise=true , address=104.36.82.23:4469 , piece_id=CI6CU5IBYK37MBIQW554EW6PPXVYA6ZF2CJPVZCIXMFFLXZ7UO7Q, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.487179958 (21h49m26.002282521s) [noise=true , address=104.36.82.62:4420 , piece_id=7S6ALI3DNOM7IURBTYY3DLALDQV7R7LLHPQCA4Q6G4HA2SOZ2YMA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.487192580 (21h49m26.002295147s) [seg_pos={0 0}, upload=22550]: Segment reached optimal threshold of 14 pieces.
Aug 8 07:28:51.487306760 (21h49m26.002409325s) [noise=true , address=104.36.82.45:4907 , piece_id=ZF2SKF5O2WVNHM7CHQ7E4U7HVKQWCJGOZEVIDXO3OU4XHVUFPIGA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12URx18Ckh7VaAKJ1hdQQucMRu5qiDrAraqaycjYqxETgwZ4uWS): protocol: expected piece hash; context canceled; EOF
Aug 8 07:28:51.487325587 (21h49m26.00242815s) [noise=true , address=104.36.82.9:4008 , piece_id=EGIGCEJHELTHOJZWPFWNHCP672MIXAHZIQUK2WDHER77VVJ4NZZA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.487748790 (21h49m26.002851367s) [noise=true , address=193.149.164.143:4693 , piece_id=RCLLY6ABOIXYK6BBO5EWKTDDJ4MUH6IP2WGIHSTER4TEPSF2FYVA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LHoCE9CnM32tbMNqAB7HpmArSVLTycv9o6kSSFNUJsoDeERkQ): protocol: read tcp 159.223.143.39:37688->193.149.164.143:4693: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 07:28:51.487858041 (21h49m26.002960604s) [noise=true , address=45.91.116.11:4120 , piece_id=274GAU5MCZAZOHB3ESPDSJP23Z5FZB636CWGN7ELS2HYY5OFZ7RA, seg_pos={0 0}, upload=22550]: Done uploading piece. err: <nil>
Aug 8 07:28:51.487890658 (21h49m26.002993237s) [seg_pos={0 0}, upload=22550]: Done waiting for segment. successful: 16 optimal: 14 errs: <nil>
Aug 8 07:28:51.487971404 (21h49m26.00307399s) [upload=22550]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 8 07:28:51.487976917 (21h49m26.003079481s) [upload=22550]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 8 07:28:51.647177827 (21h49m26.162280406s) [upload=22550]: Done uploading segments...
Aug 8 07:28:51.647194442 (21h49m26.162297008s) [upload=22550]: Upload finished. err: <nil>
Aug 8 07:29:34.528501362 (21h50m9.043603939s) [upload=22562]: Starting upload
Aug 8 07:29:34.528548565 (21h50m9.043651143s) [upload=22562]: Done starting upload
Aug 8 07:29:34.528615374 (21h50m9.043717942s) [upload=22562]: Uploading segments...
Aug 8 07:29:34.528903298 (21h50m9.044005874s) [upload=22562]: Got next segment. Inline: false
Aug 8 07:29:34.528940157 (21h50m9.044042733s) [upload=22562]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 8 07:29:34.528944789 (21h50m9.044047366s) [upload=22562]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 8 07:29:34.536881082 (21h50m9.051983672s) [upload=22562]: Next segment err: metaclient: Too Many Requests
Aug 8 07:29:34.536925024 (21h50m9.052027602s) [upload=22562]: Done uploading segments...
Aug 8 07:29:34.536943164 (21h50m9.052045741s) [upload=22562]: Upload finished. err: metaclient: Too Many Requests
Duration: 39.653778ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [..............X.............................................] duration:
187.1755ms
FAIL (2025-08-08 07:29:20 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:1024 (size:1024) ======LOGS====== d=21870]: Done uploading segments... Aug 8 06:48:53.275183069 (21h9m27.790285647s) [upload=21870]: Upload finished. err: <nil> Aug 8 06:49:08.545547742 (21h9m43.060650311s) [upload=21875]: Starting upload Aug 8 06:49:08.545594244 (21h9m43.06069682s) [upload=21875]: Done starting upload Aug 8 06:49:08.545651807 (21h9m43.060754385s) [upload=21875]: Uploading segments... Aug 8 06:49:08.545688834 (21h9m43.060791416s) [upload=21875]: Got next segment. Inline: true Aug 8 06:49:08.545693597 (21h9m43.060796163s) [upload=21875]: Waiting for error group managing segments... Aug 8 06:49:08.545736511 (21h9m43.060839081s) [upload=21875]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:49:08.545742589 (21h9m43.060845161s) [upload=21875]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:49:08.545747536 (21h9m43.060850111s) [upload=21875]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:49:08.608446156 (21h9m43.123548745s) [upload=21875]: Done uploading segments... Aug 8 06:49:08.608468645 (21h9m43.123571226s) [upload=21875]: Upload finished. err: <nil> Aug 8 06:49:53.145025965 (21h10m27.660128534s) [upload=21886]: Starting upload Aug 8 06:49:53.145086326 (21h10m27.660188904s) [upload=21886]: Done starting upload Aug 8 06:49:53.145166436 (21h10m27.660269004s) [upload=21886]: Uploading segments... Aug 8 06:49:53.145207405 (21h10m27.66030999s) [upload=21886]: Got next segment. Inline: true Aug 8 06:49:53.145212962 (21h10m27.660315539s) [upload=21886]: Waiting for error group managing segments... Aug 8 06:49:53.145265613 (21h10m27.660368179s) [upload=21886]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:49:53.145270200 (21h10m27.660372767s) [upload=21886]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:49:53.145274109 (21h10m27.660376686s) [upload=21886]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:49:53.205847742 (21h10m27.720950321s) [upload=21886]: Done uploading segments... Aug 8 06:49:53.205867747 (21h10m27.720970327s) [upload=21886]: Upload finished. err: <nil> Aug 8 06:50:45.962226830 (21h11m20.477329397s) [upload=21901]: Starting upload Aug 8 06:50:45.962960747 (21h11m20.47806339s) [upload=21901]: Done starting upload Aug 8 06:50:45.963081429 (21h11m20.478184007s) [upload=21901]: Uploading segments... Aug 8 06:50:45.963344944 (21h11m20.478447529s) [upload=21901]: Got next segment. Inline: true Aug 8 06:50:45.963354188 (21h11m20.478456768s) [upload=21901]: Waiting for error group managing segments... Aug 8 06:50:45.963408259 (21h11m20.478510836s) [upload=21901]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:50:45.963413437 (21h11m20.478516016s) [upload=21901]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:50:45.963418686 (21h11m20.478521263s) [upload=21901]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:50:46.026766649 (21h11m20.541869227s) [upload=21901]: Done uploading segments... Aug 8 06:50:46.026795865 (21h11m20.541898441s) [upload=21901]: Upload finished. err: <nil> Aug 8 06:51:58.480831248 (21h12m32.995933826s) [upload=21921]: Starting upload Aug 8 06:51:58.480895665 (21h12m32.995998248s) [upload=21921]: Done starting upload Aug 8 06:51:58.481023418 (21h12m32.996125988s) [upload=21921]: Uploading segments... Aug 8 06:51:58.481086289 (21h12m32.996188877s) [upload=21921]: Got next segment. Inline: true Aug 8 06:51:58.481091513 (21h12m32.99619409s) [upload=21921]: Waiting for error group managing segments... Aug 8 06:51:58.481149620 (21h12m32.996252197s) [upload=21921]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:51:58.481154789 (21h12m32.996257366s) [upload=21921]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:51:58.481158546 (21h12m32.996261124s) [upload=21921]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:51:58.722055331 (21h12m33.237157908s) [upload=21921]: Done uploading segments... Aug 8 06:51:58.722073114 (21h12m33.23717568s) [upload=21921]: Upload finished. err: <nil> Aug 8 06:52:27.059254149 (21h13m1.57435673s) [upload=21928]: Starting upload Aug 8 06:52:27.059321812 (21h13m1.574424392s) [upload=21928]: Done starting upload Aug 8 06:52:27.059401946 (21h13m1.574504512s) [upload=21928]: Uploading segments... Aug 8 06:52:27.059448476 (21h13m1.574551053s) [upload=21928]: Got next segment. Inline: true Aug 8 06:52:27.059454258 (21h13m1.574556833s) [upload=21928]: Waiting for error group managing segments... Aug 8 06:52:27.059501468 (21h13m1.574604035s) [upload=21928]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:52:27.059506838 (21h13m1.574609416s) [upload=21928]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:52:27.059510827 (21h13m1.574613404s) [upload=21928]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:52:27.120385847 (21h13m1.635488418s) [upload=21928]: Done uploading segments... Aug 8 06:52:27.120403445 (21h13m1.635506012s) [upload=21928]: Upload finished. err: <nil> Aug 8 06:52:45.219137365 (21h13m19.734239945s) [upload=21933]: Starting upload Aug 8 06:52:45.219214556 (21h13m19.734317133s) [upload=21933]: Done starting upload Aug 8 06:52:45.219284289 (21h13m19.734386877s) [upload=21933]: Uploading segments... Aug 8 06:52:45.219335736 (21h13m19.73443834s) [upload=21933]: Got next segment. Inline: true Aug 8 06:52:45.219342793 (21h13m19.734445378s) [upload=21933]: Waiting for error group managing segments... Aug 8 06:52:45.219397449 (21h13m19.734500028s) [upload=21933]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:52:45.219402775 (21h13m19.734505359s) [upload=21933]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:52:45.219407364 (21h13m19.734509958s) [upload=21933]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:52:45.282551475 (21h13m19.79765406s) [upload=21933]: Done uploading segments... Aug 8 06:52:45.282572182 (21h13m19.797674765s) [upload=21933]: Upload finished. err: <nil> Aug 8 06:53:36.462986837 (21h14m10.978089404s) [upload=21951]: Starting upload Aug 8 06:53:36.463070050 (21h14m10.978172618s) [upload=21951]: Done starting upload Aug 8 06:53:36.463162828 (21h14m10.978265396s) [upload=21951]: Uploading segments... Aug 8 06:53:36.463220839 (21h14m10.978323421s) [upload=21951]: Got next segment. Inline: true Aug 8 06:53:36.463227617 (21h14m10.978330198s) [upload=21951]: Waiting for error group managing segments... Aug 8 06:53:36.463290301 (21h14m10.978392878s) [upload=21951]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:53:36.463295593 (21h14m10.978398162s) [upload=21951]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:53:36.463299324 (21h14m10.978401891s) [upload=21951]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:53:36.745725330 (21h14m11.260827913s) [upload=21951]: Done uploading segments... Aug 8 06:53:36.745748524 (21h14m11.260851096s) [upload=21951]: Upload finished. err: <nil> Aug 8 06:54:42.211460075 (21h15m16.726562642s) [upload=21971]: Starting upload Aug 8 06:54:42.211544762 (21h15m16.726647342s) [upload=21971]: Done starting upload Aug 8 06:54:42.211670011 (21h15m16.726772576s) [upload=21971]: Uploading segments... Aug 8 06:54:42.211709612 (21h15m16.726812197s) [upload=21971]: Got next segment. Inline: true Aug 8 06:54:42.211714530 (21h15m16.726817106s) [upload=21971]: Waiting for error group managing segments... Aug 8 06:54:42.211773234 (21h15m16.726875811s) [upload=21971]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:54:42.211777907 (21h15m16.726880484s) [upload=21971]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:54:42.211781406 (21h15m16.726883982s) [upload=21971]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:54:42.285628155 (21h15m16.800730732s) [upload=21971]: Done uploading segments... Aug 8 06:54:42.285645434 (21h15m16.800748001s) [upload=21971]: Upload finished. err: <nil> Aug 8 06:55:07.196164830 (21h15m41.711267417s) [upload=21978]: Starting upload Aug 8 06:55:07.196239324 (21h15m41.711341924s) [upload=21978]: Done starting upload Aug 8 06:55:07.196316739 (21h15m41.711419326s) [upload=21978]: Uploading segments... Aug 8 06:55:07.196366345 (21h15m41.711468934s) [upload=21978]: Got next segment. Inline: true Aug 8 06:55:07.196372407 (21h15m41.711474978s) [upload=21978]: Waiting for error group managing segments... Aug 8 06:55:07.196426504 (21h15m41.711529089s) [upload=21978]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:55:07.196433124 (21h15m41.711535688s) [upload=21978]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:55:07.196438529 (21h15m41.711541093s) [upload=21978]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:55:07.258380171 (21h15m41.773482756s) [upload=21978]: Done uploading segments... Aug 8 06:55:07.258413055 (21h15m41.773515634s) [upload=21978]: Upload finished. err: <nil> Aug 8 06:56:17.251734653 (21h16m51.766837222s) [upload=21994]: Starting upload Aug 8 06:56:17.251784283 (21h16m51.766886851s) [upload=21994]: Done starting upload Aug 8 06:56:17.251882450 (21h16m51.766985019s) [upload=21994]: Uploading segments... Aug 8 06:56:17.251922005 (21h16m51.767024587s) [upload=21994]: Got next segment. Inline: true Aug 8 06:56:17.251926655 (21h16m51.767029231s) [upload=21994]: Waiting for error group managing segments... Aug 8 06:56:17.251978681 (21h16m51.767081246s) [upload=21994]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:56:17.251983158 (21h16m51.767085734s) [upload=21994]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:56:17.251986429 (21h16m51.767089005s) [upload=21994]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:56:17.318329259 (21h16m51.833431843s) [upload=21994]: Done uploading segments... Aug 8 06:56:17.318350310 (21h16m51.833452887s) [upload=21994]: Upload finished. err: <nil>
Duration: 187.202313ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [...............X............................................] duration:
202.054212ms
FAIL (2025-08-08 07:28:44 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:1024 (size:1024) ======LOGS====== d=21870]: Done uploading segments... Aug 8 06:48:53.275183069 (21h9m27.790285647s) [upload=21870]: Upload finished. err: <nil> Aug 8 06:49:08.545547742 (21h9m43.060650311s) [upload=21875]: Starting upload Aug 8 06:49:08.545594244 (21h9m43.06069682s) [upload=21875]: Done starting upload Aug 8 06:49:08.545651807 (21h9m43.060754385s) [upload=21875]: Uploading segments... Aug 8 06:49:08.545688834 (21h9m43.060791416s) [upload=21875]: Got next segment. Inline: true Aug 8 06:49:08.545693597 (21h9m43.060796163s) [upload=21875]: Waiting for error group managing segments... Aug 8 06:49:08.545736511 (21h9m43.060839081s) [upload=21875]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:49:08.545742589 (21h9m43.060845161s) [upload=21875]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:49:08.545747536 (21h9m43.060850111s) [upload=21875]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:49:08.608446156 (21h9m43.123548745s) [upload=21875]: Done uploading segments... Aug 8 06:49:08.608468645 (21h9m43.123571226s) [upload=21875]: Upload finished. err: <nil> Aug 8 06:49:53.145025965 (21h10m27.660128534s) [upload=21886]: Starting upload Aug 8 06:49:53.145086326 (21h10m27.660188904s) [upload=21886]: Done starting upload Aug 8 06:49:53.145166436 (21h10m27.660269004s) [upload=21886]: Uploading segments... Aug 8 06:49:53.145207405 (21h10m27.66030999s) [upload=21886]: Got next segment. Inline: true Aug 8 06:49:53.145212962 (21h10m27.660315539s) [upload=21886]: Waiting for error group managing segments... Aug 8 06:49:53.145265613 (21h10m27.660368179s) [upload=21886]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:49:53.145270200 (21h10m27.660372767s) [upload=21886]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:49:53.145274109 (21h10m27.660376686s) [upload=21886]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:49:53.205847742 (21h10m27.720950321s) [upload=21886]: Done uploading segments... Aug 8 06:49:53.205867747 (21h10m27.720970327s) [upload=21886]: Upload finished. err: <nil> Aug 8 06:50:45.962226830 (21h11m20.477329397s) [upload=21901]: Starting upload Aug 8 06:50:45.962960747 (21h11m20.47806339s) [upload=21901]: Done starting upload Aug 8 06:50:45.963081429 (21h11m20.478184007s) [upload=21901]: Uploading segments... Aug 8 06:50:45.963344944 (21h11m20.478447529s) [upload=21901]: Got next segment. Inline: true Aug 8 06:50:45.963354188 (21h11m20.478456768s) [upload=21901]: Waiting for error group managing segments... Aug 8 06:50:45.963408259 (21h11m20.478510836s) [upload=21901]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:50:45.963413437 (21h11m20.478516016s) [upload=21901]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:50:45.963418686 (21h11m20.478521263s) [upload=21901]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:50:46.026766649 (21h11m20.541869227s) [upload=21901]: Done uploading segments... Aug 8 06:50:46.026795865 (21h11m20.541898441s) [upload=21901]: Upload finished. err: <nil> Aug 8 06:51:58.480831248 (21h12m32.995933826s) [upload=21921]: Starting upload Aug 8 06:51:58.480895665 (21h12m32.995998248s) [upload=21921]: Done starting upload Aug 8 06:51:58.481023418 (21h12m32.996125988s) [upload=21921]: Uploading segments... Aug 8 06:51:58.481086289 (21h12m32.996188877s) [upload=21921]: Got next segment. Inline: true Aug 8 06:51:58.481091513 (21h12m32.99619409s) [upload=21921]: Waiting for error group managing segments... Aug 8 06:51:58.481149620 (21h12m32.996252197s) [upload=21921]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:51:58.481154789 (21h12m32.996257366s) [upload=21921]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:51:58.481158546 (21h12m32.996261124s) [upload=21921]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:51:58.722055331 (21h12m33.237157908s) [upload=21921]: Done uploading segments... Aug 8 06:51:58.722073114 (21h12m33.23717568s) [upload=21921]: Upload finished. err: <nil> Aug 8 06:52:27.059254149 (21h13m1.57435673s) [upload=21928]: Starting upload Aug 8 06:52:27.059321812 (21h13m1.574424392s) [upload=21928]: Done starting upload Aug 8 06:52:27.059401946 (21h13m1.574504512s) [upload=21928]: Uploading segments... Aug 8 06:52:27.059448476 (21h13m1.574551053s) [upload=21928]: Got next segment. Inline: true Aug 8 06:52:27.059454258 (21h13m1.574556833s) [upload=21928]: Waiting for error group managing segments... Aug 8 06:52:27.059501468 (21h13m1.574604035s) [upload=21928]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:52:27.059506838 (21h13m1.574609416s) [upload=21928]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:52:27.059510827 (21h13m1.574613404s) [upload=21928]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:52:27.120385847 (21h13m1.635488418s) [upload=21928]: Done uploading segments... Aug 8 06:52:27.120403445 (21h13m1.635506012s) [upload=21928]: Upload finished. err: <nil> Aug 8 06:52:45.219137365 (21h13m19.734239945s) [upload=21933]: Starting upload Aug 8 06:52:45.219214556 (21h13m19.734317133s) [upload=21933]: Done starting upload Aug 8 06:52:45.219284289 (21h13m19.734386877s) [upload=21933]: Uploading segments... Aug 8 06:52:45.219335736 (21h13m19.73443834s) [upload=21933]: Got next segment. Inline: true Aug 8 06:52:45.219342793 (21h13m19.734445378s) [upload=21933]: Waiting for error group managing segments... Aug 8 06:52:45.219397449 (21h13m19.734500028s) [upload=21933]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:52:45.219402775 (21h13m19.734505359s) [upload=21933]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:52:45.219407364 (21h13m19.734509958s) [upload=21933]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:52:45.282551475 (21h13m19.79765406s) [upload=21933]: Done uploading segments... Aug 8 06:52:45.282572182 (21h13m19.797674765s) [upload=21933]: Upload finished. err: <nil> Aug 8 06:53:36.462986837 (21h14m10.978089404s) [upload=21951]: Starting upload Aug 8 06:53:36.463070050 (21h14m10.978172618s) [upload=21951]: Done starting upload Aug 8 06:53:36.463162828 (21h14m10.978265396s) [upload=21951]: Uploading segments... Aug 8 06:53:36.463220839 (21h14m10.978323421s) [upload=21951]: Got next segment. Inline: true Aug 8 06:53:36.463227617 (21h14m10.978330198s) [upload=21951]: Waiting for error group managing segments... Aug 8 06:53:36.463290301 (21h14m10.978392878s) [upload=21951]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:53:36.463295593 (21h14m10.978398162s) [upload=21951]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:53:36.463299324 (21h14m10.978401891s) [upload=21951]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:53:36.745725330 (21h14m11.260827913s) [upload=21951]: Done uploading segments... Aug 8 06:53:36.745748524 (21h14m11.260851096s) [upload=21951]: Upload finished. err: <nil> Aug 8 06:54:42.211460075 (21h15m16.726562642s) [upload=21971]: Starting upload Aug 8 06:54:42.211544762 (21h15m16.726647342s) [upload=21971]: Done starting upload Aug 8 06:54:42.211670011 (21h15m16.726772576s) [upload=21971]: Uploading segments... Aug 8 06:54:42.211709612 (21h15m16.726812197s) [upload=21971]: Got next segment. Inline: true Aug 8 06:54:42.211714530 (21h15m16.726817106s) [upload=21971]: Waiting for error group managing segments... Aug 8 06:54:42.211773234 (21h15m16.726875811s) [upload=21971]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:54:42.211777907 (21h15m16.726880484s) [upload=21971]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:54:42.211781406 (21h15m16.726883982s) [upload=21971]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:54:42.285628155 (21h15m16.800730732s) [upload=21971]: Done uploading segments... Aug 8 06:54:42.285645434 (21h15m16.800748001s) [upload=21971]: Upload finished. err: <nil> Aug 8 06:55:07.196164830 (21h15m41.711267417s) [upload=21978]: Starting upload Aug 8 06:55:07.196239324 (21h15m41.711341924s) [upload=21978]: Done starting upload Aug 8 06:55:07.196316739 (21h15m41.711419326s) [upload=21978]: Uploading segments... Aug 8 06:55:07.196366345 (21h15m41.711468934s) [upload=21978]: Got next segment. Inline: true Aug 8 06:55:07.196372407 (21h15m41.711474978s) [upload=21978]: Waiting for error group managing segments... Aug 8 06:55:07.196426504 (21h15m41.711529089s) [upload=21978]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:55:07.196433124 (21h15m41.711535688s) [upload=21978]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:55:07.196438529 (21h15m41.711541093s) [upload=21978]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:55:07.258380171 (21h15m41.773482756s) [upload=21978]: Done uploading segments... Aug 8 06:55:07.258413055 (21h15m41.773515634s) [upload=21978]: Upload finished. err: <nil> Aug 8 06:56:17.251734653 (21h16m51.766837222s) [upload=21994]: Starting upload Aug 8 06:56:17.251784283 (21h16m51.766886851s) [upload=21994]: Done starting upload Aug 8 06:56:17.251882450 (21h16m51.766985019s) [upload=21994]: Uploading segments... Aug 8 06:56:17.251922005 (21h16m51.767024587s) [upload=21994]: Got next segment. Inline: true Aug 8 06:56:17.251926655 (21h16m51.767029231s) [upload=21994]: Waiting for error group managing segments... Aug 8 06:56:17.251978681 (21h16m51.767081246s) [upload=21994]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:56:17.251983158 (21h16m51.767085734s) [upload=21994]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:56:17.251986429 (21h16m51.767089005s) [upload=21994]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:56:17.318329259 (21h16m51.833431843s) [upload=21994]: Done uploading segments... Aug 8 06:56:17.318350310 (21h16m51.833452887s) [upload=21994]: Upload finished. err: <nil>
Duration: 202.080856ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [................X...........................................] duration:
424.787606ms
FAIL (2025-08-08 07:28:26 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:1024 (size:1024) ======LOGS====== d=21870]: Done uploading segments... Aug 8 06:48:53.275183069 (21h9m27.790285647s) [upload=21870]: Upload finished. err: <nil> Aug 8 06:49:08.545547742 (21h9m43.060650311s) [upload=21875]: Starting upload Aug 8 06:49:08.545594244 (21h9m43.06069682s) [upload=21875]: Done starting upload Aug 8 06:49:08.545651807 (21h9m43.060754385s) [upload=21875]: Uploading segments... Aug 8 06:49:08.545688834 (21h9m43.060791416s) [upload=21875]: Got next segment. Inline: true Aug 8 06:49:08.545693597 (21h9m43.060796163s) [upload=21875]: Waiting for error group managing segments... Aug 8 06:49:08.545736511 (21h9m43.060839081s) [upload=21875]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:49:08.545742589 (21h9m43.060845161s) [upload=21875]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:49:08.545747536 (21h9m43.060850111s) [upload=21875]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:49:08.608446156 (21h9m43.123548745s) [upload=21875]: Done uploading segments... Aug 8 06:49:08.608468645 (21h9m43.123571226s) [upload=21875]: Upload finished. err: <nil> Aug 8 06:49:53.145025965 (21h10m27.660128534s) [upload=21886]: Starting upload Aug 8 06:49:53.145086326 (21h10m27.660188904s) [upload=21886]: Done starting upload Aug 8 06:49:53.145166436 (21h10m27.660269004s) [upload=21886]: Uploading segments... Aug 8 06:49:53.145207405 (21h10m27.66030999s) [upload=21886]: Got next segment. Inline: true Aug 8 06:49:53.145212962 (21h10m27.660315539s) [upload=21886]: Waiting for error group managing segments... Aug 8 06:49:53.145265613 (21h10m27.660368179s) [upload=21886]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:49:53.145270200 (21h10m27.660372767s) [upload=21886]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:49:53.145274109 (21h10m27.660376686s) [upload=21886]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:49:53.205847742 (21h10m27.720950321s) [upload=21886]: Done uploading segments... Aug 8 06:49:53.205867747 (21h10m27.720970327s) [upload=21886]: Upload finished. err: <nil> Aug 8 06:50:45.962226830 (21h11m20.477329397s) [upload=21901]: Starting upload Aug 8 06:50:45.962960747 (21h11m20.47806339s) [upload=21901]: Done starting upload Aug 8 06:50:45.963081429 (21h11m20.478184007s) [upload=21901]: Uploading segments... Aug 8 06:50:45.963344944 (21h11m20.478447529s) [upload=21901]: Got next segment. Inline: true Aug 8 06:50:45.963354188 (21h11m20.478456768s) [upload=21901]: Waiting for error group managing segments... Aug 8 06:50:45.963408259 (21h11m20.478510836s) [upload=21901]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:50:45.963413437 (21h11m20.478516016s) [upload=21901]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:50:45.963418686 (21h11m20.478521263s) [upload=21901]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:50:46.026766649 (21h11m20.541869227s) [upload=21901]: Done uploading segments... Aug 8 06:50:46.026795865 (21h11m20.541898441s) [upload=21901]: Upload finished. err: <nil> Aug 8 06:51:58.480831248 (21h12m32.995933826s) [upload=21921]: Starting upload Aug 8 06:51:58.480895665 (21h12m32.995998248s) [upload=21921]: Done starting upload Aug 8 06:51:58.481023418 (21h12m32.996125988s) [upload=21921]: Uploading segments... Aug 8 06:51:58.481086289 (21h12m32.996188877s) [upload=21921]: Got next segment. Inline: true Aug 8 06:51:58.481091513 (21h12m32.99619409s) [upload=21921]: Waiting for error group managing segments... Aug 8 06:51:58.481149620 (21h12m32.996252197s) [upload=21921]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:51:58.481154789 (21h12m32.996257366s) [upload=21921]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:51:58.481158546 (21h12m32.996261124s) [upload=21921]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:51:58.722055331 (21h12m33.237157908s) [upload=21921]: Done uploading segments... Aug 8 06:51:58.722073114 (21h12m33.23717568s) [upload=21921]: Upload finished. err: <nil> Aug 8 06:52:27.059254149 (21h13m1.57435673s) [upload=21928]: Starting upload Aug 8 06:52:27.059321812 (21h13m1.574424392s) [upload=21928]: Done starting upload Aug 8 06:52:27.059401946 (21h13m1.574504512s) [upload=21928]: Uploading segments... Aug 8 06:52:27.059448476 (21h13m1.574551053s) [upload=21928]: Got next segment. Inline: true Aug 8 06:52:27.059454258 (21h13m1.574556833s) [upload=21928]: Waiting for error group managing segments... Aug 8 06:52:27.059501468 (21h13m1.574604035s) [upload=21928]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:52:27.059506838 (21h13m1.574609416s) [upload=21928]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:52:27.059510827 (21h13m1.574613404s) [upload=21928]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:52:27.120385847 (21h13m1.635488418s) [upload=21928]: Done uploading segments... Aug 8 06:52:27.120403445 (21h13m1.635506012s) [upload=21928]: Upload finished. err: <nil> Aug 8 06:52:45.219137365 (21h13m19.734239945s) [upload=21933]: Starting upload Aug 8 06:52:45.219214556 (21h13m19.734317133s) [upload=21933]: Done starting upload Aug 8 06:52:45.219284289 (21h13m19.734386877s) [upload=21933]: Uploading segments... Aug 8 06:52:45.219335736 (21h13m19.73443834s) [upload=21933]: Got next segment. Inline: true Aug 8 06:52:45.219342793 (21h13m19.734445378s) [upload=21933]: Waiting for error group managing segments... Aug 8 06:52:45.219397449 (21h13m19.734500028s) [upload=21933]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:52:45.219402775 (21h13m19.734505359s) [upload=21933]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:52:45.219407364 (21h13m19.734509958s) [upload=21933]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:52:45.282551475 (21h13m19.79765406s) [upload=21933]: Done uploading segments... Aug 8 06:52:45.282572182 (21h13m19.797674765s) [upload=21933]: Upload finished. err: <nil> Aug 8 06:53:36.462986837 (21h14m10.978089404s) [upload=21951]: Starting upload Aug 8 06:53:36.463070050 (21h14m10.978172618s) [upload=21951]: Done starting upload Aug 8 06:53:36.463162828 (21h14m10.978265396s) [upload=21951]: Uploading segments... Aug 8 06:53:36.463220839 (21h14m10.978323421s) [upload=21951]: Got next segment. Inline: true Aug 8 06:53:36.463227617 (21h14m10.978330198s) [upload=21951]: Waiting for error group managing segments... Aug 8 06:53:36.463290301 (21h14m10.978392878s) [upload=21951]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:53:36.463295593 (21h14m10.978398162s) [upload=21951]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:53:36.463299324 (21h14m10.978401891s) [upload=21951]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:53:36.745725330 (21h14m11.260827913s) [upload=21951]: Done uploading segments... Aug 8 06:53:36.745748524 (21h14m11.260851096s) [upload=21951]: Upload finished. err: <nil> Aug 8 06:54:42.211460075 (21h15m16.726562642s) [upload=21971]: Starting upload Aug 8 06:54:42.211544762 (21h15m16.726647342s) [upload=21971]: Done starting upload Aug 8 06:54:42.211670011 (21h15m16.726772576s) [upload=21971]: Uploading segments... Aug 8 06:54:42.211709612 (21h15m16.726812197s) [upload=21971]: Got next segment. Inline: true Aug 8 06:54:42.211714530 (21h15m16.726817106s) [upload=21971]: Waiting for error group managing segments... Aug 8 06:54:42.211773234 (21h15m16.726875811s) [upload=21971]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:54:42.211777907 (21h15m16.726880484s) [upload=21971]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:54:42.211781406 (21h15m16.726883982s) [upload=21971]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:54:42.285628155 (21h15m16.800730732s) [upload=21971]: Done uploading segments... Aug 8 06:54:42.285645434 (21h15m16.800748001s) [upload=21971]: Upload finished. err: <nil> Aug 8 06:55:07.196164830 (21h15m41.711267417s) [upload=21978]: Starting upload Aug 8 06:55:07.196239324 (21h15m41.711341924s) [upload=21978]: Done starting upload Aug 8 06:55:07.196316739 (21h15m41.711419326s) [upload=21978]: Uploading segments... Aug 8 06:55:07.196366345 (21h15m41.711468934s) [upload=21978]: Got next segment. Inline: true Aug 8 06:55:07.196372407 (21h15m41.711474978s) [upload=21978]: Waiting for error group managing segments... Aug 8 06:55:07.196426504 (21h15m41.711529089s) [upload=21978]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:55:07.196433124 (21h15m41.711535688s) [upload=21978]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:55:07.196438529 (21h15m41.711541093s) [upload=21978]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:55:07.258380171 (21h15m41.773482756s) [upload=21978]: Done uploading segments... Aug 8 06:55:07.258413055 (21h15m41.773515634s) [upload=21978]: Upload finished. err: <nil> Aug 8 06:56:17.251734653 (21h16m51.766837222s) [upload=21994]: Starting upload Aug 8 06:56:17.251784283 (21h16m51.766886851s) [upload=21994]: Done starting upload Aug 8 06:56:17.251882450 (21h16m51.766985019s) [upload=21994]: Uploading segments... Aug 8 06:56:17.251922005 (21h16m51.767024587s) [upload=21994]: Got next segment. Inline: true Aug 8 06:56:17.251926655 (21h16m51.767029231s) [upload=21994]: Waiting for error group managing segments... Aug 8 06:56:17.251978681 (21h16m51.767081246s) [upload=21994]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:56:17.251983158 (21h16m51.767085734s) [upload=21994]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:56:17.251986429 (21h16m51.767089005s) [upload=21994]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:56:17.318329259 (21h16m51.833431843s) [upload=21994]: Done uploading segments... Aug 8 06:56:17.318350310 (21h16m51.833452887s) [upload=21994]: Upload finished. err: <nil>
Duration: 424.814793ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [.................X..........................................] duration:
185.955339ms
FAIL (2025-08-08 07:28:08 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:1024 (size:1024) ======LOGS====== d=21870]: Done uploading segments... Aug 8 06:48:53.275183069 (21h9m27.790285647s) [upload=21870]: Upload finished. err: <nil> Aug 8 06:49:08.545547742 (21h9m43.060650311s) [upload=21875]: Starting upload Aug 8 06:49:08.545594244 (21h9m43.06069682s) [upload=21875]: Done starting upload Aug 8 06:49:08.545651807 (21h9m43.060754385s) [upload=21875]: Uploading segments... Aug 8 06:49:08.545688834 (21h9m43.060791416s) [upload=21875]: Got next segment. Inline: true Aug 8 06:49:08.545693597 (21h9m43.060796163s) [upload=21875]: Waiting for error group managing segments... Aug 8 06:49:08.545736511 (21h9m43.060839081s) [upload=21875]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:49:08.545742589 (21h9m43.060845161s) [upload=21875]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:49:08.545747536 (21h9m43.060850111s) [upload=21875]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:49:08.608446156 (21h9m43.123548745s) [upload=21875]: Done uploading segments... Aug 8 06:49:08.608468645 (21h9m43.123571226s) [upload=21875]: Upload finished. err: <nil> Aug 8 06:49:53.145025965 (21h10m27.660128534s) [upload=21886]: Starting upload Aug 8 06:49:53.145086326 (21h10m27.660188904s) [upload=21886]: Done starting upload Aug 8 06:49:53.145166436 (21h10m27.660269004s) [upload=21886]: Uploading segments... Aug 8 06:49:53.145207405 (21h10m27.66030999s) [upload=21886]: Got next segment. Inline: true Aug 8 06:49:53.145212962 (21h10m27.660315539s) [upload=21886]: Waiting for error group managing segments... Aug 8 06:49:53.145265613 (21h10m27.660368179s) [upload=21886]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:49:53.145270200 (21h10m27.660372767s) [upload=21886]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:49:53.145274109 (21h10m27.660376686s) [upload=21886]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:49:53.205847742 (21h10m27.720950321s) [upload=21886]: Done uploading segments... Aug 8 06:49:53.205867747 (21h10m27.720970327s) [upload=21886]: Upload finished. err: <nil> Aug 8 06:50:45.962226830 (21h11m20.477329397s) [upload=21901]: Starting upload Aug 8 06:50:45.962960747 (21h11m20.47806339s) [upload=21901]: Done starting upload Aug 8 06:50:45.963081429 (21h11m20.478184007s) [upload=21901]: Uploading segments... Aug 8 06:50:45.963344944 (21h11m20.478447529s) [upload=21901]: Got next segment. Inline: true Aug 8 06:50:45.963354188 (21h11m20.478456768s) [upload=21901]: Waiting for error group managing segments... Aug 8 06:50:45.963408259 (21h11m20.478510836s) [upload=21901]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:50:45.963413437 (21h11m20.478516016s) [upload=21901]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:50:45.963418686 (21h11m20.478521263s) [upload=21901]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:50:46.026766649 (21h11m20.541869227s) [upload=21901]: Done uploading segments... Aug 8 06:50:46.026795865 (21h11m20.541898441s) [upload=21901]: Upload finished. err: <nil> Aug 8 06:51:58.480831248 (21h12m32.995933826s) [upload=21921]: Starting upload Aug 8 06:51:58.480895665 (21h12m32.995998248s) [upload=21921]: Done starting upload Aug 8 06:51:58.481023418 (21h12m32.996125988s) [upload=21921]: Uploading segments... Aug 8 06:51:58.481086289 (21h12m32.996188877s) [upload=21921]: Got next segment. Inline: true Aug 8 06:51:58.481091513 (21h12m32.99619409s) [upload=21921]: Waiting for error group managing segments... Aug 8 06:51:58.481149620 (21h12m32.996252197s) [upload=21921]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:51:58.481154789 (21h12m32.996257366s) [upload=21921]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:51:58.481158546 (21h12m32.996261124s) [upload=21921]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:51:58.722055331 (21h12m33.237157908s) [upload=21921]: Done uploading segments... Aug 8 06:51:58.722073114 (21h12m33.23717568s) [upload=21921]: Upload finished. err: <nil> Aug 8 06:52:27.059254149 (21h13m1.57435673s) [upload=21928]: Starting upload Aug 8 06:52:27.059321812 (21h13m1.574424392s) [upload=21928]: Done starting upload Aug 8 06:52:27.059401946 (21h13m1.574504512s) [upload=21928]: Uploading segments... Aug 8 06:52:27.059448476 (21h13m1.574551053s) [upload=21928]: Got next segment. Inline: true Aug 8 06:52:27.059454258 (21h13m1.574556833s) [upload=21928]: Waiting for error group managing segments... Aug 8 06:52:27.059501468 (21h13m1.574604035s) [upload=21928]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:52:27.059506838 (21h13m1.574609416s) [upload=21928]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:52:27.059510827 (21h13m1.574613404s) [upload=21928]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:52:27.120385847 (21h13m1.635488418s) [upload=21928]: Done uploading segments... Aug 8 06:52:27.120403445 (21h13m1.635506012s) [upload=21928]: Upload finished. err: <nil> Aug 8 06:52:45.219137365 (21h13m19.734239945s) [upload=21933]: Starting upload Aug 8 06:52:45.219214556 (21h13m19.734317133s) [upload=21933]: Done starting upload Aug 8 06:52:45.219284289 (21h13m19.734386877s) [upload=21933]: Uploading segments... Aug 8 06:52:45.219335736 (21h13m19.73443834s) [upload=21933]: Got next segment. Inline: true Aug 8 06:52:45.219342793 (21h13m19.734445378s) [upload=21933]: Waiting for error group managing segments... Aug 8 06:52:45.219397449 (21h13m19.734500028s) [upload=21933]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:52:45.219402775 (21h13m19.734505359s) [upload=21933]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:52:45.219407364 (21h13m19.734509958s) [upload=21933]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:52:45.282551475 (21h13m19.79765406s) [upload=21933]: Done uploading segments... Aug 8 06:52:45.282572182 (21h13m19.797674765s) [upload=21933]: Upload finished. err: <nil> Aug 8 06:53:36.462986837 (21h14m10.978089404s) [upload=21951]: Starting upload Aug 8 06:53:36.463070050 (21h14m10.978172618s) [upload=21951]: Done starting upload Aug 8 06:53:36.463162828 (21h14m10.978265396s) [upload=21951]: Uploading segments... Aug 8 06:53:36.463220839 (21h14m10.978323421s) [upload=21951]: Got next segment. Inline: true Aug 8 06:53:36.463227617 (21h14m10.978330198s) [upload=21951]: Waiting for error group managing segments... Aug 8 06:53:36.463290301 (21h14m10.978392878s) [upload=21951]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:53:36.463295593 (21h14m10.978398162s) [upload=21951]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:53:36.463299324 (21h14m10.978401891s) [upload=21951]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:53:36.745725330 (21h14m11.260827913s) [upload=21951]: Done uploading segments... Aug 8 06:53:36.745748524 (21h14m11.260851096s) [upload=21951]: Upload finished. err: <nil> Aug 8 06:54:42.211460075 (21h15m16.726562642s) [upload=21971]: Starting upload Aug 8 06:54:42.211544762 (21h15m16.726647342s) [upload=21971]: Done starting upload Aug 8 06:54:42.211670011 (21h15m16.726772576s) [upload=21971]: Uploading segments... Aug 8 06:54:42.211709612 (21h15m16.726812197s) [upload=21971]: Got next segment. Inline: true Aug 8 06:54:42.211714530 (21h15m16.726817106s) [upload=21971]: Waiting for error group managing segments... Aug 8 06:54:42.211773234 (21h15m16.726875811s) [upload=21971]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:54:42.211777907 (21h15m16.726880484s) [upload=21971]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:54:42.211781406 (21h15m16.726883982s) [upload=21971]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:54:42.285628155 (21h15m16.800730732s) [upload=21971]: Done uploading segments... Aug 8 06:54:42.285645434 (21h15m16.800748001s) [upload=21971]: Upload finished. err: <nil> Aug 8 06:55:07.196164830 (21h15m41.711267417s) [upload=21978]: Starting upload Aug 8 06:55:07.196239324 (21h15m41.711341924s) [upload=21978]: Done starting upload Aug 8 06:55:07.196316739 (21h15m41.711419326s) [upload=21978]: Uploading segments... Aug 8 06:55:07.196366345 (21h15m41.711468934s) [upload=21978]: Got next segment. Inline: true Aug 8 06:55:07.196372407 (21h15m41.711474978s) [upload=21978]: Waiting for error group managing segments... Aug 8 06:55:07.196426504 (21h15m41.711529089s) [upload=21978]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:55:07.196433124 (21h15m41.711535688s) [upload=21978]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:55:07.196438529 (21h15m41.711541093s) [upload=21978]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:55:07.258380171 (21h15m41.773482756s) [upload=21978]: Done uploading segments... Aug 8 06:55:07.258413055 (21h15m41.773515634s) [upload=21978]: Upload finished. err: <nil> Aug 8 06:56:17.251734653 (21h16m51.766837222s) [upload=21994]: Starting upload Aug 8 06:56:17.251784283 (21h16m51.766886851s) [upload=21994]: Done starting upload Aug 8 06:56:17.251882450 (21h16m51.766985019s) [upload=21994]: Uploading segments... Aug 8 06:56:17.251922005 (21h16m51.767024587s) [upload=21994]: Got next segment. Inline: true Aug 8 06:56:17.251926655 (21h16m51.767029231s) [upload=21994]: Waiting for error group managing segments... Aug 8 06:56:17.251978681 (21h16m51.767081246s) [upload=21994]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:56:17.251983158 (21h16m51.767085734s) [upload=21994]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:56:17.251986429 (21h16m51.767089005s) [upload=21994]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:56:17.318329259 (21h16m51.833431843s) [upload=21994]: Done uploading segments... Aug 8 06:56:17.318350310 (21h16m51.833452887s) [upload=21994]: Upload finished. err: <nil>
Duration: 185.982221ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [..................X.........................................] duration:
188.152831ms
FAIL (2025-08-08 07:27:50 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:1024 (size:1024) ======LOGS====== d=21870]: Done uploading segments... Aug 8 06:48:53.275183069 (21h9m27.790285647s) [upload=21870]: Upload finished. err: <nil> Aug 8 06:49:08.545547742 (21h9m43.060650311s) [upload=21875]: Starting upload Aug 8 06:49:08.545594244 (21h9m43.06069682s) [upload=21875]: Done starting upload Aug 8 06:49:08.545651807 (21h9m43.060754385s) [upload=21875]: Uploading segments... Aug 8 06:49:08.545688834 (21h9m43.060791416s) [upload=21875]: Got next segment. Inline: true Aug 8 06:49:08.545693597 (21h9m43.060796163s) [upload=21875]: Waiting for error group managing segments... Aug 8 06:49:08.545736511 (21h9m43.060839081s) [upload=21875]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:49:08.545742589 (21h9m43.060845161s) [upload=21875]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:49:08.545747536 (21h9m43.060850111s) [upload=21875]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:49:08.608446156 (21h9m43.123548745s) [upload=21875]: Done uploading segments... Aug 8 06:49:08.608468645 (21h9m43.123571226s) [upload=21875]: Upload finished. err: <nil> Aug 8 06:49:53.145025965 (21h10m27.660128534s) [upload=21886]: Starting upload Aug 8 06:49:53.145086326 (21h10m27.660188904s) [upload=21886]: Done starting upload Aug 8 06:49:53.145166436 (21h10m27.660269004s) [upload=21886]: Uploading segments... Aug 8 06:49:53.145207405 (21h10m27.66030999s) [upload=21886]: Got next segment. Inline: true Aug 8 06:49:53.145212962 (21h10m27.660315539s) [upload=21886]: Waiting for error group managing segments... Aug 8 06:49:53.145265613 (21h10m27.660368179s) [upload=21886]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:49:53.145270200 (21h10m27.660372767s) [upload=21886]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:49:53.145274109 (21h10m27.660376686s) [upload=21886]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:49:53.205847742 (21h10m27.720950321s) [upload=21886]: Done uploading segments... Aug 8 06:49:53.205867747 (21h10m27.720970327s) [upload=21886]: Upload finished. err: <nil> Aug 8 06:50:45.962226830 (21h11m20.477329397s) [upload=21901]: Starting upload Aug 8 06:50:45.962960747 (21h11m20.47806339s) [upload=21901]: Done starting upload Aug 8 06:50:45.963081429 (21h11m20.478184007s) [upload=21901]: Uploading segments... Aug 8 06:50:45.963344944 (21h11m20.478447529s) [upload=21901]: Got next segment. Inline: true Aug 8 06:50:45.963354188 (21h11m20.478456768s) [upload=21901]: Waiting for error group managing segments... Aug 8 06:50:45.963408259 (21h11m20.478510836s) [upload=21901]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:50:45.963413437 (21h11m20.478516016s) [upload=21901]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:50:45.963418686 (21h11m20.478521263s) [upload=21901]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:50:46.026766649 (21h11m20.541869227s) [upload=21901]: Done uploading segments... Aug 8 06:50:46.026795865 (21h11m20.541898441s) [upload=21901]: Upload finished. err: <nil> Aug 8 06:51:58.480831248 (21h12m32.995933826s) [upload=21921]: Starting upload Aug 8 06:51:58.480895665 (21h12m32.995998248s) [upload=21921]: Done starting upload Aug 8 06:51:58.481023418 (21h12m32.996125988s) [upload=21921]: Uploading segments... Aug 8 06:51:58.481086289 (21h12m32.996188877s) [upload=21921]: Got next segment. Inline: true Aug 8 06:51:58.481091513 (21h12m32.99619409s) [upload=21921]: Waiting for error group managing segments... Aug 8 06:51:58.481149620 (21h12m32.996252197s) [upload=21921]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:51:58.481154789 (21h12m32.996257366s) [upload=21921]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:51:58.481158546 (21h12m32.996261124s) [upload=21921]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:51:58.722055331 (21h12m33.237157908s) [upload=21921]: Done uploading segments... Aug 8 06:51:58.722073114 (21h12m33.23717568s) [upload=21921]: Upload finished. err: <nil> Aug 8 06:52:27.059254149 (21h13m1.57435673s) [upload=21928]: Starting upload Aug 8 06:52:27.059321812 (21h13m1.574424392s) [upload=21928]: Done starting upload Aug 8 06:52:27.059401946 (21h13m1.574504512s) [upload=21928]: Uploading segments... Aug 8 06:52:27.059448476 (21h13m1.574551053s) [upload=21928]: Got next segment. Inline: true Aug 8 06:52:27.059454258 (21h13m1.574556833s) [upload=21928]: Waiting for error group managing segments... Aug 8 06:52:27.059501468 (21h13m1.574604035s) [upload=21928]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:52:27.059506838 (21h13m1.574609416s) [upload=21928]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:52:27.059510827 (21h13m1.574613404s) [upload=21928]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:52:27.120385847 (21h13m1.635488418s) [upload=21928]: Done uploading segments... Aug 8 06:52:27.120403445 (21h13m1.635506012s) [upload=21928]: Upload finished. err: <nil> Aug 8 06:52:45.219137365 (21h13m19.734239945s) [upload=21933]: Starting upload Aug 8 06:52:45.219214556 (21h13m19.734317133s) [upload=21933]: Done starting upload Aug 8 06:52:45.219284289 (21h13m19.734386877s) [upload=21933]: Uploading segments... Aug 8 06:52:45.219335736 (21h13m19.73443834s) [upload=21933]: Got next segment. Inline: true Aug 8 06:52:45.219342793 (21h13m19.734445378s) [upload=21933]: Waiting for error group managing segments... Aug 8 06:52:45.219397449 (21h13m19.734500028s) [upload=21933]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:52:45.219402775 (21h13m19.734505359s) [upload=21933]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:52:45.219407364 (21h13m19.734509958s) [upload=21933]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:52:45.282551475 (21h13m19.79765406s) [upload=21933]: Done uploading segments... Aug 8 06:52:45.282572182 (21h13m19.797674765s) [upload=21933]: Upload finished. err: <nil> Aug 8 06:53:36.462986837 (21h14m10.978089404s) [upload=21951]: Starting upload Aug 8 06:53:36.463070050 (21h14m10.978172618s) [upload=21951]: Done starting upload Aug 8 06:53:36.463162828 (21h14m10.978265396s) [upload=21951]: Uploading segments... Aug 8 06:53:36.463220839 (21h14m10.978323421s) [upload=21951]: Got next segment. Inline: true Aug 8 06:53:36.463227617 (21h14m10.978330198s) [upload=21951]: Waiting for error group managing segments... Aug 8 06:53:36.463290301 (21h14m10.978392878s) [upload=21951]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:53:36.463295593 (21h14m10.978398162s) [upload=21951]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:53:36.463299324 (21h14m10.978401891s) [upload=21951]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:53:36.745725330 (21h14m11.260827913s) [upload=21951]: Done uploading segments... Aug 8 06:53:36.745748524 (21h14m11.260851096s) [upload=21951]: Upload finished. err: <nil> Aug 8 06:54:42.211460075 (21h15m16.726562642s) [upload=21971]: Starting upload Aug 8 06:54:42.211544762 (21h15m16.726647342s) [upload=21971]: Done starting upload Aug 8 06:54:42.211670011 (21h15m16.726772576s) [upload=21971]: Uploading segments... Aug 8 06:54:42.211709612 (21h15m16.726812197s) [upload=21971]: Got next segment. Inline: true Aug 8 06:54:42.211714530 (21h15m16.726817106s) [upload=21971]: Waiting for error group managing segments... Aug 8 06:54:42.211773234 (21h15m16.726875811s) [upload=21971]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:54:42.211777907 (21h15m16.726880484s) [upload=21971]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:54:42.211781406 (21h15m16.726883982s) [upload=21971]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:54:42.285628155 (21h15m16.800730732s) [upload=21971]: Done uploading segments... Aug 8 06:54:42.285645434 (21h15m16.800748001s) [upload=21971]: Upload finished. err: <nil> Aug 8 06:55:07.196164830 (21h15m41.711267417s) [upload=21978]: Starting upload Aug 8 06:55:07.196239324 (21h15m41.711341924s) [upload=21978]: Done starting upload Aug 8 06:55:07.196316739 (21h15m41.711419326s) [upload=21978]: Uploading segments... Aug 8 06:55:07.196366345 (21h15m41.711468934s) [upload=21978]: Got next segment. Inline: true Aug 8 06:55:07.196372407 (21h15m41.711474978s) [upload=21978]: Waiting for error group managing segments... Aug 8 06:55:07.196426504 (21h15m41.711529089s) [upload=21978]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:55:07.196433124 (21h15m41.711535688s) [upload=21978]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:55:07.196438529 (21h15m41.711541093s) [upload=21978]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:55:07.258380171 (21h15m41.773482756s) [upload=21978]: Done uploading segments... Aug 8 06:55:07.258413055 (21h15m41.773515634s) [upload=21978]: Upload finished. err: <nil> Aug 8 06:56:17.251734653 (21h16m51.766837222s) [upload=21994]: Starting upload Aug 8 06:56:17.251784283 (21h16m51.766886851s) [upload=21994]: Done starting upload Aug 8 06:56:17.251882450 (21h16m51.766985019s) [upload=21994]: Uploading segments... Aug 8 06:56:17.251922005 (21h16m51.767024587s) [upload=21994]: Got next segment. Inline: true Aug 8 06:56:17.251926655 (21h16m51.767029231s) [upload=21994]: Waiting for error group managing segments... Aug 8 06:56:17.251978681 (21h16m51.767081246s) [upload=21994]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:56:17.251983158 (21h16m51.767085734s) [upload=21994]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:56:17.251986429 (21h16m51.767089005s) [upload=21994]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:56:17.318329259 (21h16m51.833431843s) [upload=21994]: Done uploading segments... Aug 8 06:56:17.318350310 (21h16m51.833452887s) [upload=21994]: Upload finished. err: <nil>
Duration: 188.180971ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [...................X........................................] duration:
190.316901ms
FAIL (2025-08-08 07:26:45 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:1024 (size:1024) ======LOGS====== d=21870]: Done uploading segments... Aug 8 06:48:53.275183069 (21h9m27.790285647s) [upload=21870]: Upload finished. err: <nil> Aug 8 06:49:08.545547742 (21h9m43.060650311s) [upload=21875]: Starting upload Aug 8 06:49:08.545594244 (21h9m43.06069682s) [upload=21875]: Done starting upload Aug 8 06:49:08.545651807 (21h9m43.060754385s) [upload=21875]: Uploading segments... Aug 8 06:49:08.545688834 (21h9m43.060791416s) [upload=21875]: Got next segment. Inline: true Aug 8 06:49:08.545693597 (21h9m43.060796163s) [upload=21875]: Waiting for error group managing segments... Aug 8 06:49:08.545736511 (21h9m43.060839081s) [upload=21875]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:49:08.545742589 (21h9m43.060845161s) [upload=21875]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:49:08.545747536 (21h9m43.060850111s) [upload=21875]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:49:08.608446156 (21h9m43.123548745s) [upload=21875]: Done uploading segments... Aug 8 06:49:08.608468645 (21h9m43.123571226s) [upload=21875]: Upload finished. err: <nil> Aug 8 06:49:53.145025965 (21h10m27.660128534s) [upload=21886]: Starting upload Aug 8 06:49:53.145086326 (21h10m27.660188904s) [upload=21886]: Done starting upload Aug 8 06:49:53.145166436 (21h10m27.660269004s) [upload=21886]: Uploading segments... Aug 8 06:49:53.145207405 (21h10m27.66030999s) [upload=21886]: Got next segment. Inline: true Aug 8 06:49:53.145212962 (21h10m27.660315539s) [upload=21886]: Waiting for error group managing segments... Aug 8 06:49:53.145265613 (21h10m27.660368179s) [upload=21886]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:49:53.145270200 (21h10m27.660372767s) [upload=21886]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:49:53.145274109 (21h10m27.660376686s) [upload=21886]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:49:53.205847742 (21h10m27.720950321s) [upload=21886]: Done uploading segments... Aug 8 06:49:53.205867747 (21h10m27.720970327s) [upload=21886]: Upload finished. err: <nil> Aug 8 06:50:45.962226830 (21h11m20.477329397s) [upload=21901]: Starting upload Aug 8 06:50:45.962960747 (21h11m20.47806339s) [upload=21901]: Done starting upload Aug 8 06:50:45.963081429 (21h11m20.478184007s) [upload=21901]: Uploading segments... Aug 8 06:50:45.963344944 (21h11m20.478447529s) [upload=21901]: Got next segment. Inline: true Aug 8 06:50:45.963354188 (21h11m20.478456768s) [upload=21901]: Waiting for error group managing segments... Aug 8 06:50:45.963408259 (21h11m20.478510836s) [upload=21901]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:50:45.963413437 (21h11m20.478516016s) [upload=21901]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:50:45.963418686 (21h11m20.478521263s) [upload=21901]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:50:46.026766649 (21h11m20.541869227s) [upload=21901]: Done uploading segments... Aug 8 06:50:46.026795865 (21h11m20.541898441s) [upload=21901]: Upload finished. err: <nil> Aug 8 06:51:58.480831248 (21h12m32.995933826s) [upload=21921]: Starting upload Aug 8 06:51:58.480895665 (21h12m32.995998248s) [upload=21921]: Done starting upload Aug 8 06:51:58.481023418 (21h12m32.996125988s) [upload=21921]: Uploading segments... Aug 8 06:51:58.481086289 (21h12m32.996188877s) [upload=21921]: Got next segment. Inline: true Aug 8 06:51:58.481091513 (21h12m32.99619409s) [upload=21921]: Waiting for error group managing segments... Aug 8 06:51:58.481149620 (21h12m32.996252197s) [upload=21921]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:51:58.481154789 (21h12m32.996257366s) [upload=21921]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:51:58.481158546 (21h12m32.996261124s) [upload=21921]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:51:58.722055331 (21h12m33.237157908s) [upload=21921]: Done uploading segments... Aug 8 06:51:58.722073114 (21h12m33.23717568s) [upload=21921]: Upload finished. err: <nil> Aug 8 06:52:27.059254149 (21h13m1.57435673s) [upload=21928]: Starting upload Aug 8 06:52:27.059321812 (21h13m1.574424392s) [upload=21928]: Done starting upload Aug 8 06:52:27.059401946 (21h13m1.574504512s) [upload=21928]: Uploading segments... Aug 8 06:52:27.059448476 (21h13m1.574551053s) [upload=21928]: Got next segment. Inline: true Aug 8 06:52:27.059454258 (21h13m1.574556833s) [upload=21928]: Waiting for error group managing segments... Aug 8 06:52:27.059501468 (21h13m1.574604035s) [upload=21928]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:52:27.059506838 (21h13m1.574609416s) [upload=21928]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:52:27.059510827 (21h13m1.574613404s) [upload=21928]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:52:27.120385847 (21h13m1.635488418s) [upload=21928]: Done uploading segments... Aug 8 06:52:27.120403445 (21h13m1.635506012s) [upload=21928]: Upload finished. err: <nil> Aug 8 06:52:45.219137365 (21h13m19.734239945s) [upload=21933]: Starting upload Aug 8 06:52:45.219214556 (21h13m19.734317133s) [upload=21933]: Done starting upload Aug 8 06:52:45.219284289 (21h13m19.734386877s) [upload=21933]: Uploading segments... Aug 8 06:52:45.219335736 (21h13m19.73443834s) [upload=21933]: Got next segment. Inline: true Aug 8 06:52:45.219342793 (21h13m19.734445378s) [upload=21933]: Waiting for error group managing segments... Aug 8 06:52:45.219397449 (21h13m19.734500028s) [upload=21933]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:52:45.219402775 (21h13m19.734505359s) [upload=21933]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:52:45.219407364 (21h13m19.734509958s) [upload=21933]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:52:45.282551475 (21h13m19.79765406s) [upload=21933]: Done uploading segments... Aug 8 06:52:45.282572182 (21h13m19.797674765s) [upload=21933]: Upload finished. err: <nil> Aug 8 06:53:36.462986837 (21h14m10.978089404s) [upload=21951]: Starting upload Aug 8 06:53:36.463070050 (21h14m10.978172618s) [upload=21951]: Done starting upload Aug 8 06:53:36.463162828 (21h14m10.978265396s) [upload=21951]: Uploading segments... Aug 8 06:53:36.463220839 (21h14m10.978323421s) [upload=21951]: Got next segment. Inline: true Aug 8 06:53:36.463227617 (21h14m10.978330198s) [upload=21951]: Waiting for error group managing segments... Aug 8 06:53:36.463290301 (21h14m10.978392878s) [upload=21951]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:53:36.463295593 (21h14m10.978398162s) [upload=21951]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:53:36.463299324 (21h14m10.978401891s) [upload=21951]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:53:36.745725330 (21h14m11.260827913s) [upload=21951]: Done uploading segments... Aug 8 06:53:36.745748524 (21h14m11.260851096s) [upload=21951]: Upload finished. err: <nil> Aug 8 06:54:42.211460075 (21h15m16.726562642s) [upload=21971]: Starting upload Aug 8 06:54:42.211544762 (21h15m16.726647342s) [upload=21971]: Done starting upload Aug 8 06:54:42.211670011 (21h15m16.726772576s) [upload=21971]: Uploading segments... Aug 8 06:54:42.211709612 (21h15m16.726812197s) [upload=21971]: Got next segment. Inline: true Aug 8 06:54:42.211714530 (21h15m16.726817106s) [upload=21971]: Waiting for error group managing segments... Aug 8 06:54:42.211773234 (21h15m16.726875811s) [upload=21971]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:54:42.211777907 (21h15m16.726880484s) [upload=21971]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:54:42.211781406 (21h15m16.726883982s) [upload=21971]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:54:42.285628155 (21h15m16.800730732s) [upload=21971]: Done uploading segments... Aug 8 06:54:42.285645434 (21h15m16.800748001s) [upload=21971]: Upload finished. err: <nil> Aug 8 06:55:07.196164830 (21h15m41.711267417s) [upload=21978]: Starting upload Aug 8 06:55:07.196239324 (21h15m41.711341924s) [upload=21978]: Done starting upload Aug 8 06:55:07.196316739 (21h15m41.711419326s) [upload=21978]: Uploading segments... Aug 8 06:55:07.196366345 (21h15m41.711468934s) [upload=21978]: Got next segment. Inline: true Aug 8 06:55:07.196372407 (21h15m41.711474978s) [upload=21978]: Waiting for error group managing segments... Aug 8 06:55:07.196426504 (21h15m41.711529089s) [upload=21978]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:55:07.196433124 (21h15m41.711535688s) [upload=21978]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:55:07.196438529 (21h15m41.711541093s) [upload=21978]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:55:07.258380171 (21h15m41.773482756s) [upload=21978]: Done uploading segments... Aug 8 06:55:07.258413055 (21h15m41.773515634s) [upload=21978]: Upload finished. err: <nil> Aug 8 06:56:17.251734653 (21h16m51.766837222s) [upload=21994]: Starting upload Aug 8 06:56:17.251784283 (21h16m51.766886851s) [upload=21994]: Done starting upload Aug 8 06:56:17.251882450 (21h16m51.766985019s) [upload=21994]: Uploading segments... Aug 8 06:56:17.251922005 (21h16m51.767024587s) [upload=21994]: Got next segment. Inline: true Aug 8 06:56:17.251926655 (21h16m51.767029231s) [upload=21994]: Waiting for error group managing segments... Aug 8 06:56:17.251978681 (21h16m51.767081246s) [upload=21994]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:56:17.251983158 (21h16m51.767085734s) [upload=21994]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:56:17.251986429 (21h16m51.767089005s) [upload=21994]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:56:17.318329259 (21h16m51.833431843s) [upload=21994]: Done uploading segments... Aug 8 06:56:17.318350310 (21h16m51.833452887s) [upload=21994]: Upload finished. err: <nil>
Duration: 190.343928ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [....................X.......................................] duration:
185.626177ms
FAIL (2025-08-08 07:25:45 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:1024 (size:1024) ======LOGS====== d=21870]: Done uploading segments... Aug 8 06:48:53.275183069 (21h9m27.790285647s) [upload=21870]: Upload finished. err: <nil> Aug 8 06:49:08.545547742 (21h9m43.060650311s) [upload=21875]: Starting upload Aug 8 06:49:08.545594244 (21h9m43.06069682s) [upload=21875]: Done starting upload Aug 8 06:49:08.545651807 (21h9m43.060754385s) [upload=21875]: Uploading segments... Aug 8 06:49:08.545688834 (21h9m43.060791416s) [upload=21875]: Got next segment. Inline: true Aug 8 06:49:08.545693597 (21h9m43.060796163s) [upload=21875]: Waiting for error group managing segments... Aug 8 06:49:08.545736511 (21h9m43.060839081s) [upload=21875]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:49:08.545742589 (21h9m43.060845161s) [upload=21875]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:49:08.545747536 (21h9m43.060850111s) [upload=21875]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:49:08.608446156 (21h9m43.123548745s) [upload=21875]: Done uploading segments... Aug 8 06:49:08.608468645 (21h9m43.123571226s) [upload=21875]: Upload finished. err: <nil> Aug 8 06:49:53.145025965 (21h10m27.660128534s) [upload=21886]: Starting upload Aug 8 06:49:53.145086326 (21h10m27.660188904s) [upload=21886]: Done starting upload Aug 8 06:49:53.145166436 (21h10m27.660269004s) [upload=21886]: Uploading segments... Aug 8 06:49:53.145207405 (21h10m27.66030999s) [upload=21886]: Got next segment. Inline: true Aug 8 06:49:53.145212962 (21h10m27.660315539s) [upload=21886]: Waiting for error group managing segments... Aug 8 06:49:53.145265613 (21h10m27.660368179s) [upload=21886]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:49:53.145270200 (21h10m27.660372767s) [upload=21886]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:49:53.145274109 (21h10m27.660376686s) [upload=21886]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:49:53.205847742 (21h10m27.720950321s) [upload=21886]: Done uploading segments... Aug 8 06:49:53.205867747 (21h10m27.720970327s) [upload=21886]: Upload finished. err: <nil> Aug 8 06:50:45.962226830 (21h11m20.477329397s) [upload=21901]: Starting upload Aug 8 06:50:45.962960747 (21h11m20.47806339s) [upload=21901]: Done starting upload Aug 8 06:50:45.963081429 (21h11m20.478184007s) [upload=21901]: Uploading segments... Aug 8 06:50:45.963344944 (21h11m20.478447529s) [upload=21901]: Got next segment. Inline: true Aug 8 06:50:45.963354188 (21h11m20.478456768s) [upload=21901]: Waiting for error group managing segments... Aug 8 06:50:45.963408259 (21h11m20.478510836s) [upload=21901]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:50:45.963413437 (21h11m20.478516016s) [upload=21901]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:50:45.963418686 (21h11m20.478521263s) [upload=21901]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:50:46.026766649 (21h11m20.541869227s) [upload=21901]: Done uploading segments... Aug 8 06:50:46.026795865 (21h11m20.541898441s) [upload=21901]: Upload finished. err: <nil> Aug 8 06:51:58.480831248 (21h12m32.995933826s) [upload=21921]: Starting upload Aug 8 06:51:58.480895665 (21h12m32.995998248s) [upload=21921]: Done starting upload Aug 8 06:51:58.481023418 (21h12m32.996125988s) [upload=21921]: Uploading segments... Aug 8 06:51:58.481086289 (21h12m32.996188877s) [upload=21921]: Got next segment. Inline: true Aug 8 06:51:58.481091513 (21h12m32.99619409s) [upload=21921]: Waiting for error group managing segments... Aug 8 06:51:58.481149620 (21h12m32.996252197s) [upload=21921]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:51:58.481154789 (21h12m32.996257366s) [upload=21921]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:51:58.481158546 (21h12m32.996261124s) [upload=21921]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:51:58.722055331 (21h12m33.237157908s) [upload=21921]: Done uploading segments... Aug 8 06:51:58.722073114 (21h12m33.23717568s) [upload=21921]: Upload finished. err: <nil> Aug 8 06:52:27.059254149 (21h13m1.57435673s) [upload=21928]: Starting upload Aug 8 06:52:27.059321812 (21h13m1.574424392s) [upload=21928]: Done starting upload Aug 8 06:52:27.059401946 (21h13m1.574504512s) [upload=21928]: Uploading segments... Aug 8 06:52:27.059448476 (21h13m1.574551053s) [upload=21928]: Got next segment. Inline: true Aug 8 06:52:27.059454258 (21h13m1.574556833s) [upload=21928]: Waiting for error group managing segments... Aug 8 06:52:27.059501468 (21h13m1.574604035s) [upload=21928]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:52:27.059506838 (21h13m1.574609416s) [upload=21928]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:52:27.059510827 (21h13m1.574613404s) [upload=21928]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:52:27.120385847 (21h13m1.635488418s) [upload=21928]: Done uploading segments... Aug 8 06:52:27.120403445 (21h13m1.635506012s) [upload=21928]: Upload finished. err: <nil> Aug 8 06:52:45.219137365 (21h13m19.734239945s) [upload=21933]: Starting upload Aug 8 06:52:45.219214556 (21h13m19.734317133s) [upload=21933]: Done starting upload Aug 8 06:52:45.219284289 (21h13m19.734386877s) [upload=21933]: Uploading segments... Aug 8 06:52:45.219335736 (21h13m19.73443834s) [upload=21933]: Got next segment. Inline: true Aug 8 06:52:45.219342793 (21h13m19.734445378s) [upload=21933]: Waiting for error group managing segments... Aug 8 06:52:45.219397449 (21h13m19.734500028s) [upload=21933]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:52:45.219402775 (21h13m19.734505359s) [upload=21933]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:52:45.219407364 (21h13m19.734509958s) [upload=21933]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:52:45.282551475 (21h13m19.79765406s) [upload=21933]: Done uploading segments... Aug 8 06:52:45.282572182 (21h13m19.797674765s) [upload=21933]: Upload finished. err: <nil> Aug 8 06:53:36.462986837 (21h14m10.978089404s) [upload=21951]: Starting upload Aug 8 06:53:36.463070050 (21h14m10.978172618s) [upload=21951]: Done starting upload Aug 8 06:53:36.463162828 (21h14m10.978265396s) [upload=21951]: Uploading segments... Aug 8 06:53:36.463220839 (21h14m10.978323421s) [upload=21951]: Got next segment. Inline: true Aug 8 06:53:36.463227617 (21h14m10.978330198s) [upload=21951]: Waiting for error group managing segments... Aug 8 06:53:36.463290301 (21h14m10.978392878s) [upload=21951]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:53:36.463295593 (21h14m10.978398162s) [upload=21951]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:53:36.463299324 (21h14m10.978401891s) [upload=21951]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:53:36.745725330 (21h14m11.260827913s) [upload=21951]: Done uploading segments... Aug 8 06:53:36.745748524 (21h14m11.260851096s) [upload=21951]: Upload finished. err: <nil> Aug 8 06:54:42.211460075 (21h15m16.726562642s) [upload=21971]: Starting upload Aug 8 06:54:42.211544762 (21h15m16.726647342s) [upload=21971]: Done starting upload Aug 8 06:54:42.211670011 (21h15m16.726772576s) [upload=21971]: Uploading segments... Aug 8 06:54:42.211709612 (21h15m16.726812197s) [upload=21971]: Got next segment. Inline: true Aug 8 06:54:42.211714530 (21h15m16.726817106s) [upload=21971]: Waiting for error group managing segments... Aug 8 06:54:42.211773234 (21h15m16.726875811s) [upload=21971]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:54:42.211777907 (21h15m16.726880484s) [upload=21971]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:54:42.211781406 (21h15m16.726883982s) [upload=21971]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:54:42.285628155 (21h15m16.800730732s) [upload=21971]: Done uploading segments... Aug 8 06:54:42.285645434 (21h15m16.800748001s) [upload=21971]: Upload finished. err: <nil> Aug 8 06:55:07.196164830 (21h15m41.711267417s) [upload=21978]: Starting upload Aug 8 06:55:07.196239324 (21h15m41.711341924s) [upload=21978]: Done starting upload Aug 8 06:55:07.196316739 (21h15m41.711419326s) [upload=21978]: Uploading segments... Aug 8 06:55:07.196366345 (21h15m41.711468934s) [upload=21978]: Got next segment. Inline: true Aug 8 06:55:07.196372407 (21h15m41.711474978s) [upload=21978]: Waiting for error group managing segments... Aug 8 06:55:07.196426504 (21h15m41.711529089s) [upload=21978]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:55:07.196433124 (21h15m41.711535688s) [upload=21978]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:55:07.196438529 (21h15m41.711541093s) [upload=21978]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:55:07.258380171 (21h15m41.773482756s) [upload=21978]: Done uploading segments... Aug 8 06:55:07.258413055 (21h15m41.773515634s) [upload=21978]: Upload finished. err: <nil> Aug 8 06:56:17.251734653 (21h16m51.766837222s) [upload=21994]: Starting upload Aug 8 06:56:17.251784283 (21h16m51.766886851s) [upload=21994]: Done starting upload Aug 8 06:56:17.251882450 (21h16m51.766985019s) [upload=21994]: Uploading segments... Aug 8 06:56:17.251922005 (21h16m51.767024587s) [upload=21994]: Got next segment. Inline: true Aug 8 06:56:17.251926655 (21h16m51.767029231s) [upload=21994]: Waiting for error group managing segments... Aug 8 06:56:17.251978681 (21h16m51.767081246s) [upload=21994]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:56:17.251983158 (21h16m51.767085734s) [upload=21994]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:56:17.251986429 (21h16m51.767089005s) [upload=21994]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:56:17.318329259 (21h16m51.833431843s) [upload=21994]: Done uploading segments... Aug 8 06:56:17.318350310 (21h16m51.833452887s) [upload=21994]: Upload finished. err: <nil>
Duration: 185.648925ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [.....................X......................................] duration:
215.870409ms
FAIL (2025-08-08 07:24:44 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:1024 (size:1024) ======LOGS====== d=21870]: Done uploading segments... Aug 8 06:48:53.275183069 (21h9m27.790285647s) [upload=21870]: Upload finished. err: <nil> Aug 8 06:49:08.545547742 (21h9m43.060650311s) [upload=21875]: Starting upload Aug 8 06:49:08.545594244 (21h9m43.06069682s) [upload=21875]: Done starting upload Aug 8 06:49:08.545651807 (21h9m43.060754385s) [upload=21875]: Uploading segments... Aug 8 06:49:08.545688834 (21h9m43.060791416s) [upload=21875]: Got next segment. Inline: true Aug 8 06:49:08.545693597 (21h9m43.060796163s) [upload=21875]: Waiting for error group managing segments... Aug 8 06:49:08.545736511 (21h9m43.060839081s) [upload=21875]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:49:08.545742589 (21h9m43.060845161s) [upload=21875]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:49:08.545747536 (21h9m43.060850111s) [upload=21875]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:49:08.608446156 (21h9m43.123548745s) [upload=21875]: Done uploading segments... Aug 8 06:49:08.608468645 (21h9m43.123571226s) [upload=21875]: Upload finished. err: <nil> Aug 8 06:49:53.145025965 (21h10m27.660128534s) [upload=21886]: Starting upload Aug 8 06:49:53.145086326 (21h10m27.660188904s) [upload=21886]: Done starting upload Aug 8 06:49:53.145166436 (21h10m27.660269004s) [upload=21886]: Uploading segments... Aug 8 06:49:53.145207405 (21h10m27.66030999s) [upload=21886]: Got next segment. Inline: true Aug 8 06:49:53.145212962 (21h10m27.660315539s) [upload=21886]: Waiting for error group managing segments... Aug 8 06:49:53.145265613 (21h10m27.660368179s) [upload=21886]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:49:53.145270200 (21h10m27.660372767s) [upload=21886]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:49:53.145274109 (21h10m27.660376686s) [upload=21886]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:49:53.205847742 (21h10m27.720950321s) [upload=21886]: Done uploading segments... Aug 8 06:49:53.205867747 (21h10m27.720970327s) [upload=21886]: Upload finished. err: <nil> Aug 8 06:50:45.962226830 (21h11m20.477329397s) [upload=21901]: Starting upload Aug 8 06:50:45.962960747 (21h11m20.47806339s) [upload=21901]: Done starting upload Aug 8 06:50:45.963081429 (21h11m20.478184007s) [upload=21901]: Uploading segments... Aug 8 06:50:45.963344944 (21h11m20.478447529s) [upload=21901]: Got next segment. Inline: true Aug 8 06:50:45.963354188 (21h11m20.478456768s) [upload=21901]: Waiting for error group managing segments... Aug 8 06:50:45.963408259 (21h11m20.478510836s) [upload=21901]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:50:45.963413437 (21h11m20.478516016s) [upload=21901]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:50:45.963418686 (21h11m20.478521263s) [upload=21901]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:50:46.026766649 (21h11m20.541869227s) [upload=21901]: Done uploading segments... Aug 8 06:50:46.026795865 (21h11m20.541898441s) [upload=21901]: Upload finished. err: <nil> Aug 8 06:51:58.480831248 (21h12m32.995933826s) [upload=21921]: Starting upload Aug 8 06:51:58.480895665 (21h12m32.995998248s) [upload=21921]: Done starting upload Aug 8 06:51:58.481023418 (21h12m32.996125988s) [upload=21921]: Uploading segments... Aug 8 06:51:58.481086289 (21h12m32.996188877s) [upload=21921]: Got next segment. Inline: true Aug 8 06:51:58.481091513 (21h12m32.99619409s) [upload=21921]: Waiting for error group managing segments... Aug 8 06:51:58.481149620 (21h12m32.996252197s) [upload=21921]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:51:58.481154789 (21h12m32.996257366s) [upload=21921]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:51:58.481158546 (21h12m32.996261124s) [upload=21921]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:51:58.722055331 (21h12m33.237157908s) [upload=21921]: Done uploading segments... Aug 8 06:51:58.722073114 (21h12m33.23717568s) [upload=21921]: Upload finished. err: <nil> Aug 8 06:52:27.059254149 (21h13m1.57435673s) [upload=21928]: Starting upload Aug 8 06:52:27.059321812 (21h13m1.574424392s) [upload=21928]: Done starting upload Aug 8 06:52:27.059401946 (21h13m1.574504512s) [upload=21928]: Uploading segments... Aug 8 06:52:27.059448476 (21h13m1.574551053s) [upload=21928]: Got next segment. Inline: true Aug 8 06:52:27.059454258 (21h13m1.574556833s) [upload=21928]: Waiting for error group managing segments... Aug 8 06:52:27.059501468 (21h13m1.574604035s) [upload=21928]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:52:27.059506838 (21h13m1.574609416s) [upload=21928]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:52:27.059510827 (21h13m1.574613404s) [upload=21928]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:52:27.120385847 (21h13m1.635488418s) [upload=21928]: Done uploading segments... Aug 8 06:52:27.120403445 (21h13m1.635506012s) [upload=21928]: Upload finished. err: <nil> Aug 8 06:52:45.219137365 (21h13m19.734239945s) [upload=21933]: Starting upload Aug 8 06:52:45.219214556 (21h13m19.734317133s) [upload=21933]: Done starting upload Aug 8 06:52:45.219284289 (21h13m19.734386877s) [upload=21933]: Uploading segments... Aug 8 06:52:45.219335736 (21h13m19.73443834s) [upload=21933]: Got next segment. Inline: true Aug 8 06:52:45.219342793 (21h13m19.734445378s) [upload=21933]: Waiting for error group managing segments... Aug 8 06:52:45.219397449 (21h13m19.734500028s) [upload=21933]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:52:45.219402775 (21h13m19.734505359s) [upload=21933]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:52:45.219407364 (21h13m19.734509958s) [upload=21933]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:52:45.282551475 (21h13m19.79765406s) [upload=21933]: Done uploading segments... Aug 8 06:52:45.282572182 (21h13m19.797674765s) [upload=21933]: Upload finished. err: <nil> Aug 8 06:53:36.462986837 (21h14m10.978089404s) [upload=21951]: Starting upload Aug 8 06:53:36.463070050 (21h14m10.978172618s) [upload=21951]: Done starting upload Aug 8 06:53:36.463162828 (21h14m10.978265396s) [upload=21951]: Uploading segments... Aug 8 06:53:36.463220839 (21h14m10.978323421s) [upload=21951]: Got next segment. Inline: true Aug 8 06:53:36.463227617 (21h14m10.978330198s) [upload=21951]: Waiting for error group managing segments... Aug 8 06:53:36.463290301 (21h14m10.978392878s) [upload=21951]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:53:36.463295593 (21h14m10.978398162s) [upload=21951]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:53:36.463299324 (21h14m10.978401891s) [upload=21951]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:53:36.745725330 (21h14m11.260827913s) [upload=21951]: Done uploading segments... Aug 8 06:53:36.745748524 (21h14m11.260851096s) [upload=21951]: Upload finished. err: <nil> Aug 8 06:54:42.211460075 (21h15m16.726562642s) [upload=21971]: Starting upload Aug 8 06:54:42.211544762 (21h15m16.726647342s) [upload=21971]: Done starting upload Aug 8 06:54:42.211670011 (21h15m16.726772576s) [upload=21971]: Uploading segments... Aug 8 06:54:42.211709612 (21h15m16.726812197s) [upload=21971]: Got next segment. Inline: true Aug 8 06:54:42.211714530 (21h15m16.726817106s) [upload=21971]: Waiting for error group managing segments... Aug 8 06:54:42.211773234 (21h15m16.726875811s) [upload=21971]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:54:42.211777907 (21h15m16.726880484s) [upload=21971]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:54:42.211781406 (21h15m16.726883982s) [upload=21971]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:54:42.285628155 (21h15m16.800730732s) [upload=21971]: Done uploading segments... Aug 8 06:54:42.285645434 (21h15m16.800748001s) [upload=21971]: Upload finished. err: <nil> Aug 8 06:55:07.196164830 (21h15m41.711267417s) [upload=21978]: Starting upload Aug 8 06:55:07.196239324 (21h15m41.711341924s) [upload=21978]: Done starting upload Aug 8 06:55:07.196316739 (21h15m41.711419326s) [upload=21978]: Uploading segments... Aug 8 06:55:07.196366345 (21h15m41.711468934s) [upload=21978]: Got next segment. Inline: true Aug 8 06:55:07.196372407 (21h15m41.711474978s) [upload=21978]: Waiting for error group managing segments... Aug 8 06:55:07.196426504 (21h15m41.711529089s) [upload=21978]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:55:07.196433124 (21h15m41.711535688s) [upload=21978]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:55:07.196438529 (21h15m41.711541093s) [upload=21978]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:55:07.258380171 (21h15m41.773482756s) [upload=21978]: Done uploading segments... Aug 8 06:55:07.258413055 (21h15m41.773515634s) [upload=21978]: Upload finished. err: <nil> Aug 8 06:56:17.251734653 (21h16m51.766837222s) [upload=21994]: Starting upload Aug 8 06:56:17.251784283 (21h16m51.766886851s) [upload=21994]: Done starting upload Aug 8 06:56:17.251882450 (21h16m51.766985019s) [upload=21994]: Uploading segments... Aug 8 06:56:17.251922005 (21h16m51.767024587s) [upload=21994]: Got next segment. Inline: true Aug 8 06:56:17.251926655 (21h16m51.767029231s) [upload=21994]: Waiting for error group managing segments... Aug 8 06:56:17.251978681 (21h16m51.767081246s) [upload=21994]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:56:17.251983158 (21h16m51.767085734s) [upload=21994]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:56:17.251986429 (21h16m51.767089005s) [upload=21994]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:56:17.318329259 (21h16m51.833431843s) [upload=21994]: Done uploading segments... Aug 8 06:56:17.318350310 (21h16m51.833452887s) [upload=21994]: Upload finished. err: <nil>
Duration: 215.90067ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [......................X.....................................] duration:
186.45674ms
FAIL (2025-08-08 07:24:18 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:1024 (size:1024) ======LOGS====== d=21870]: Done uploading segments... Aug 8 06:48:53.275183069 (21h9m27.790285647s) [upload=21870]: Upload finished. err: <nil> Aug 8 06:49:08.545547742 (21h9m43.060650311s) [upload=21875]: Starting upload Aug 8 06:49:08.545594244 (21h9m43.06069682s) [upload=21875]: Done starting upload Aug 8 06:49:08.545651807 (21h9m43.060754385s) [upload=21875]: Uploading segments... Aug 8 06:49:08.545688834 (21h9m43.060791416s) [upload=21875]: Got next segment. Inline: true Aug 8 06:49:08.545693597 (21h9m43.060796163s) [upload=21875]: Waiting for error group managing segments... Aug 8 06:49:08.545736511 (21h9m43.060839081s) [upload=21875]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:49:08.545742589 (21h9m43.060845161s) [upload=21875]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:49:08.545747536 (21h9m43.060850111s) [upload=21875]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:49:08.608446156 (21h9m43.123548745s) [upload=21875]: Done uploading segments... Aug 8 06:49:08.608468645 (21h9m43.123571226s) [upload=21875]: Upload finished. err: <nil> Aug 8 06:49:53.145025965 (21h10m27.660128534s) [upload=21886]: Starting upload Aug 8 06:49:53.145086326 (21h10m27.660188904s) [upload=21886]: Done starting upload Aug 8 06:49:53.145166436 (21h10m27.660269004s) [upload=21886]: Uploading segments... Aug 8 06:49:53.145207405 (21h10m27.66030999s) [upload=21886]: Got next segment. Inline: true Aug 8 06:49:53.145212962 (21h10m27.660315539s) [upload=21886]: Waiting for error group managing segments... Aug 8 06:49:53.145265613 (21h10m27.660368179s) [upload=21886]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:49:53.145270200 (21h10m27.660372767s) [upload=21886]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:49:53.145274109 (21h10m27.660376686s) [upload=21886]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:49:53.205847742 (21h10m27.720950321s) [upload=21886]: Done uploading segments... Aug 8 06:49:53.205867747 (21h10m27.720970327s) [upload=21886]: Upload finished. err: <nil> Aug 8 06:50:45.962226830 (21h11m20.477329397s) [upload=21901]: Starting upload Aug 8 06:50:45.962960747 (21h11m20.47806339s) [upload=21901]: Done starting upload Aug 8 06:50:45.963081429 (21h11m20.478184007s) [upload=21901]: Uploading segments... Aug 8 06:50:45.963344944 (21h11m20.478447529s) [upload=21901]: Got next segment. Inline: true Aug 8 06:50:45.963354188 (21h11m20.478456768s) [upload=21901]: Waiting for error group managing segments... Aug 8 06:50:45.963408259 (21h11m20.478510836s) [upload=21901]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:50:45.963413437 (21h11m20.478516016s) [upload=21901]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:50:45.963418686 (21h11m20.478521263s) [upload=21901]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:50:46.026766649 (21h11m20.541869227s) [upload=21901]: Done uploading segments... Aug 8 06:50:46.026795865 (21h11m20.541898441s) [upload=21901]: Upload finished. err: <nil> Aug 8 06:51:58.480831248 (21h12m32.995933826s) [upload=21921]: Starting upload Aug 8 06:51:58.480895665 (21h12m32.995998248s) [upload=21921]: Done starting upload Aug 8 06:51:58.481023418 (21h12m32.996125988s) [upload=21921]: Uploading segments... Aug 8 06:51:58.481086289 (21h12m32.996188877s) [upload=21921]: Got next segment. Inline: true Aug 8 06:51:58.481091513 (21h12m32.99619409s) [upload=21921]: Waiting for error group managing segments... Aug 8 06:51:58.481149620 (21h12m32.996252197s) [upload=21921]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:51:58.481154789 (21h12m32.996257366s) [upload=21921]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:51:58.481158546 (21h12m32.996261124s) [upload=21921]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:51:58.722055331 (21h12m33.237157908s) [upload=21921]: Done uploading segments... Aug 8 06:51:58.722073114 (21h12m33.23717568s) [upload=21921]: Upload finished. err: <nil> Aug 8 06:52:27.059254149 (21h13m1.57435673s) [upload=21928]: Starting upload Aug 8 06:52:27.059321812 (21h13m1.574424392s) [upload=21928]: Done starting upload Aug 8 06:52:27.059401946 (21h13m1.574504512s) [upload=21928]: Uploading segments... Aug 8 06:52:27.059448476 (21h13m1.574551053s) [upload=21928]: Got next segment. Inline: true Aug 8 06:52:27.059454258 (21h13m1.574556833s) [upload=21928]: Waiting for error group managing segments... Aug 8 06:52:27.059501468 (21h13m1.574604035s) [upload=21928]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:52:27.059506838 (21h13m1.574609416s) [upload=21928]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:52:27.059510827 (21h13m1.574613404s) [upload=21928]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:52:27.120385847 (21h13m1.635488418s) [upload=21928]: Done uploading segments... Aug 8 06:52:27.120403445 (21h13m1.635506012s) [upload=21928]: Upload finished. err: <nil> Aug 8 06:52:45.219137365 (21h13m19.734239945s) [upload=21933]: Starting upload Aug 8 06:52:45.219214556 (21h13m19.734317133s) [upload=21933]: Done starting upload Aug 8 06:52:45.219284289 (21h13m19.734386877s) [upload=21933]: Uploading segments... Aug 8 06:52:45.219335736 (21h13m19.73443834s) [upload=21933]: Got next segment. Inline: true Aug 8 06:52:45.219342793 (21h13m19.734445378s) [upload=21933]: Waiting for error group managing segments... Aug 8 06:52:45.219397449 (21h13m19.734500028s) [upload=21933]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:52:45.219402775 (21h13m19.734505359s) [upload=21933]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:52:45.219407364 (21h13m19.734509958s) [upload=21933]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:52:45.282551475 (21h13m19.79765406s) [upload=21933]: Done uploading segments... Aug 8 06:52:45.282572182 (21h13m19.797674765s) [upload=21933]: Upload finished. err: <nil> Aug 8 06:53:36.462986837 (21h14m10.978089404s) [upload=21951]: Starting upload Aug 8 06:53:36.463070050 (21h14m10.978172618s) [upload=21951]: Done starting upload Aug 8 06:53:36.463162828 (21h14m10.978265396s) [upload=21951]: Uploading segments... Aug 8 06:53:36.463220839 (21h14m10.978323421s) [upload=21951]: Got next segment. Inline: true Aug 8 06:53:36.463227617 (21h14m10.978330198s) [upload=21951]: Waiting for error group managing segments... Aug 8 06:53:36.463290301 (21h14m10.978392878s) [upload=21951]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:53:36.463295593 (21h14m10.978398162s) [upload=21951]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:53:36.463299324 (21h14m10.978401891s) [upload=21951]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:53:36.745725330 (21h14m11.260827913s) [upload=21951]: Done uploading segments... Aug 8 06:53:36.745748524 (21h14m11.260851096s) [upload=21951]: Upload finished. err: <nil> Aug 8 06:54:42.211460075 (21h15m16.726562642s) [upload=21971]: Starting upload Aug 8 06:54:42.211544762 (21h15m16.726647342s) [upload=21971]: Done starting upload Aug 8 06:54:42.211670011 (21h15m16.726772576s) [upload=21971]: Uploading segments... Aug 8 06:54:42.211709612 (21h15m16.726812197s) [upload=21971]: Got next segment. Inline: true Aug 8 06:54:42.211714530 (21h15m16.726817106s) [upload=21971]: Waiting for error group managing segments... Aug 8 06:54:42.211773234 (21h15m16.726875811s) [upload=21971]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:54:42.211777907 (21h15m16.726880484s) [upload=21971]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:54:42.211781406 (21h15m16.726883982s) [upload=21971]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:54:42.285628155 (21h15m16.800730732s) [upload=21971]: Done uploading segments... Aug 8 06:54:42.285645434 (21h15m16.800748001s) [upload=21971]: Upload finished. err: <nil> Aug 8 06:55:07.196164830 (21h15m41.711267417s) [upload=21978]: Starting upload Aug 8 06:55:07.196239324 (21h15m41.711341924s) [upload=21978]: Done starting upload Aug 8 06:55:07.196316739 (21h15m41.711419326s) [upload=21978]: Uploading segments... Aug 8 06:55:07.196366345 (21h15m41.711468934s) [upload=21978]: Got next segment. Inline: true Aug 8 06:55:07.196372407 (21h15m41.711474978s) [upload=21978]: Waiting for error group managing segments... Aug 8 06:55:07.196426504 (21h15m41.711529089s) [upload=21978]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:55:07.196433124 (21h15m41.711535688s) [upload=21978]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:55:07.196438529 (21h15m41.711541093s) [upload=21978]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:55:07.258380171 (21h15m41.773482756s) [upload=21978]: Done uploading segments... Aug 8 06:55:07.258413055 (21h15m41.773515634s) [upload=21978]: Upload finished. err: <nil> Aug 8 06:56:17.251734653 (21h16m51.766837222s) [upload=21994]: Starting upload Aug 8 06:56:17.251784283 (21h16m51.766886851s) [upload=21994]: Done starting upload Aug 8 06:56:17.251882450 (21h16m51.766985019s) [upload=21994]: Uploading segments... Aug 8 06:56:17.251922005 (21h16m51.767024587s) [upload=21994]: Got next segment. Inline: true Aug 8 06:56:17.251926655 (21h16m51.767029231s) [upload=21994]: Waiting for error group managing segments... Aug 8 06:56:17.251978681 (21h16m51.767081246s) [upload=21994]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:56:17.251983158 (21h16m51.767085734s) [upload=21994]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:56:17.251986429 (21h16m51.767089005s) [upload=21994]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:56:17.318329259 (21h16m51.833431843s) [upload=21994]: Done uploading segments... Aug 8 06:56:17.318350310 (21h16m51.833452887s) [upload=21994]: Upload finished. err: <nil>
Duration: 186.483234ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [.......................X....................................] duration:
203.020149ms
FAIL (2025-08-08 07:23:44 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:1024 (size:1024) ======LOGS====== d=21870]: Done uploading segments... Aug 8 06:48:53.275183069 (21h9m27.790285647s) [upload=21870]: Upload finished. err: <nil> Aug 8 06:49:08.545547742 (21h9m43.060650311s) [upload=21875]: Starting upload Aug 8 06:49:08.545594244 (21h9m43.06069682s) [upload=21875]: Done starting upload Aug 8 06:49:08.545651807 (21h9m43.060754385s) [upload=21875]: Uploading segments... Aug 8 06:49:08.545688834 (21h9m43.060791416s) [upload=21875]: Got next segment. Inline: true Aug 8 06:49:08.545693597 (21h9m43.060796163s) [upload=21875]: Waiting for error group managing segments... Aug 8 06:49:08.545736511 (21h9m43.060839081s) [upload=21875]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:49:08.545742589 (21h9m43.060845161s) [upload=21875]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:49:08.545747536 (21h9m43.060850111s) [upload=21875]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:49:08.608446156 (21h9m43.123548745s) [upload=21875]: Done uploading segments... Aug 8 06:49:08.608468645 (21h9m43.123571226s) [upload=21875]: Upload finished. err: <nil> Aug 8 06:49:53.145025965 (21h10m27.660128534s) [upload=21886]: Starting upload Aug 8 06:49:53.145086326 (21h10m27.660188904s) [upload=21886]: Done starting upload Aug 8 06:49:53.145166436 (21h10m27.660269004s) [upload=21886]: Uploading segments... Aug 8 06:49:53.145207405 (21h10m27.66030999s) [upload=21886]: Got next segment. Inline: true Aug 8 06:49:53.145212962 (21h10m27.660315539s) [upload=21886]: Waiting for error group managing segments... Aug 8 06:49:53.145265613 (21h10m27.660368179s) [upload=21886]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:49:53.145270200 (21h10m27.660372767s) [upload=21886]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:49:53.145274109 (21h10m27.660376686s) [upload=21886]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:49:53.205847742 (21h10m27.720950321s) [upload=21886]: Done uploading segments... Aug 8 06:49:53.205867747 (21h10m27.720970327s) [upload=21886]: Upload finished. err: <nil> Aug 8 06:50:45.962226830 (21h11m20.477329397s) [upload=21901]: Starting upload Aug 8 06:50:45.962960747 (21h11m20.47806339s) [upload=21901]: Done starting upload Aug 8 06:50:45.963081429 (21h11m20.478184007s) [upload=21901]: Uploading segments... Aug 8 06:50:45.963344944 (21h11m20.478447529s) [upload=21901]: Got next segment. Inline: true Aug 8 06:50:45.963354188 (21h11m20.478456768s) [upload=21901]: Waiting for error group managing segments... Aug 8 06:50:45.963408259 (21h11m20.478510836s) [upload=21901]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:50:45.963413437 (21h11m20.478516016s) [upload=21901]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:50:45.963418686 (21h11m20.478521263s) [upload=21901]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:50:46.026766649 (21h11m20.541869227s) [upload=21901]: Done uploading segments... Aug 8 06:50:46.026795865 (21h11m20.541898441s) [upload=21901]: Upload finished. err: <nil> Aug 8 06:51:58.480831248 (21h12m32.995933826s) [upload=21921]: Starting upload Aug 8 06:51:58.480895665 (21h12m32.995998248s) [upload=21921]: Done starting upload Aug 8 06:51:58.481023418 (21h12m32.996125988s) [upload=21921]: Uploading segments... Aug 8 06:51:58.481086289 (21h12m32.996188877s) [upload=21921]: Got next segment. Inline: true Aug 8 06:51:58.481091513 (21h12m32.99619409s) [upload=21921]: Waiting for error group managing segments... Aug 8 06:51:58.481149620 (21h12m32.996252197s) [upload=21921]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:51:58.481154789 (21h12m32.996257366s) [upload=21921]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:51:58.481158546 (21h12m32.996261124s) [upload=21921]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:51:58.722055331 (21h12m33.237157908s) [upload=21921]: Done uploading segments... Aug 8 06:51:58.722073114 (21h12m33.23717568s) [upload=21921]: Upload finished. err: <nil> Aug 8 06:52:27.059254149 (21h13m1.57435673s) [upload=21928]: Starting upload Aug 8 06:52:27.059321812 (21h13m1.574424392s) [upload=21928]: Done starting upload Aug 8 06:52:27.059401946 (21h13m1.574504512s) [upload=21928]: Uploading segments... Aug 8 06:52:27.059448476 (21h13m1.574551053s) [upload=21928]: Got next segment. Inline: true Aug 8 06:52:27.059454258 (21h13m1.574556833s) [upload=21928]: Waiting for error group managing segments... Aug 8 06:52:27.059501468 (21h13m1.574604035s) [upload=21928]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:52:27.059506838 (21h13m1.574609416s) [upload=21928]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:52:27.059510827 (21h13m1.574613404s) [upload=21928]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:52:27.120385847 (21h13m1.635488418s) [upload=21928]: Done uploading segments... Aug 8 06:52:27.120403445 (21h13m1.635506012s) [upload=21928]: Upload finished. err: <nil> Aug 8 06:52:45.219137365 (21h13m19.734239945s) [upload=21933]: Starting upload Aug 8 06:52:45.219214556 (21h13m19.734317133s) [upload=21933]: Done starting upload Aug 8 06:52:45.219284289 (21h13m19.734386877s) [upload=21933]: Uploading segments... Aug 8 06:52:45.219335736 (21h13m19.73443834s) [upload=21933]: Got next segment. Inline: true Aug 8 06:52:45.219342793 (21h13m19.734445378s) [upload=21933]: Waiting for error group managing segments... Aug 8 06:52:45.219397449 (21h13m19.734500028s) [upload=21933]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:52:45.219402775 (21h13m19.734505359s) [upload=21933]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:52:45.219407364 (21h13m19.734509958s) [upload=21933]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:52:45.282551475 (21h13m19.79765406s) [upload=21933]: Done uploading segments... Aug 8 06:52:45.282572182 (21h13m19.797674765s) [upload=21933]: Upload finished. err: <nil> Aug 8 06:53:36.462986837 (21h14m10.978089404s) [upload=21951]: Starting upload Aug 8 06:53:36.463070050 (21h14m10.978172618s) [upload=21951]: Done starting upload Aug 8 06:53:36.463162828 (21h14m10.978265396s) [upload=21951]: Uploading segments... Aug 8 06:53:36.463220839 (21h14m10.978323421s) [upload=21951]: Got next segment. Inline: true Aug 8 06:53:36.463227617 (21h14m10.978330198s) [upload=21951]: Waiting for error group managing segments... Aug 8 06:53:36.463290301 (21h14m10.978392878s) [upload=21951]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:53:36.463295593 (21h14m10.978398162s) [upload=21951]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:53:36.463299324 (21h14m10.978401891s) [upload=21951]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:53:36.745725330 (21h14m11.260827913s) [upload=21951]: Done uploading segments... Aug 8 06:53:36.745748524 (21h14m11.260851096s) [upload=21951]: Upload finished. err: <nil> Aug 8 06:54:42.211460075 (21h15m16.726562642s) [upload=21971]: Starting upload Aug 8 06:54:42.211544762 (21h15m16.726647342s) [upload=21971]: Done starting upload Aug 8 06:54:42.211670011 (21h15m16.726772576s) [upload=21971]: Uploading segments... Aug 8 06:54:42.211709612 (21h15m16.726812197s) [upload=21971]: Got next segment. Inline: true Aug 8 06:54:42.211714530 (21h15m16.726817106s) [upload=21971]: Waiting for error group managing segments... Aug 8 06:54:42.211773234 (21h15m16.726875811s) [upload=21971]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:54:42.211777907 (21h15m16.726880484s) [upload=21971]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:54:42.211781406 (21h15m16.726883982s) [upload=21971]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:54:42.285628155 (21h15m16.800730732s) [upload=21971]: Done uploading segments... Aug 8 06:54:42.285645434 (21h15m16.800748001s) [upload=21971]: Upload finished. err: <nil> Aug 8 06:55:07.196164830 (21h15m41.711267417s) [upload=21978]: Starting upload Aug 8 06:55:07.196239324 (21h15m41.711341924s) [upload=21978]: Done starting upload Aug 8 06:55:07.196316739 (21h15m41.711419326s) [upload=21978]: Uploading segments... Aug 8 06:55:07.196366345 (21h15m41.711468934s) [upload=21978]: Got next segment. Inline: true Aug 8 06:55:07.196372407 (21h15m41.711474978s) [upload=21978]: Waiting for error group managing segments... Aug 8 06:55:07.196426504 (21h15m41.711529089s) [upload=21978]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:55:07.196433124 (21h15m41.711535688s) [upload=21978]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:55:07.196438529 (21h15m41.711541093s) [upload=21978]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:55:07.258380171 (21h15m41.773482756s) [upload=21978]: Done uploading segments... Aug 8 06:55:07.258413055 (21h15m41.773515634s) [upload=21978]: Upload finished. err: <nil> Aug 8 06:56:17.251734653 (21h16m51.766837222s) [upload=21994]: Starting upload Aug 8 06:56:17.251784283 (21h16m51.766886851s) [upload=21994]: Done starting upload Aug 8 06:56:17.251882450 (21h16m51.766985019s) [upload=21994]: Uploading segments... Aug 8 06:56:17.251922005 (21h16m51.767024587s) [upload=21994]: Got next segment. Inline: true Aug 8 06:56:17.251926655 (21h16m51.767029231s) [upload=21994]: Waiting for error group managing segments... Aug 8 06:56:17.251978681 (21h16m51.767081246s) [upload=21994]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:56:17.251983158 (21h16m51.767085734s) [upload=21994]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:56:17.251986429 (21h16m51.767089005s) [upload=21994]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:56:17.318329259 (21h16m51.833431843s) [upload=21994]: Done uploading segments... Aug 8 06:56:17.318350310 (21h16m51.833452887s) [upload=21994]: Upload finished. err: <nil>
Duration: 203.084388ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [........................X...................................] duration:
209.8766ms
FAIL (2025-08-08 07:23:07 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:1024 (size:1024) ======LOGS====== d=21870]: Done uploading segments... Aug 8 06:48:53.275183069 (21h9m27.790285647s) [upload=21870]: Upload finished. err: <nil> Aug 8 06:49:08.545547742 (21h9m43.060650311s) [upload=21875]: Starting upload Aug 8 06:49:08.545594244 (21h9m43.06069682s) [upload=21875]: Done starting upload Aug 8 06:49:08.545651807 (21h9m43.060754385s) [upload=21875]: Uploading segments... Aug 8 06:49:08.545688834 (21h9m43.060791416s) [upload=21875]: Got next segment. Inline: true Aug 8 06:49:08.545693597 (21h9m43.060796163s) [upload=21875]: Waiting for error group managing segments... Aug 8 06:49:08.545736511 (21h9m43.060839081s) [upload=21875]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:49:08.545742589 (21h9m43.060845161s) [upload=21875]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:49:08.545747536 (21h9m43.060850111s) [upload=21875]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:49:08.608446156 (21h9m43.123548745s) [upload=21875]: Done uploading segments... Aug 8 06:49:08.608468645 (21h9m43.123571226s) [upload=21875]: Upload finished. err: <nil> Aug 8 06:49:53.145025965 (21h10m27.660128534s) [upload=21886]: Starting upload Aug 8 06:49:53.145086326 (21h10m27.660188904s) [upload=21886]: Done starting upload Aug 8 06:49:53.145166436 (21h10m27.660269004s) [upload=21886]: Uploading segments... Aug 8 06:49:53.145207405 (21h10m27.66030999s) [upload=21886]: Got next segment. Inline: true Aug 8 06:49:53.145212962 (21h10m27.660315539s) [upload=21886]: Waiting for error group managing segments... Aug 8 06:49:53.145265613 (21h10m27.660368179s) [upload=21886]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:49:53.145270200 (21h10m27.660372767s) [upload=21886]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:49:53.145274109 (21h10m27.660376686s) [upload=21886]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:49:53.205847742 (21h10m27.720950321s) [upload=21886]: Done uploading segments... Aug 8 06:49:53.205867747 (21h10m27.720970327s) [upload=21886]: Upload finished. err: <nil> Aug 8 06:50:45.962226830 (21h11m20.477329397s) [upload=21901]: Starting upload Aug 8 06:50:45.962960747 (21h11m20.47806339s) [upload=21901]: Done starting upload Aug 8 06:50:45.963081429 (21h11m20.478184007s) [upload=21901]: Uploading segments... Aug 8 06:50:45.963344944 (21h11m20.478447529s) [upload=21901]: Got next segment. Inline: true Aug 8 06:50:45.963354188 (21h11m20.478456768s) [upload=21901]: Waiting for error group managing segments... Aug 8 06:50:45.963408259 (21h11m20.478510836s) [upload=21901]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:50:45.963413437 (21h11m20.478516016s) [upload=21901]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:50:45.963418686 (21h11m20.478521263s) [upload=21901]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:50:46.026766649 (21h11m20.541869227s) [upload=21901]: Done uploading segments... Aug 8 06:50:46.026795865 (21h11m20.541898441s) [upload=21901]: Upload finished. err: <nil> Aug 8 06:51:58.480831248 (21h12m32.995933826s) [upload=21921]: Starting upload Aug 8 06:51:58.480895665 (21h12m32.995998248s) [upload=21921]: Done starting upload Aug 8 06:51:58.481023418 (21h12m32.996125988s) [upload=21921]: Uploading segments... Aug 8 06:51:58.481086289 (21h12m32.996188877s) [upload=21921]: Got next segment. Inline: true Aug 8 06:51:58.481091513 (21h12m32.99619409s) [upload=21921]: Waiting for error group managing segments... Aug 8 06:51:58.481149620 (21h12m32.996252197s) [upload=21921]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:51:58.481154789 (21h12m32.996257366s) [upload=21921]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:51:58.481158546 (21h12m32.996261124s) [upload=21921]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:51:58.722055331 (21h12m33.237157908s) [upload=21921]: Done uploading segments... Aug 8 06:51:58.722073114 (21h12m33.23717568s) [upload=21921]: Upload finished. err: <nil> Aug 8 06:52:27.059254149 (21h13m1.57435673s) [upload=21928]: Starting upload Aug 8 06:52:27.059321812 (21h13m1.574424392s) [upload=21928]: Done starting upload Aug 8 06:52:27.059401946 (21h13m1.574504512s) [upload=21928]: Uploading segments... Aug 8 06:52:27.059448476 (21h13m1.574551053s) [upload=21928]: Got next segment. Inline: true Aug 8 06:52:27.059454258 (21h13m1.574556833s) [upload=21928]: Waiting for error group managing segments... Aug 8 06:52:27.059501468 (21h13m1.574604035s) [upload=21928]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:52:27.059506838 (21h13m1.574609416s) [upload=21928]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:52:27.059510827 (21h13m1.574613404s) [upload=21928]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:52:27.120385847 (21h13m1.635488418s) [upload=21928]: Done uploading segments... Aug 8 06:52:27.120403445 (21h13m1.635506012s) [upload=21928]: Upload finished. err: <nil> Aug 8 06:52:45.219137365 (21h13m19.734239945s) [upload=21933]: Starting upload Aug 8 06:52:45.219214556 (21h13m19.734317133s) [upload=21933]: Done starting upload Aug 8 06:52:45.219284289 (21h13m19.734386877s) [upload=21933]: Uploading segments... Aug 8 06:52:45.219335736 (21h13m19.73443834s) [upload=21933]: Got next segment. Inline: true Aug 8 06:52:45.219342793 (21h13m19.734445378s) [upload=21933]: Waiting for error group managing segments... Aug 8 06:52:45.219397449 (21h13m19.734500028s) [upload=21933]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:52:45.219402775 (21h13m19.734505359s) [upload=21933]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:52:45.219407364 (21h13m19.734509958s) [upload=21933]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:52:45.282551475 (21h13m19.79765406s) [upload=21933]: Done uploading segments... Aug 8 06:52:45.282572182 (21h13m19.797674765s) [upload=21933]: Upload finished. err: <nil> Aug 8 06:53:36.462986837 (21h14m10.978089404s) [upload=21951]: Starting upload Aug 8 06:53:36.463070050 (21h14m10.978172618s) [upload=21951]: Done starting upload Aug 8 06:53:36.463162828 (21h14m10.978265396s) [upload=21951]: Uploading segments... Aug 8 06:53:36.463220839 (21h14m10.978323421s) [upload=21951]: Got next segment. Inline: true Aug 8 06:53:36.463227617 (21h14m10.978330198s) [upload=21951]: Waiting for error group managing segments... Aug 8 06:53:36.463290301 (21h14m10.978392878s) [upload=21951]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:53:36.463295593 (21h14m10.978398162s) [upload=21951]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:53:36.463299324 (21h14m10.978401891s) [upload=21951]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:53:36.745725330 (21h14m11.260827913s) [upload=21951]: Done uploading segments... Aug 8 06:53:36.745748524 (21h14m11.260851096s) [upload=21951]: Upload finished. err: <nil> Aug 8 06:54:42.211460075 (21h15m16.726562642s) [upload=21971]: Starting upload Aug 8 06:54:42.211544762 (21h15m16.726647342s) [upload=21971]: Done starting upload Aug 8 06:54:42.211670011 (21h15m16.726772576s) [upload=21971]: Uploading segments... Aug 8 06:54:42.211709612 (21h15m16.726812197s) [upload=21971]: Got next segment. Inline: true Aug 8 06:54:42.211714530 (21h15m16.726817106s) [upload=21971]: Waiting for error group managing segments... Aug 8 06:54:42.211773234 (21h15m16.726875811s) [upload=21971]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:54:42.211777907 (21h15m16.726880484s) [upload=21971]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:54:42.211781406 (21h15m16.726883982s) [upload=21971]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:54:42.285628155 (21h15m16.800730732s) [upload=21971]: Done uploading segments... Aug 8 06:54:42.285645434 (21h15m16.800748001s) [upload=21971]: Upload finished. err: <nil> Aug 8 06:55:07.196164830 (21h15m41.711267417s) [upload=21978]: Starting upload Aug 8 06:55:07.196239324 (21h15m41.711341924s) [upload=21978]: Done starting upload Aug 8 06:55:07.196316739 (21h15m41.711419326s) [upload=21978]: Uploading segments... Aug 8 06:55:07.196366345 (21h15m41.711468934s) [upload=21978]: Got next segment. Inline: true Aug 8 06:55:07.196372407 (21h15m41.711474978s) [upload=21978]: Waiting for error group managing segments... Aug 8 06:55:07.196426504 (21h15m41.711529089s) [upload=21978]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:55:07.196433124 (21h15m41.711535688s) [upload=21978]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:55:07.196438529 (21h15m41.711541093s) [upload=21978]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:55:07.258380171 (21h15m41.773482756s) [upload=21978]: Done uploading segments... Aug 8 06:55:07.258413055 (21h15m41.773515634s) [upload=21978]: Upload finished. err: <nil> Aug 8 06:56:17.251734653 (21h16m51.766837222s) [upload=21994]: Starting upload Aug 8 06:56:17.251784283 (21h16m51.766886851s) [upload=21994]: Done starting upload Aug 8 06:56:17.251882450 (21h16m51.766985019s) [upload=21994]: Uploading segments... Aug 8 06:56:17.251922005 (21h16m51.767024587s) [upload=21994]: Got next segment. Inline: true Aug 8 06:56:17.251926655 (21h16m51.767029231s) [upload=21994]: Waiting for error group managing segments... Aug 8 06:56:17.251978681 (21h16m51.767081246s) [upload=21994]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 06:56:17.251983158 (21h16m51.767085734s) [upload=21994]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 06:56:17.251986429 (21h16m51.767089005s) [upload=21994]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 06:56:17.318329259 (21h16m51.833431843s) [upload=21994]: Done uploading segments... Aug 8 06:56:17.318350310 (21h16m51.833452887s) [upload=21994]: Upload finished. err: <nil>
Duration: 209.90075ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|