将设为首页浏览此站
开启辅助访问 天气与日历 收藏本站联系我们切换到窄版

易陆发现论坛

 找回密码
 开始注册
查看: 228|回复: 1
收起左侧

HEALTH_WARN Degraded data redundancy ... has slow ops

[复制链接]
发表于 2021-10-25 17:00:42 | 显示全部楼层 |阅读模式

马上注册,结交更多好友,享用更多功能,让你轻松玩转社区。

您需要 登录 才可以下载或查看,没有帐号?开始注册

x
HEALTH_WARN Degraded data redundancy: 822/429494 objects degraded (0.191%), 115 pgs degraded; 154 slow ops, oldest one blocked for 389 sec, mon.ceph1 has slow ops
! |% g& T& w4 y$ j' OPG_DEGRADED Degraded data redundancy: 822/429494 objects degraded (0.191%), 115 pgs degraded; d; l$ j* T7 v+ S  w9 `
    pg 8.473 is active+undersized+degraded, acting [18,9]
+ U% s: {/ \# G; t    pg 8.483 is active+undersized+degraded, acting [9,20]
8 W9 Y, e4 }; m4 d    pg 8.48a is active+undersized+degraded, acting [11,21]+ D9 y+ m1 q' ?5 |& R
    pg 8.49d is active+undersized+degraded, acting [23,6]% a) V8 l% d, p
    pg 8.4a7 is active+undersized+degraded, acting [11,20]
+ b6 |( L- g6 n6 r9 N    pg 8.4a8 is active+undersized+degraded, acting [8,20]
+ |+ [  \* L# H; Q    pg 8.4a9 is active+undersized+degraded, acting [18,8]
+ _, P2 C& E$ H  u5 Q! T! c    pg 8.4b2 is active+undersized+degraded, acting [11,21]: V/ W! D( F2 j4 a
    pg 8.4c8 is active+undersized+degraded, acting [7,22]
  {* i- Z! _7 x4 |    pg 8.4e4 is active+undersized+degraded, acting [9,20]9 B6 |2 p: t( |7 ?) ]1 u! C
    pg 8.4e7 is active+undersized+degraded, acting [18,11]$ y5 `: ]1 _7 r. L
    pg 8.4f2 is active+undersized+degraded, acting [23,8]& w( b2 T5 j6 X& [% w
    pg 8.4fb is active+undersized+degraded, acting [18,7]
' {3 a- o6 w6 x4 w% j    pg 8.505 is active+undersized+degraded, acting [8,21]
( u3 f/ B, E! ]0 `3 J0 `% C' }, R4 ]    pg 8.54e is active+undersized+degraded, acting [8,22]
( F5 h) T1 s' x4 t) v2 P    pg 8.566 is active+undersized+degraded, acting [23,10]
, \; w9 V. O4 I  F5 C0 }: v: q    pg 8.567 is active+undersized+degraded, acting [8,19]% [: p4 S' v/ A/ S2 t  ~
    pg 8.57c is active+undersized+degraded, acting [23,9]
' v( M( l0 M9 B; V5 {$ U- f    pg 8.57d is active+undersized+degraded, acting [8,22]
6 z( C8 X- F8 y    pg 8.596 is active+undersized+degraded, acting [23,8]1 @4 i$ s* D3 e+ R; w
    pg 8.5b3 is active+undersized+degraded, acting [9,18]
- x$ n" a5 G0 u. H4 o+ h    pg 8.5fc is active+undersized+degraded, acting [8,19]- M9 g% ~. E6 _. C1 Q1 a  b
    pg 8.606 is active+undersized+degraded, acting [8,19]
$ G+ R! k$ X6 a8 E# L    pg 8.627 is active+undersized+degraded, acting [11,22]7 {9 t$ q2 t( w0 f1 A! g; f: Y
    pg 8.636 is active+recovery_wait+degraded, acting [15,21,10]
2 B& h1 a5 D4 }# b- G    pg 8.651 is active+undersized+degraded, acting [23,6]7 K! r0 [3 B: i3 L/ d
    pg 8.65b is active+undersized+degraded, acting [11,21]
- Y8 {( v2 R4 T: J8 e6 M    pg 8.666 is active+undersized+degraded, acting [23,6]0 y, k8 ]9 v$ X3 g1 Z, V, {: X
    pg 8.679 is active+undersized+degraded, acting [11,21]
8 P  r5 ^- ], k; D8 q! X    pg 8.6a3 is active+undersized+degraded, acting [7,22]
3 F3 c1 _2 C: x9 a    pg 8.6c3 is active+undersized+degraded, acting [23,6]* c2 ]! p  r7 I' R  h
    pg 8.6e1 is active+undersized+degraded, acting [8,18]' ~$ ~  Y0 a$ K; B6 M
    pg 8.6ed is active+undersized+degraded, acting [8,22]& m. C/ g# `( v; Q
    pg 8.701 is active+undersized+degraded, acting [7,19]0 }* K+ E3 B3 D2 b$ J4 S
    pg 8.724 is active+undersized+degraded, acting [23,10]
$ m  p. Q! f, _: ^: L) k( Y    pg 8.728 is active+undersized+degraded, acting [18,10]! ~, V1 A, G/ R
    pg 8.72f is active+undersized+degraded, acting [7,23]
/ [  l9 A) @4 A1 m( A$ Z9 Y& j    pg 8.732 is active+undersized+degraded, acting [9,18], S  p, g) |+ ?& o& X4 ]" d
    pg 8.739 is active+undersized+degraded, acting [9,22]# g/ M. Z5 ~' \. l7 g
    pg 8.76a is active+undersized+degraded, acting [11,21]
+ M- r" @" b( s    pg 8.770 is active+undersized+degraded, acting [23,9]- u) F7 k; x7 W8 S5 O  O4 Z& h! H
    pg 8.781 is active+undersized+degraded, acting [7,20]8 O. t9 v  K0 n' K
    pg 8.791 is active+undersized+degraded, acting [11,22]
  c' T3 ]4 ?8 }& w* s4 I    pg 8.7b5 is active+undersized+degraded, acting [23,10]1 n" {: v$ Q$ ~+ a( ^
    pg 8.7bb is active+undersized+degraded, acting [8,18]
5 u+ r  g  m3 V% @    pg 8.7be is active+recovery_wait+degraded, acting [15,19,6]5 ^$ w3 T( h# R& j
    pg 8.7c3 is active+undersized+degraded, acting [23,9], E) G! z. v$ K- h
    pg 8.7d4 is active+undersized+degraded, acting [9,18]
& \( O0 |. x1 H: `; J! b9 ]* L) q    pg 8.7da is active+undersized+degraded, acting [18,10]0 i9 N! X  b2 A9 i& ~
    pg 8.7e5 is active+undersized+degraded, acting [18,8]
0 T9 B0 }' u/ l: w3 ?1 ]    pg 8.7fa is active+undersized+degraded, acting [8,21]
" }" ~& O; M) R2 {3 E' J, L4 w0 XSLOW_OPS 154 slow ops, oldest one blocked for 389 sec, mon.ceph1 has slow ops
% i$ u! D! d8 a, ]# E4 k5 B; ?) H3 ~3 ]7 S( D3 G
 楼主| 发表于 2021-10-25 17:04:32 | 显示全部楼层
解决办法:
: w1 Y: T$ f, L' `systemctl restart ceph-mon@`hostname -s`.service
6 C( O. X* v+ g  |2 g/ G3 L
9 s$ v( U! E1 S/ h4 o; |3 G
2 B' Q, _# P/ S, o! B, j) N报mon的错误被清除了。2 s; }; l/ C5 \0 y$ h( b
/ m+ E; L7 T  L. H' {6 |1 H* R/ {; \
[root@ceph1 ~]# ceph health detail + J3 `  t* p& x1 S- J# q* U& m
HEALTH_WARN 6 osds down; Reduced data availability: 170 pgs inactive; Degraded data redundancy: 9370/429503 objects degraded (2.182%), 1102 pgs degraded1 T& G2 w5 k- z* G! d) I/ b' N: o/ P! i
OSD_DOWN 6 osds down7 Y, t) K/ Q* n, U
    osd.20 (root=hdd,host=hdd-ceph3) is down
1 M9 s, H( z1 e/ }# v    osd.22 (root=hdd,host=hdd-ceph3) is down
) p# u, d8 D- p! D# F% B3 F    osd.23 (root=hdd,host=hdd-ceph3) is down" n/ Y; ~) o8 }1 }) q" n
    osd.25 (root=ssd-openstack,host=ssd-compute1) is down9 ]* y; o" M1 m" X( b' y
    osd.28 (root=ssd-openstack,host=ssd-compute2) is down: @2 t; B5 v; J# A! Z# q
    osd.29 (root=ssd-openstack,host=ssd-compute2) is down
; l- l  R& l/ uPG_AVAILABILITY Reduced data availability: 170 pgs inactive
% H) O3 r6 }" t, L    pg 7.13a is stuck inactive for 268.657552, current state undersized+peered, last acting [7]
: w# f, X2 B8 j- f    pg 7.140 is stuck inactive for 173.462639, current state undersized+peered, last acting [6]
) |4 D. A* u* z) ^8 K! L" m    pg 7.149 is stuck inactive for 188.507770, current state undersized+peered, last acting [15]
/ h3 D' @* E+ P8 b: M" t8 i6 `- ^; @    pg 7.14a is stuck inactive for 296.614537, current state undersized+peered, last acting [11]  ^; y; b+ J% O& _* r3 S
    pg 7.15a is stuck inactive for 230.993717, current state undersized+peered, last acting [7]
) a1 R. s/ H  g( V    pg 7.160 is stuck inactive for 157.298697, current state undersized+peered, last acting [16]6 G8 R7 l  ^$ C5 z* y& T
    pg 7.162 is stuck inactive for 168.378304, current state undersized+peered, last acting [8], f' l9 l$ N$ x+ e3 s, q( a
    pg 7.163 is stuck inactive for 230.964708, current state undersized+peered, last acting [11]; k* r; T& a. ?8 W1 q
    pg 7.164 is stuck inactive for 193.535188, current state undersized+peered, last acting [12]
+ y2 `9 h! P8 |. f    pg 7.166 is stuck inactive for 157.314779, current state undersized+peered, last acting [9]7 P+ j0 S# b3 f' [) z
    pg 7.167 is stuck inactive for 231.001296, current state undersized+peered, last acting [14]
: v) [$ N+ F3 ?    pg 7.16f is stuck inactive for 157.301826, current state undersized+peered, last acting [6]4 `3 u* o2 s2 V1 Q& R
    pg 7.18e is stuck inactive for 230.971182, current state undersized+peered, last acting [17]
3 K3 F" \9 _! _2 W9 s$ p  T    pg 7.196 is stuck inactive for 157.295252, current state undersized+peered, last acting [10]
' `0 I/ [  r+ w7 C- u    pg 7.332 is stuck inactive for 188.514026, current state undersized+peered, last acting [15]" z; \- f1 I7 R
    pg 7.333 is stuck inactive for 230.989099, current state undersized+peered, last acting [17]! n/ Z2 q2 Q, {3 u* U
    pg 7.347 is stuck inactive for 243.608764, current state undersized+peered, last acting [14]
4 f0 @0 T, j2 S0 a+ h* _; t2 k    pg 7.352 is stuck inactive for 230.961993, current state undersized+peered, last acting [17]+ U$ N9 _" U6 y! Q, l! S2 i" s2 P
    pg 7.358 is stuck inactive for 296.668417, current state undersized+peered, last acting [13]8 a" ]- C. a/ _  A3 e, H9 _/ A5 u
    pg 7.35b is stuck inactive for 296.611193, current state undersized+peered, last acting [9]; E4 j% f' g" N% @- W/ O  J
    pg 7.364 is stuck inactive for 231.014057, current state undersized+peered, last acting [7]
6 Y; a  n" U; z# C% D/ A    pg 7.372 is stuck inactive for 230.976751, current state undersized+peered, last acting [11]# g+ G( S6 l8 x* L& R+ u/ ?2 O
    pg 7.374 is stuck inactive for 157.314500, current state undersized+peered, last acting [13]& }% L* b: M4 l( Y, x1 w
    pg 7.375 is stuck inactive for 268.647540, current state undersized+peered, last acting [7]
* Z  g& k* h" G1 \  s" C" w; I    pg 7.376 is stuck inactive for 157.302772, current state undersized+peered, last acting [7]
: q. p& ?( d4 ?    pg 7.37c is stuck inactive for 157.292778, current state undersized+peered, last acting [9]
8 B2 ^% C- s- p8 f& J    pg 7.381 is stuck inactive for 268.670664, current state undersized+peered, last acting [7]
9 Z; l6 {, Q- a9 {    pg 7.38a is stuck inactive for 157.311084, current state undersized+peered, last acting [6]# P/ y0 m5 q# M4 G* ~9 k
    pg 7.38b is stuck inactive for 230.974616, current state undersized+peered, last acting [10]5 K$ o! t1 _' {
    pg 7.391 is stuck inactive for 253.571528, current state undersized+peered, last acting [10]
# U! a( X. }' E2 O* R    pg 7.399 is stuck inactive for 231.014969, current state undersized+peered, last acting [7]
# F7 ?5 X4 i% x4 ~7 ?) ?    pg 7.3a0 is stuck inactive for 157.314500, current state undersized+peered, last acting [17]
5 S9 w4 ?0 L* Z! p" d: m    pg 7.3a3 is stuck inactive for 157.310316, current state undersized+peered, last acting [16]  A8 w6 M. S% L* K/ z' Z
    pg 7.3ad is stuck inactive for 243.592394, current state undersized+peered, last acting [14]/ \% r1 T$ {! w& e) Z9 m
    pg 7.3af is stuck inactive for 188.502747, current state undersized+peered, last acting [15]
3 s/ p6 D( l9 d0 Q3 p# J8 M/ V    pg 7.3b2 is stuck inactive for 168.373475, current state undersized+peered, last acting [8]7 Z4 q4 e2 G4 I  f' ?  [
    pg 7.3b4 is stuck inactive for 173.465040, current state undersized+peered, last acting [6]
1 E1 ?% E4 w. x" D3 I    pg 7.3b5 is stuck inactive for 157.322812, current state undersized+peered, last acting [9]/ @, ~4 I0 @  ]4 G
    pg 7.3b6 is stuck inactive for 230.969301, current state undersized+peered, last acting [11]
" v) F1 A/ U! c; [    pg 7.3b9 is stuck inactive for 157.296625, current state undersized+peered, last acting [14]* {4 t# D+ Y2 l' p* c
    pg 7.3ba is stuck inactive for 231.015066, current state undersized+peered, last acting [7]8 P! u) J. H. W9 N
    pg 7.3be is stuck inactive for 157.311582, current state undersized+peered, last acting [8], l: `8 a0 L- T/ n* @% p7 o
    pg 7.3c5 is stuck inactive for 168.378059, current state undersized+peered, last acting [8]" g" g1 A' w1 X  \& n7 D
    pg 7.3cd is stuck inactive for 230.974691, current state undersized+peered, last acting [10]+ k( O% x! X. W/ {* b$ ~
    pg 7.3cf is stuck inactive for 168.383761, current state undersized+peered, last acting [8]. L9 L/ d% V' b& C" e
    pg 7.3df is stuck inactive for 157.314279, current state undersized+peered, last acting [10]
9 U) \4 ?  U; I( {  q    pg 7.3e3 is stuck inactive for 296.651325, current state undersized+peered, last acting [16]
( b+ F7 n$ x) B' F9 g7 \: u8 Y    pg 7.3eb is stuck inactive for 173.465968, current state undersized+peered, last acting [6]% {1 h4 N0 E. ?# v6 j5 m) D
    pg 7.3f8 is stuck inactive for 230.983836, current state undersized+peered, last acting [11]
# T$ z. t, s* F% y. o8 S    pg 7.3fb is stuck inactive for 157.305951, current state undersized+peered, last acting [16]
5 J/ Y- j! G+ _' q    pg 7.3ff is stuck inactive for 268.659557, current state undersized+peered, last acting [7], F! ~+ N  T+ D7 ~
PG_DEGRADED Degraded data redundancy: 9370/429503 objects degraded (2.182%), 1102 pgs degraded
  G' ]: [2 m1 d. \4 j3 a    pg 8.795 is active+undersized+degraded, acting [11,17]" p6 z2 @+ u% B6 y$ [* l
    pg 8.79b is active+undersized+degraded, acting [10,12]
- \1 A% j( I8 g3 k5 t3 ?    pg 8.79c is active+undersized+degraded, acting [13,10]3 o/ A; t! Y# e% g  R7 j  Q" [" m
    pg 8.79f is active+undersized+degraded, acting [9,13]
  V1 {, I( ]4 f  ?9 Z) P& J2 o& t    pg 8.7a0 is active+undersized+degraded, acting [8,17]0 z7 l" ~" t" a1 Y7 U
    pg 8.7a1 is active+undersized+degraded, acting [16,9]3 F, r$ o0 }% x4 ?" p0 \
    pg 8.7a2 is active+undersized+degraded, acting [10,17]
- c2 _" p( i6 ~( w" k* r    pg 8.7a3 is active+undersized+degraded, acting [17,11]
& r+ ]+ j% }( k. A$ A$ r/ t    pg 8.7a5 is active+undersized+degraded, acting [11,17]
1 r  Z  p/ g, I. _    pg 8.7a8 is active+undersized+degraded, acting [13,7]  |! I; d/ q) Q- p- E
    pg 8.7a9 is active+undersized+degraded, acting [15,7]
2 s' E9 `, J" R; U; i3 c    pg 8.7ad is active+undersized+degraded, acting [15,9]
- @; a" d/ S9 }6 c4 N$ K" m    pg 8.7ae is active+undersized+degraded, acting [14,10]0 c0 @( U! c6 d% E9 ~- n: ]& {
    pg 8.7b0 is active+undersized+degraded, acting [12,6]
$ |6 \5 l" N0 @    pg 8.7b5 is active+undersized+degraded, acting [10,15]8 S9 X8 r( k! K* H5 G/ l
    pg 8.7b6 is active+undersized+degraded, acting [15,9]- i3 o9 P# `- v: ]
    pg 8.7b8 is active+undersized+degraded, acting [8,16]
4 R2 ?% O2 ~' P$ B( `4 Z  ]# U( T    pg 8.7ba is active+undersized+degraded, acting [13,6]
+ }. f) W3 C" a  W    pg 8.7bd is active+undersized+degraded, acting [7,13]: x- |3 _2 k. R2 Q4 U. C
    pg 8.7bf is active+undersized+degraded, acting [17,11]& R. n1 T$ r! R% w4 }
    pg 8.7c1 is active+undersized+degraded, acting [12,10]) A8 L( P3 D* R; w9 I6 r9 s
    pg 8.7c2 is active+undersized+degraded, acting [6,12]- L& M: Q5 ?1 |# G2 Y1 K, h/ n# k
    pg 8.7c3 is active+undersized+degraded, acting [9,15]8 y2 ~+ e" ?, t. Q/ f% Q# H
    pg 8.7c4 is active+undersized+degraded, acting [6,16]
/ q. U+ ]& _3 c' I    pg 8.7c5 is active+undersized+degraded, acting [8,14]/ |8 L! P: P+ [+ B* o
    pg 8.7c6 is active+undersized+degraded, acting [17,7]( J. a+ d& [- |# `+ X- [- v
    pg 8.7c7 is active+undersized+degraded, acting [16,11]
5 B+ a! E. i; W7 B& B( i/ n1 b& g2 V    pg 8.7ca is active+undersized+degraded, acting [16,11]
2 o! B& |0 l" }. P8 V' n    pg 8.7cd is active+undersized+degraded, acting [13,6]
& h+ w0 p- _: Y7 K& K/ _6 c& X    pg 8.7ce is active+undersized+degraded, acting [13,8]
; E! w0 @2 W8 B7 w    pg 8.7d0 is active+undersized+degraded, acting [17,10]
6 g5 u4 |/ c" {% l9 u+ \    pg 8.7d5 is active+undersized+degraded, acting [14,11]: A4 O2 L% [6 x7 Y' n- p, p8 s
    pg 8.7d8 is active+undersized+degraded, acting [13,10]
/ X# ~. P" I( t* u' I& r8 ?    pg 8.7d9 is active+undersized+degraded, acting [9,13]
# H" p! h2 j3 R    pg 8.7dd is active+undersized+degraded, acting [12,11]1 F9 ~& D, }1 c; v9 q
    pg 8.7df is active+undersized+degraded, acting [11,16]
1 V2 {) u3 ^6 N% ]% v/ h+ N& A4 ^    pg 8.7e0 is active+undersized+degraded, acting [17,6]
" z# M+ m; a' {, _9 J    pg 8.7e1 is active+undersized+degraded, acting [14,10]* e$ r9 Z& o, j6 m0 U
    pg 8.7e2 is active+undersized+degraded, acting [9,13]/ |1 d! b. O" W7 N
    pg 8.7e3 is active+undersized+degraded, acting [14,9]7 l$ t3 a( b; b- u3 K$ \- ~$ D+ _
    pg 8.7e7 is active+undersized+degraded, acting [6,15]/ Y  S( m* u% Q: d. {$ S
    pg 8.7ea is active+undersized+degraded, acting [9,16]
* e; D" \: O# Q  ?# _0 @7 P/ s    pg 8.7eb is active+undersized+degraded, acting [14,9]7 V* Z' [8 j8 R# c
    pg 8.7ec is active+undersized+degraded, acting [16,11]
6 S! ?$ I1 R+ O# e5 C    pg 8.7ed is active+undersized+degraded, acting [14,10]
+ Q6 u9 y( V3 Y6 L' A/ u& ]! `    pg 8.7f0 is active+undersized+degraded, acting [17,6]
, B! T  e3 Q( m: t- _& }    pg 8.7f1 is active+undersized+degraded, acting [15,7]5 A. _/ ~) E& l; p% Q6 s1 ~
    pg 8.7f3 is active+undersized+degraded, acting [9,17]+ M( c) w6 D) K: }
    pg 8.7f4 is active+undersized+degraded, acting [9,14]- R  Z  T1 @' S
    pg 8.7fb is active+undersized+degraded, acting [9,17], e' M2 r, E/ D6 U4 A/ C. ]- q! F
    pg 8.7fc is active+undersized+degraded, acting [15,8]
. A7 L  f4 q7 b+ `( f
您需要登录后才可以回帖 登录 | 开始注册

本版积分规则

关闭

站长推荐上一条 /4 下一条

如有购买积分卡请联系497906712

QQ|返回首页|Archiver|手机版|小黑屋|易陆发现 点击这里给我发消息

GMT+8, 2021-11-28 18:39 , Processed in 0.042465 second(s), 23 queries .

Powered by LR.LINUX.cloud bbs168x X3.2 Licensed

© 2012-2022 Comsenz Inc.

快速回复 返回顶部 返回列表