将设为首页浏览此站
开启辅助访问 天气与日历 收藏本站联系我们切换到窄版

易陆发现论坛

 找回密码
 开始注册
查看: 135|回复: 1
收起左侧

记一次ceph存储同步问题noout,nobackfill,norecover flag(s) set Degraded data r

[复制链接]
发表于 2021-12-20 11:00:13 | 显示全部楼层 |阅读模式

马上注册,结交更多好友,享用更多功能,让你轻松玩转社区。

您需要 登录 才可以下载或查看,没有帐号?开始注册

x
检查ceph 状态:
+ O. Z9 u4 ~! T[root@compute01 ~]# ceph -s* }# F( N# e0 _
  cluster:8 p2 f7 Z6 X1 w/ ~+ l% Q
    id:     a6d0359e-8295-4e62-8620-54273462e79f
& R* {2 E5 B; \9 ^1 ]: S    health: HEALTH_WARN
, n( z8 e2 |7 T+ I% s5 d( F            noout,nobackfill,norecover flag(s) set  g& o2 ^3 g) O/ ^2 G- g
            Degraded data redundancy: 3793/96483 objects degraded (3.931%), 876 pgs degraded, 512 pgs undersized: g( v2 n! E" @" j0 t! J! x! b
            6 pgs not deep-scrubbed in time
7 a( p- V( m& p2 A# \! T0 y            clock skew detected on mon.compute02, mon.compute03. k: X8 l" K0 K
& U; h) ^3 \0 S$ y+ c9 ^! R5 v
  services:
3 T; _9 t7 _* }7 J8 G    mon: 3 daemons, quorum compute01,compute02,compute03 (age 22m)
, O2 Z7 e, E( |& C) u7 q    mgr: compute02(active, since 4d), standbys: compute03, compute01
" \! }+ O+ O7 q% L1 S1 w) D+ [    osd: 6 osds: 6 up (since 22m), 6 in (since 2w); 512 remapped pgs4 F  t! w9 V, m& Z8 F
         flags noout,nobackfill,norecover% s& {$ @5 H8 X  M; ~

  }. ^# P( U* v6 Y- f' P/ o  data:" u  I/ v* g. r) r. X$ \0 [3 g4 p9 T
    pools:   5 pools, 1408 pgs( T7 f3 |0 i+ `, A
    objects: 32.65k objects, 132 GiB
2 K+ ~; j/ c& {    usage:   381 GiB used, 4.9 TiB / 5.2 TiB avail
  X6 {8 I: k4 u1 Z3 k    pgs:     3793/96483 objects degraded (3.931%)& r( @( j. M& W. q/ R
             450 active+recovery_wait+undersized+degraded+remapped4 x/ D  O6 w6 V- H5 L
             444 active+clean3 ?8 E/ d8 l1 Z: t; `0 h  T
             362 active+recovery_wait+degraded0 A6 k5 @! t9 V+ G" {
             87  active+recovery_wait
: k- P2 W5 e. [' m9 U" C             62  active+undersized+degraded+remapped+backfill_wait
  K! \- p- J4 n) n/ y             2   active+recovering+degraded4 o/ {- {7 Q7 e5 E  d5 l
             1   active+recovering' V6 \& a$ S5 S/ E, H
! g. C. j+ n. @0 P2 a) E7 A% I0 n1 l
  io:
- h& i* a  `4 r    client:   156 KiB/s rd, 78 KiB/s wr, 14 op/s rd, 8 op/s wr, O5 Y. {6 J, i: p9 E
7 ?  O5 r' v* v% ?: D/ q
. _! ~- J: z# c. L8 E9 t
设置ceph osd 开启recover和覆盖同步配置:0 y7 k" q0 Z7 A- g9 J- q: n. M' b& B

$ v2 p: C$ e+ Y: [$ G' O4 c; b3 S
[root@compute01 ~]# ceph osd unset norecover
* _$ v; H' T% [norecover is unset
( `2 U- \. t) a, ?. ~% X[root@compute01 ~]# ceph osd unset nobackfill . Y8 X9 p- q3 c8 y$ R
nobackfill is unset
# |$ f" Y0 i6 d# m+ P8 p
; M' b, K% `. B/ b查看状态:0 R, d) P# y4 n9 J7 H$ J  u
[root@compute01 ~]# ceph -s6 E: x: @& f- F- c
  cluster:
! U$ U5 ]7 _4 [5 I1 A" \/ Y' h    id:     a6d0359e-8295-4e62-8620-54273462e79f
7 H; u$ R" e/ Q1 J0 P5 @    health: HEALTH_WARN9 Z$ F* s& C6 v: f7 E
            noout flag(s) set
( o  G" C0 X- T- ?- Q3 ^9 I; U            Degraded data redundancy: 3537/96486 objects degraded (3.666%), 705 pgs degraded, 512 pgs undersized
8 U* {( r( {6 s3 N  K! @/ c            6 pgs not deep-scrubbed in time1 m5 _6 Z: {+ E4 D# y
            clock skew detected on mon.compute02, mon.compute03
4 B3 y6 g" s. S8 `  Y8 h
1 \: b$ |- i% q9 D4 @5 Q" x& T  services:
3 M( C! q$ B# g) b2 p$ J    mon: 3 daemons, quorum compute01,compute02,compute03 (age 23m)3 ~& C' _; f; D$ \) F7 ~+ y; Z
    mgr: compute02(active, since 4d), standbys: compute03, compute013 J3 y; T* U3 F( k. ~& r
    osd: 6 osds: 6 up (since 23m), 6 in (since 2w); 512 remapped pgs$ f7 N5 {- Z( j! D
         flags noout
/ c, D  X( y5 b7 v- F2 H8 b, Y
) ^6 W7 b; O& H" T8 Y  data:4 Z. y, s# a! C, u7 K8 o4 c. U
    pools:   5 pools, 1408 pgs4 r9 h# W. ?  z5 h" `
    objects: 32.65k objects, 132 GiB
( N0 a- e' J) V- G- j+ `3 W    usage:   382 GiB used, 4.9 TiB / 5.2 TiB avail
, |4 ]9 r0 O8 U# u) F& {0 k- L, _    pgs:     3537/96486 objects degraded (3.666%)' n6 O, M7 S/ r. T- V! _
             649 active+clean
3 o; F/ g2 |& j. M+ p             450 active+recovery_wait+undersized+degraded+remapped
3 u" P: E; i/ c: ~1 a9 m             191 active+recovery_wait+degraded; m( t9 x: Q4 y- i7 `; W
             62  active+undersized+degraded+remapped+backfill_wait
3 m, D- o& G% ^  G/ H             54  active+recovery_wait
- K; z& d# d' o/ b" i) I& g  Z% h             2   active+recovering+degraded- z& S. G& m- f  Z0 w4 x

! `/ O1 a( X  F/ V  io:0 B' A  i$ U# }& e2 x/ E# z
    client:   204 KiB/s rd, 797 KiB/s wr, 147 op/s rd, 42 op/s wr( v% N7 y0 D3 e) Y" m
    recovery: 89 MiB/s, 21 objects/s
9 y' L% u; w  r% u) u) q0 Y% [3 n 上面看到数据开始在同步了,覆盖了。
$ d  Z% z$ p! x$ G; x) Q, [) J) {
0 F  _+ u! w# O' ]2 `: j* A查看osd是否都正常:
5 r5 E, m- C6 \! ?. U) U[root@compute01 ~]# ceph osd tree7 X6 t3 ?) Q. O# z& C; Q- @6 P
ID  CLASS WEIGHT  TYPE NAME              STATUS REWEIGHT PRI-AFF
2 f" j: T) w/ Z: t/ X0 @1 I; ?: Z-11             0 host ssd-compute05                             
" k5 H6 D% W# B8 L% z8 m -3       6.00000 root ssd                                       0 q3 F& f0 {( D* f. d
-4       2.00000     host ssd-compute01                        
! q" L/ x  Q! M. G( s3 p- ~  0   ssd 1.00000         osd.0              up  1.00000 1.00000
; ~0 {, r7 w5 [3 q3 E, d6 o  1   ssd 1.00000         osd.1              up  1.00000 1.00000
, D* ^, i; S  X) B6 t( x -5       2.00000     host ssd-compute02                        
0 [+ B& F& b& X: ]7 p* K  2   ssd 1.00000         osd.2              up  1.00000 1.00000
* B! b/ {7 r, p# r  3   ssd 1.00000         osd.3              up  1.00000 1.00000
  J1 m  h6 b/ m$ P# S: l -6       2.00000     host ssd-compute03                         ) I- ^, M1 d( `
  5   ssd 1.00000         osd.5              up  1.00000 1.00000
, E8 F+ S, z) i6 F  6   ssd 1.00000         osd.6              up  1.00000 1.00000
6 z# S" C& Y. Z) d8 O -1             0 root default                                   / ~, E6 N/ S7 q  q" E) s. e
. Q: U- C4 o: a  H! H2 n
[root@compute01 ~]# ceph -s6 `$ T+ @: l9 z, A: ^
  cluster:# c4 I, a; m8 T/ R' i/ V+ `2 B
    id:     a6d0359e-8295-4e62-8620-54273462e79f
" M/ a& S+ U$ @6 E    health: HEALTH_WARN& Q7 y( K  e) E; }2 u
            noout flag(s) set
% a2 R% H# H' ~2 H            Reduced data availability: 3 pgs peering2 y! {1 q# r7 w; g4 u: x
            Degraded data redundancy: 2173/96486 objects degraded (2.252%), 125 pgs degraded, 133 pgs undersized
+ I( ^( z$ ?' M' w- o- A' ]# C            6 pgs not deep-scrubbed in time
- M1 b$ L0 a; ^3 x+ d: |% q            clock skew detected on mon.compute02, mon.compute03$ ]# U9 t$ |4 S8 p1 O* N; r
; O* ?2 O2 z& W3 _+ G2 t
  services:1 ?6 H, A5 e7 I1 |, D1 n
    mon: 3 daemons, quorum compute01,compute02,compute03 (age 24m)
+ }' M7 B& y: Y6 ^    mgr: compute02(active, since 4d), standbys: compute03, compute010 u3 t1 {1 q" \& b* P4 ?
    osd: 6 osds: 6 up (since 24m), 6 in (since 2w); 104 remapped pgs9 P! C; m+ H: {& I8 g2 q
         flags noout
" x, a% O" z0 q) j- h! ^! \* V
8 ?7 ?" g/ ]% _: h9 i  data:! R1 _. ^+ P: e! M% N* X
    pools:   5 pools, 1408 pgs- O$ l. v$ G! p2 m
    objects: 32.65k objects, 132 GiB
2 R2 ?# D% t, _/ {* r    usage:   383 GiB used, 4.9 TiB / 5.2 TiB avail
# ?  e7 J4 K; s+ i% _    pgs:     0.568% pgs not active
- u+ @) Z4 C& N             2173/96486 objects degraded (2.252%); D' j2 \1 @4 R5 K, [% f" a
             1267 active+clean
$ g5 x& i, ?# ]; z# ^: o; \             62   active+undersized+degraded+remapped+backfill_wait
9 n# i3 R% l+ T& [( _# c7 C8 b) P             62   active+recovery_wait+undersized+degraded+remapped
1 u6 P2 c+ r' C+ w) C             8    active+undersized+remapped
. G0 c# z/ B4 m0 |" J" m             8    peering0 i8 c* U+ {$ Y& X* A& v  u3 w* x; U
             1    active+recovering+undersized+degraded+remapped
( A" Y, R) h6 m# ^$ F  V1 F% T
- Q) ~( g9 Q! D# s  io:
- @2 F) g& m( ?1 \; r8 I8 }( @  z; @    client:   407 KiB/s rd, 792 KiB/s wr, 34 op/s rd, 52 op/s wr! _* D9 K# r, U- L- `. |- R& i: m) U
    recovery: 72 MiB/s, 2 keys/s, 22 objects/s' R3 p% f7 ~/ t8 f2 P& i

, n; [/ O, S& q. V9 L  s" M* [$ N8 Q! `" W7 q4 G" Y8 h
查看时间:
: x2 ~/ B, z6 }5 f4 g$ K# x8 G: @[root@compute01 ~]# date% M8 C4 O, o- v+ w1 M* H4 p9 q
Mon Dec 20 10:32:23 CST 20217 \! z) G; F. ?* V1 I. O! \
[root@compute01 ~]# date
5 ~: n+ E( m6 H4 ?0 F: O+ nMon Dec 20 10:32:38 CST 2021
- R* l; h/ k7 R" C: `" ][root@compute01 ~]# date
1 A0 ~0 @9 o# S: k% a, G$ LMon Dec 20 10:33:00 CST 2021! K+ I1 A( Z0 Z% i( `

! y- g: v* d& l4 C# u! ^[root@compute01 ~]# chronyc sources ' u: ~, {0 V4 G* f; ]
210 Number of sources = 2
1 I. M; }- p. z& V; lMS Name/IP address         Stratum Poll Reach LastRx Last sample               
# V# x# w& A  {# f===============================================================================
: A( Z; T; J) b6 n7 s6 ?" C^? compute01                     0   9     0     -     +0ns[   +0ns] +/-    0ns
6 r' p+ F1 V/ f. K^? compute02                     0   9     0     -     +0ns[   +0ns] +/-    0ns
- V. G( R* w# `$ a/ o! T4 x[root@compute01 ~]# vim /etc/chrony.conf ; q9 d, A: Q9 O7 u, f; O: M

5 H) L  d- `  A  L, x0 r& Y7 e[root@compute01 ~]# systemctl restart chronyd.service 0 g) V- h% ]2 D& ~
[root@compute01 ~]# chronyc sources
, i4 X3 K% F2 U( G210 Number of sources = 2
' z4 r2 j* c4 M" a1 I8 kMS Name/IP address         Stratum Poll Reach LastRx Last sample               + M- i. p0 x1 l- r/ ]! U' b
===============================================================================) U& i5 ?1 \0 c3 _, h4 b
^* compute01                    10   6    17     3  +2667ns[-7752ns] +/-   28us
( }6 ]) w* u. G, X/ O^? compute02                     0   6     0     -     +0ns[   +0ns] +/-    0ns
3 j$ J, P$ m! b/ j/ o[root@compute01 ~]# 1 e6 z7 h; ~5 B  ?- R
[root@compute01 ~]#
$ V8 h6 @3 E9 q! A  [7 E. d[root@compute01 ~]# vim /etc/chrony.conf 5 |% p: e/ x. Q1 x4 j( ?% e2 x9 S
[root@compute01 ~]# date9 |7 ]0 n; g; x
Mon Dec 20 10:36:29 CST 20211 x% |/ a% R. j1 W
- H1 n0 E  T# }4 o2 s/ l0 y. U/ u
: Q7 W- n* o9 a, C! a. Z8 I
时间同步即可:
" V9 p6 m! _1 V" l' _. I4 A/ f0 M! |- \
[root@compute01 ~]# ceph -s% C& [5 F  X' K- O) V6 P
  cluster:
9 X9 D( T6 G8 s    id:     a6d0359e-8295-4e62-8620-54273462e79f9 y! ~, k4 q, ]! B0 g8 y$ g3 Q
    health: HEALTH_WARN$ k$ k; b% Z- L9 \2 f/ ?
            noout flag(s) set( i; R7 o( w! B+ |% J3 M
            1 pgs not deep-scrubbed in time3 Q) N# L) S9 }8 @
# B, V( u3 O" l/ b7 I
  services:
, F! L; m2 Y2 c2 R    mon: 3 daemons, quorum compute01,compute02,compute03 (age 48s)9 b+ v0 v* w: Y, P; s. c. \
    mgr: compute02(active, since 4d), standbys: compute03, compute01, J: t3 Z/ z& S" `. ?$ U
    osd: 6 osds: 6 up (since 28m), 6 in (since 2w)) w! w- E# E& X6 |3 P
         flags noout* o) i! @# X. d: o6 `4 U( D4 V

+ q) B) {% A! G2 Z% o1 }  data:
, Z9 |" V* c  @5 r2 e& v8 F- A    pools:   5 pools, 1408 pgs# j2 D" p! W8 |0 k" j
    objects: 32.65k objects, 132 GiB" J* z/ X- r) J, O* f
    usage:   384 GiB used, 4.9 TiB / 5.2 TiB avail
. D3 |+ p+ ~3 b5 B- }) g1 j$ W    pgs:     1407 active+clean, |! X4 @4 M5 Q, F
             1    active+clean+scrubbing+deep+ V) v/ C' A9 Z; J5 j, l

0 P3 S( [7 {: j3 |4 T/ L! k  io:
/ o6 e; v* h9 ^; N! ]; g% h0 q    client:   20 KiB/s rd, 499 KiB/s wr, 4 op/s rd, 18 op/s wr
7 X, ]6 a" C) e6 b% F- H$ b; J  @8 b5 c6 Q" Q! c2 _. ~
查看ceph存储是否正常:
! u8 [$ p& s6 C9 W2 ^
- s+ ?% ~* I7 k8 Q[root@compute01 ~]# ceph -s
3 d) a& O& Y7 |1 ~4 _9 W  cluster:
2 G  E# N+ O. C0 y7 \. i    id:     a6d0359e-8295-4e62-8620-54273462e79f
" \1 w& J" C; ?4 R    health: HEALTH_WARN
; O+ B9 ~" K7 S! q5 K; q8 [            noout flag(s) set2 ^. a: j) K& @+ B% }' ]/ M
            1 pgs not deep-scrubbed in time
# U) M4 i! v$ Z3 W7 o
# ^$ C8 {: t; v7 E3 {1 E  services:$ P: v/ T. I, `0 v
    mon: 3 daemons, quorum compute01,compute02,compute03 (age 60s)$ ?9 v; K$ [, H' h! t
    mgr: compute02(active, since 4d), standbys: compute03, compute010 Z& r1 E: h/ k: i8 ?5 `
    osd: 6 osds: 6 up (since 28m), 6 in (since 2w)4 ~  ]1 I' P& A
         flags noout
0 g& c  ^/ q$ }% Y/ A" \, Q6 I5 b  T+ C# F7 `
  data:1 m7 Y: c2 s1 G1 h) H8 K
    pools:   5 pools, 1408 pgs, F4 V+ R6 y7 ?0 P
    objects: 32.65k objects, 132 GiB, `7 C0 \! O: {2 a' B4 a
    usage:   384 GiB used, 4.9 TiB / 5.2 TiB avail+ i! v7 d; L: ^6 q7 F3 Y
    pgs:     1407 active+clean  Z2 |6 T/ z8 O7 A) G8 G
             1    active+clean+scrubbing+deep
  w+ |9 \5 Y8 t$ C, @1 v' g, C6 w- }7 P4 H0 |5 p1 D0 ~
  io:2 C/ V+ Z0 r  A" p9 {* y
    client:   120 KiB/s rd, 595 KiB/s wr, 13 op/s rd, 33 op/s wr. q' p4 J5 I- }) i; g

" G, A- p/ k9 c! a
- q& E% ]1 M% g$ y7 @0 D[root@compute01 ~]# ceph osd unset noout
4 {4 b5 C2 ]6 y+ d% A" k% v) Vnoout is unset- u  u, l' {/ c  \
7 r1 W' U; m2 \% j
检查状态:1 U/ f( P5 H5 }! h  K: W
[root@compute01 ~]# ceph -s3 E( w+ b' g3 M' a
  cluster:/ H1 L7 p) s8 K/ V, g6 S
    id:     a6d0359e-8295-4e62-8620-54273462e79f, q( ]& M1 Y) ]9 N+ f
    health: HEALTH_OK( b7 W2 @; ~+ x, P* z

+ F/ c7 l( n2 v! A/ r3 S  services:" ?/ q5 x* i$ p4 g% k
    mon: 3 daemons, quorum compute01,compute02,compute03 (age 100s)* x# k# H; x/ l& K
    mgr: compute02(active, since 4d), standbys: compute03, compute01. G7 t$ l/ k% N) b/ L7 J0 m1 A3 \7 P5 G
    osd: 6 osds: 6 up (since 29m), 6 in (since 2w)4 S% U% _% N5 d0 G" I0 U% t
* ?0 \% N8 d4 r' R* i4 P
  data:  l5 h8 ?( O. R( u
    pools:   5 pools, 1408 pgs- ]3 {$ d% X0 b3 X$ F+ D
    objects: 32.65k objects, 132 GiB
* C% M: x# |; G& T5 C    usage:   384 GiB used, 4.9 TiB / 5.2 TiB avail# u/ ]' @' @' \  C! R4 W
    pgs:     1408 active+clean1 \5 `2 l8 R7 a* {
8 C% z4 S5 v; x; I
  io:$ S4 r/ ?; I  {1 a
    client:   539 KiB/s rd, 285 KiB/s wr, 191 op/s rd, 26 op/s wr8 J  R7 I9 @  A+ i

2 F4 q& a- n4 l  E7 o[root@compute01 ~]# ceph -s4 D0 z- r$ g3 F+ M
  cluster:
- @% E0 z( s7 P* e7 v" W  K    id:     a6d0359e-8295-4e62-8620-54273462e79f4 ]$ L5 N- H! H0 y& v
    health: HEALTH_OK/ E1 r7 X+ O2 ~8 Z5 K1 d

* H5 {4 ]7 g$ m3 E! \& z+ P5 {  services:
2 _3 {6 g7 W6 n/ `. j    mon: 3 daemons, quorum compute01,compute02,compute03 (age 102s)1 D" k. H8 \( M
    mgr: compute02(active, since 4d), standbys: compute03, compute01  H+ k; t7 Q6 i4 M9 w( m  s
    osd: 6 osds: 6 up (since 29m), 6 in (since 2w)3 Y7 o9 t! d+ U8 q3 i. n2 j  x& ?
) Y! u# i  V! F/ q
  data:9 o  k4 D3 D' n4 Z4 Y
    pools:   5 pools, 1408 pgs+ V8 T/ l2 v6 G- s6 G( c
    objects: 32.65k objects, 132 GiB
% I3 M2 ?' G# w# M% J% K6 |    usage:   384 GiB used, 4.9 TiB / 5.2 TiB avail
/ o3 {4 k0 _- D1 I3 Z2 g    pgs:     1407 active+clean/ [' ~' ]) T+ C0 Z8 C- l
             1    active+clean+scrubbing+deep) \+ W4 J; n6 b+ Q: T8 K& |

5 G& r7 h& ^% k1 F/ G5 H( E  io:0 D% |0 t' }1 S$ m2 K
    client:   110 KiB/s rd, 196 KiB/s wr, 12 op/s rd, 18 op/s wr9 K# B; l: ?8 p* T! k6 K  X
+ [5 Z+ J* q4 {# E- S7 V" A
[root@compute01 ~]# ceph -s$ `& P6 `+ [0 T$ M$ q
  cluster:6 F* Q+ p; }8 _7 v8 F" ]
    id:     a6d0359e-8295-4e62-8620-54273462e79f
9 W, u" B% Z1 t  c5 M! i2 O6 u' S    health: HEALTH_OK; k, [. h) u/ _6 S# z% Q9 k
! Q. [; i. S5 d
  services:
5 G0 s6 s# K) u    mon: 3 daemons, quorum compute01,compute02,compute03 (age 105s)
' M7 l" h: s! ^/ f- [+ \$ ?/ L    mgr: compute02(active, since 4d), standbys: compute03, compute01/ b7 _) z$ I7 T6 V2 a% Y
    osd: 6 osds: 6 up (since 29m), 6 in (since 2w)
3 U/ _0 D7 N  d  H0 Y& L2 b% w7 ]' W. z
  data:
# y( r) g+ m; ~+ W  h    pools:   5 pools, 1408 pgs
6 ~- \; r6 A, o. T$ Z* l, w    objects: 32.65k objects, 132 GiB
6 f! l, t1 F* K0 z& _( e    usage:   384 GiB used, 4.9 TiB / 5.2 TiB avail
8 b6 O+ b3 @/ v) `  C, [: u" u    pgs:     1407 active+clean
9 e1 G! ]% p4 t; {) Q  w! G/ g6 _             1    active+clean+scrubbing+deep( y! C, q2 w1 _. W; [9 j
. ^8 ?% M8 G% [" N# q
  io:
* i) z6 b2 P7 n. g* S7 @    client:   74 KiB/s rd, 193 KiB/s wr, 10 op/s rd, 18 op/s wr
, S: ?! F" [" i8 R$ Y, U
; U) L5 D: ^1 Y" Q' f' r% H[root@compute01 ~]# ceph -s
6 ?. A3 r! u8 L, T) B5 u8 U0 X  cluster:$ b7 Y9 n4 L' B- O- x' P2 b
    id:     a6d0359e-8295-4e62-8620-54273462e79f
# K# a6 B  \6 I# c( n' C    health: HEALTH_OK
1 ?8 C2 [5 S, b6 W6 s0 }, m
; Y& y! S- I; ?" J! u  services:
; K6 i( \# u4 d4 T1 a4 U  k    mon: 3 daemons, quorum compute01,compute02,compute03 (age 3m)* ~$ B# W0 K; w4 @  e% o0 M  S
    mgr: compute02(active, since 4d), standbys: compute03, compute01* F1 z! }( v0 S) S( t4 V# A
    osd: 6 osds: 6 up (since 30m), 6 in (since 2w)3 ?9 a' I' K* e0 K. F
/ {# ^1 ?. Q( Y# N! r" ^; o
  data:0 K, ?0 C! g5 i
    pools:   5 pools, 1408 pgs7 `* t% O: R5 b8 |! h' M
    objects: 32.65k objects, 132 GiB1 o1 C1 r1 e$ `- m
    usage:   384 GiB used, 4.9 TiB / 5.2 TiB avail, y2 \9 P2 x* p8 O% j5 k4 S
    pgs:     1408 active+clean
$ `+ Q  J4 D. S2 |) o& X
/ u1 q) q" B( h* ]- p  io:/ A5 q( ^, k9 ]; x( F: I
    client:   840 KiB/s rd, 1.4 MiB/s wr, 63 op/s rd, 43 op/s wr
' M& O( _' S# X" L3 S- n/ M% q6 M" @6 f( |7 W% c5 q
" b; u6 D" y: g) O% r2 M) l
" `8 T, w$ w# K% P! q) ?8 C8 x% [
* F6 P$ x2 r+ u5 W% @5 L" d
总结,此过程中出现的问题,一是时间同步服务器有问题,二是ceph 设置了不剔除osd,不同步,不分配问题。
4 p& r" H1 X0 A, {+ Q% W' E
6 x4 Z" U* c( T4 a/ Y/ {
! ?; A7 ], k* \6 y' p$ P
 楼主| 发表于 2021-12-20 11:43:30 | 显示全部楼层
ceph存储同步数据需要时间同步,如果时间不同步,需要调整时间同步服务器,检查为什么不能同步时间。   以前chrony服务配置的时候,不用设置allow ,现在可能需要设置。或者以前也需要设置,只是没有注意这个细节性的地方。
. F: c+ {# |, C. z) M/ a0 F9 i; a同步时间服务器,第一修改配置,第二检查配置allow 允许的范围,第三其他节点同步时间。第四启动服务。" r( F7 U7 A( [/ ~$ v2 r
您需要登录后才可以回帖 登录 | 开始注册

本版积分规则

关闭

站长推荐上一条 /4 下一条

如有购买积分卡请联系497906712

QQ|返回首页|Archiver|手机版|小黑屋|易陆发现 点击这里给我发消息

GMT+8, 2022-1-22 14:07 , Processed in 0.075884 second(s), 21 queries .

Powered by LR.LINUX.cloud bbs168x X3.2 Licensed

© 2012-2022 Comsenz Inc.

快速回复 返回顶部 返回列表