Misplaced Objejcts Help
Last week, we had a mishap on our DEV server, where we fully ran out of disk space.
I had gone ahead and attached an extra OSD on one of my nodes.
Ceph started recovering, but seems that it's quite stuck with misplaced objects.
This is my ceph status:
bash-5.1$ ceph status
cluster:
id: eb1668db-a628-4df9-8c83-583a25a2005e
health: HEALTH_OK
services:
mon: 3 daemons, quorum c,d,e (age 3d)
mgr: b(active, since 3w), standbys: a
mds: 1/1 daemons up, 1 hot standby
osd: 4 osds: 4 up (since 3d), 4 in (since 3d); 95 remapped pgs
rgw: 1 daemon active (1 hosts, 1 zones)
data:
volumes: 1/1 healthy
pools: 12 pools, 233 pgs
objects: 560.41k objects, 1.3 TiB
usage: 2.1 TiB used, 1.8 TiB / 3.9 TiB avail
pgs: 280344/1616532 objects misplaced (17.342%)
139 active+clean
94 active+clean+remapped
io:
client: 3.2 KiB/s rd, 4.9 MiB/s wr, 4 op/s rd, 209 op/s wr
The 94 Active + clean + remapped has been like this for 3 days.
The objects misplaced is increasing,.
Placement Groups (PGs)
- Previous Snapshot:
- Misplaced Objects: 270,300/1,560,704 (17.319%).
- PG States:
active+clean
: 139.active+clean+remapped
: 94.
- Current Snapshot:
- Misplaced Objects: 280,344/1,616,532 (17.342%).
- PG States:
active+clean
: 139.active+clean+remapped
: 94.
- Change:
- Misplaced objects increased by 10,044.
- The ratio of misplaced objects increased slightly from 17.319% to 17.342%.
- No changes in PG states.
My previous snapshot was on Friday midday...
Current Snapshot is now Saturday evening.
How can i rectify this?