Forum

You need to log in to create posts and topics.

CrateDB ClusterBlockException: blocked by: [FORBIDDEN/12/index read-only / allow delete (api)]

This was so annoying I had to document it and its solution.

This error happens when your CrateDB is low on disk space, and it will automatically switch elasticsearch indices to read-only.  The threshold is defined in crate/config/crate.yml:

# Defines the threshold on which CrateDB enforces a read-only block on every

# index that has at least one shard allocated on a node with at least one disk

# exceeding the flood stage.

cluster.routing.allocation.disk.watermark.flood_stage = 90%

 

The problem is that even if you freed up space, it doesn't automatically switch it back.  So after you free up disk space, you have to manually change it yourself.  Fortunately the directions are here: https://www.codingbot.net/2019/01/elasticsearch-cratedb-logstash-clusterblockexception-forbidden-index-read-only.html

In particular, the ALTER TABLE ... SET ("blocks.read_only_allow_delete" = FALSE); worked, like this:

ALTER TABLE volttron.data SET ("blocks.read_only_allow_delete" = FALSE);  

// repeat for all your volttron crate db tables.