How to Clean Up CouchDB database

Image source: https://couchdb.apache.org

Intro

CouchDB is very special database: it keeps all the deleted records, it keeps versioning off the records, and it keeps as well some of the data used once but deleted or changed in a particular view or in a particular database view design document. And normally there is no easy or let say save way, to delete the records in the database (till the current stable version we use in production). Unless you delete fully the Database and re-create it again from scratch. And I was amazed that this is actually the suggested solution…

But there are some things to be done to free up a lot of space in CouchDB. It is called database compaction, plus view clean up operations. So in this post I will share some info on how to free up some space, in some cases a lot of disk space. Depends on the Database size and scale you can end up freeing GBs or even TBs.

API calls to compact and clean up

Ok, in short this can be achieved by using following API calls in combination (one after another) executed for each database and for each design document into that database:

Compact the entire database:
https://{database_url}/{database}/_compact
- this API call will trigger compaction of the entire database
- more info here: https://docs.couchdb.org/en/2.2.0/api/database/compact.html#

Compact a design document part of a database: https://{database_url}/{database}/_compact/{design_document}
- this API call will trigger compaction of a design document from the database
- more info here: https://docs.couchdb.org/en/2.2.0/api/database/compact.html#db-compact-design-doc

All Views clean up for the entire database:
https://{database_url}/{database}/_view_cleanup
- this API call will trigger view clean up of all the views in the database
- more info here:
https://docs.couchdb.org/en/2.2.0/api/database/compact.html#db-view-cleanup

According to the documentation:

Compaction: compresses the disk database file by performing the following operations:
- Writes a new, optimised, version of the database file, removing any unused sections from the new version during write. Because a new file is temporarily created for this purpose, you may require up to twice the current storage space of the specified database in order for the compaction routine to complete.
- Removes old revisions of documents from the database, up to the per-database limit specified by the _revs_limit database parameter.

View Clean up:
- removes view index files that are no longer required by CouchDB as a result of changed views within design documents. As the view filename is based on a hash of the view functions, over time old views will remain, consuming storage. This call cleans up the cached view output on disk for a given view.

API Calls with CURL

Those API calls wrapped with curl for a specific database will look like something like this:

curl -H “Content-Type: application/json” -X POST “https://{database_url}/{database}/_compact" -u {username}:{password}

curl -H “Content-Type: application/json” -X POST “https://{database_url}/{database}/_compact/{design_document}" -u {username}:{password}

curl -H “Content-Type: application/json” -X POST “https://{database_url}/{database}/_view_cleanup" -u {username}:{password}

Again to repeat the fact that you will have to multiply and execute the calls above for each database and for each design document…