Adventure: Building with NATS Jetstream KV Store -Part 4
Welcome back! Here we are again! Going on my adventure series on the NATS JetStream Key/Value Store! Welcome to Part 4 of our series exploring the NATS JetStream KV Store! Where did we leave off? We left off exploring various commands such create, get, update and put; as well as how check a keys history and revert the keys value to an older revision if we like! Let's find out how to delete! Deleting a Key Ok this one can be a little confusing because there's a few things going on here. Let's just try it though. Let's start from scratch again, so delete all your Buckets. Let's create a new Bucket with history=3 and use put to add some values... fill it up! $ nats kv add --history=3 Bucket1 Information for Key-Value Store Bucket Bucket1 created 2025-02-07T13:50:55-05:00 Configuration: Bucket Name: Bucket1 History Kept: 3 Values Stored: 0 Compressed: false Backing Store Kind: JetStream Bucket Size: 0 B Maximum Bucket Size: unlimited Maximum Value Size: unlimited Maximum Age: unlimited JetStream Stream: KV_Bucket1 Storage: File Cluster Information: Name: Leader: NAHBJSFROLS2WSIXPPDBIUZXWJX6XIAKK36PTRF72ONLECU22XPPNV23 $ nats kv put Bucket1 Key1 Value1 Value1 $ nats kv put Bucket1 Key1 Value2 Value2 $ nats kv put Bucket1 Key1 Value3 Value3 Now let's delete! We do this with the del command. $ nats kv del Bucket1 Key1 ? Delete key Bucket1 > Key1? Yes Ok what now? Let's try to get the Key. $ nats kv get Bucket1 Key1 nats: error: nats: key not found Ok cool. Let's try make sure it's gone. $ nats kv ls Bucket1 No keys found in bucket That looks good. Ok what else can we do to confirm? Let's try it all. $ nats kv ls ╭────────────────────────────────────────────────────────────────────────────╮ │ Key-Value Buckets │ ├─────────┬─────────────┬─────────────────────┬───────┬────────┬─────────────┤ │ Bucket │ Description │ Created │ Size │ Values │ Last Update │ ├─────────┼─────────────┼─────────────────────┼───────┼────────┼─────────────┤ │ Bucket1 │ │ 2025-02-07 13:50:55 │ 185 B │ 3 │ 1m17s │ ╰─────────┴─────────────┴─────────────────────┴───────┴────────┴─────────────╯ Wait. 3 values??? What's going on? $ nats kv history Bucket1 Key1 ╭──────────────────────────────────────────────────────────────────╮ │ History for Bucket1 > Key1 │ ├──────┬──────────┬────────┬─────────────────────┬────────┬────────┤ │ Key │ Revision │ Op │ Created │ Length │ Value │ ├──────┼──────────┼────────┼─────────────────────┼────────┼────────┤ │ Key1 │ 2 │ PUT │ 2025-02-07 13:51:09 │ 6 │ Value2 │ │ Key1 │ 3 │ PUT │ 2025-02-07 13:51:10 │ 6 │ Value3 │ │ Key1 │ 4 │ DELETE │ 2025-02-07 13:53:52 │ 0 │ │ ╰──────┴──────────┴────────┴─────────────────────┴────────┴────────╯ Ahh there it is! A DELETE Op. So we did in fact delete the key. This makes sense because we couldn't get the keys value. If you see here it also has a Length of 0 and no Value. Ok so it's still there in the history as we see under Revision 4. So NATS is keeping track of this even though it's deleted. That's kind of neat and very purposeful. All events are tracked. We have 3 entries because history was set to 3. If it we're default, we would have 1 entry with a DELETE op. You get it. It's a delete marker added to our key. Well there's still history there. What if I don't want it taking up space? We can use kv compact. Compact Let's again, check our commands. Subcommands: kv add Adds a new KV Store Bucket kv put Puts a value into a key kv get Gets a value for a key kv create Puts a value into a key only if the key is new or it's last operation was a delete kv update Updates a key with a new value if the previous value matches the given revision kv del Deletes a key or the entire bucket kv purge Deletes a key from the bucket, clearing history before creating a delete marker kv history Shows the full history for a key kv revert Reverts a value to a previous revision using put kv info View the status of a KV store kv watch Watch the bucket or a specific key for updated kv ls List available buckets or the keys in a bucket kv compact Reclaim space used by deleted keys Ok, so compact reclaims space used by deleted keys. What does this mean? $ nats kv compact Bucket1 ? Purge all historic values and audit trails for deleted keys in bucket Bucket1? Yes $ nats kv history Bucket1 Key1 ╭─────────────────────────────────────────────────────────────────╮ │ History for Bucket1 > Key1 │ ├──────┬──────────┬────────┬─────────────────────┬────────┬───────┤
Welcome back!
Here we are again! Going on my adventure series on the NATS JetStream Key/Value Store!
Welcome to Part 4 of our series exploring the NATS JetStream KV Store!
Where did we leave off?
We left off exploring various commands such create, get, update and put; as well as how check a keys history and revert the keys value to an older revision if we like!
Let's find out how to delete!
Deleting a Key
Ok this one can be a little confusing because there's a few things going on here. Let's just try it though.
Let's start from scratch again, so delete all your Buckets.
Let's create a new Bucket with history=3 and use put to add some values... fill it up!
$ nats kv add --history=3 Bucket1
Information for Key-Value Store Bucket Bucket1 created 2025-02-07T13:50:55-05:00
Configuration:
Bucket Name: Bucket1
History Kept: 3
Values Stored: 0
Compressed: false
Backing Store Kind: JetStream
Bucket Size: 0 B
Maximum Bucket Size: unlimited
Maximum Value Size: unlimited
Maximum Age: unlimited
JetStream Stream: KV_Bucket1
Storage: File
Cluster Information:
Name:
Leader: NAHBJSFROLS2WSIXPPDBIUZXWJX6XIAKK36PTRF72ONLECU22XPPNV23
$ nats kv put Bucket1 Key1 Value1
Value1
$ nats kv put Bucket1 Key1 Value2
Value2
$ nats kv put Bucket1 Key1 Value3
Value3
Now let's delete! We do this with the del command.
$ nats kv del Bucket1 Key1
? Delete key Bucket1 > Key1? Yes
Ok what now? Let's try to get the Key.
$ nats kv get Bucket1 Key1
nats: error: nats: key not found
Ok cool. Let's try make sure it's gone.
$ nats kv ls Bucket1
No keys found in bucket
That looks good. Ok what else can we do to confirm? Let's try it all.
$ nats kv ls
╭────────────────────────────────────────────────────────────────────────────╮
│ Key-Value Buckets │
├─────────┬─────────────┬─────────────────────┬───────┬────────┬─────────────┤
│ Bucket │ Description │ Created │ Size │ Values │ Last Update │
├─────────┼─────────────┼─────────────────────┼───────┼────────┼─────────────┤
│ Bucket1 │ │ 2025-02-07 13:50:55 │ 185 B │ 3 │ 1m17s │
╰─────────┴─────────────┴─────────────────────┴───────┴────────┴─────────────╯
Wait. 3 values??? What's going on?
$ nats kv history Bucket1 Key1
╭──────────────────────────────────────────────────────────────────╮
│ History for Bucket1 > Key1 │
├──────┬──────────┬────────┬─────────────────────┬────────┬────────┤
│ Key │ Revision │ Op │ Created │ Length │ Value │
├──────┼──────────┼────────┼─────────────────────┼────────┼────────┤
│ Key1 │ 2 │ PUT │ 2025-02-07 13:51:09 │ 6 │ Value2 │
│ Key1 │ 3 │ PUT │ 2025-02-07 13:51:10 │ 6 │ Value3 │
│ Key1 │ 4 │ DELETE │ 2025-02-07 13:53:52 │ 0 │ │
╰──────┴──────────┴────────┴─────────────────────┴────────┴────────╯
Ahh there it is! A DELETE Op. So we did in fact delete the key.
This makes sense because we couldn't get the keys value. If you see here it also has a Length of 0 and no Value.
Ok so it's still there in the history as we see under Revision 4.
So NATS is keeping track of this even though it's deleted. That's kind of neat and very purposeful. All events are tracked. We have 3 entries because history was set to 3. If it we're default, we would have 1 entry with a DELETE op. You get it. It's a delete marker added to our key.
Well there's still history there. What if I don't want it taking up space?
We can use kv compact.
Compact
Let's again, check our commands.
Subcommands:
kv add Adds a new KV Store Bucket
kv put Puts a value into a key
kv get Gets a value for a key
kv create Puts a value into a key only if the key is new or it's last operation was a delete
kv update Updates a key with a new value if the previous value matches the given revision
kv del Deletes a key or the entire bucket
kv purge Deletes a key from the bucket, clearing history before creating a delete marker
kv history Shows the full history for a key
kv revert Reverts a value to a previous revision using put
kv info View the status of a KV store
kv watch Watch the bucket or a specific key for updated
kv ls List available buckets or the keys in a bucket
kv compact Reclaim space used by deleted keys
Ok, so compact reclaims space used by deleted keys. What does this mean?
$ nats kv compact Bucket1
? Purge all historic values and audit trails for deleted keys in bucket Bucket1? Yes
$ nats kv history Bucket1 Key1
╭─────────────────────────────────────────────────────────────────╮
│ History for Bucket1 > Key1 │
├──────┬──────────┬────────┬─────────────────────┬────────┬───────┤
│ Key │ Revision │ Op │ Created │ Length │ Value │
├──────┼──────────┼────────┼─────────────────────┼────────┼───────┤
│ Key1 │ 4 │ DELETE │ 2025-02-07 13:53:52 │ 0 │ │
╰──────┴──────────┴────────┴─────────────────────┴────────┴───────╯
Check it out! All the history is gone but we can still see that the key was marked with a DELETE op. We can also see that it happened on Revision 4 on our key.
So essentially compact will remove all the previous revisions that aren't needed anymore for every key marked with a DELETE op.
There's another command that can do this for us on a key by key basis. Let's check it out, it's called purge
Purging Keys
Purge can make this easier by deleting and removing the history in one step for a specified key. You can check out the commands above, as we've printed them out enough now!
Let's try it, but first, what would happen now if we modified Key1 again?
$ nats kv put Bucket1 Key1 Value1
Value1
$ nats kv history Bucket1 Key1
╭──────────────────────────────────────────────────────────────────╮
│ History for Bucket1 > Key1 │
├──────┬──────────┬────────┬─────────────────────┬────────┬────────┤
│ Key │ Revision │ Op │ Created │ Length │ Value │
├──────┼──────────┼────────┼─────────────────────┼────────┼────────┤
│ Key1 │ 4 │ DELETE │ 2025-02-07 13:53:52 │ 0 │ │
│ Key1 │ 5 │ PUT │ 2025-02-07 14:04:24 │ 6 │ Value1 │
╰──────┴──────────┴────────┴─────────────────────┴────────┴────────╯
Ohh bully! We appended a new revision to our history which is now 2! We kept track that the recent op was a PUT and we have a value again! We can get it and everything! Try it if you want!
So as you can see, you can use del to remove a key while preserving its history.
If needed, you can compact the Bucket to permanently remove all history of deleted keys.
Let's try purge
$ nats kv purge Bucket1 Key1
? Purge key Bucket1 > Key1? Yes
$ nats kv history Bucket1 Key1
╭────────────────────────────────────────────────────────────────╮
│ History for Bucket1 > Key1 │
├──────┬──────────┬───────┬─────────────────────┬────────┬───────┤
│ Key │ Revision │ Op │ Created │ Length │ Value │
├──────┼──────────┼───────┼─────────────────────┼────────┼───────┤
│ Key1 │ 6 │ PURGE │ 2025-02-07 14:07:37 │ 0 │ │
╰──────┴──────────┴───────┴─────────────────────┴────────┴───────╯
Ahh! It added a PURGE op. This is just to show that it wasn't simply deleted, the history was also deleted, all in one command! How convenient!
Let's go over one more thing.
Bucket Retention Policy
Yes, there is a tiny bit more to learn. Retention Policy is also a thing.
If the bucket reaches its configured storage limit (e.g., --max-bytes 10MB), older revisions, including delete markers, may be removed to free up space.
There is also TTL on Buckets when you create them. Let's mess with that real quick.
Go ahead and delete all the buckets again and just start from scratch.
Let's use the TTL flag.
$ nats kv add --ttl=30s --history=3 Bucket1
Information for Key-Value Store Bucket Bucket1 created 2025-02-07T14:11:41-05:00
Configuration:
Bucket Name: Bucket1
History Kept: 3
Values Stored: 0
Compressed: false
Backing Store Kind: JetStream
Bucket Size: 0 B
Maximum Bucket Size: unlimited
Maximum Value Size: unlimited
Maximum Age: 30.00s
JetStream Stream: KV_Bucket1
Storage: File
Cluster Information:
Name:
Leader: NAHBJSFROLS2WSIXPPDBIUZXWJX6XIAKK36PTRF72ONLECU22XPPNV23
Now create some keys.
$ nats kv put Bucket1 Key1 Value1
Value1
$ nats kv put Bucket1 Key1 Value2
Value2
$ nats kv put Bucket1 Key1 Value3
Value3
$ nats kv history Bucket1 Key1
╭───────────────────────────────────────────────────────────────╮
│ History for Bucket1 > Key1 │
├──────┬──────────┬─────┬─────────────────────┬────────┬────────┤
│ Key │ Revision │ Op │ Created │ Length │ Value │
├──────┼──────────┼─────┼─────────────────────┼────────┼────────┤
│ Key1 │ 1 │ PUT │ 2025-02-07 14:12:23 │ 6 │ Value1 │
│ Key1 │ 2 │ PUT │ 2025-02-07 14:12:24 │ 6 │ Value2 │
│ Key1 │ 3 │ PUT │ 2025-02-07 14:12:25 │ 6 │ Value3 │
╰──────┴──────────┴─────┴─────────────────────┴────────┴────────╯
Let's wait for 30 seconds.
...Now try!
$ nats kv history Bucket1 Key1
nats: error: nats: key not found
Ok wow. So it's just gone huh?
Yeah we set the expiration time on the Key. We can't even check it's history!
Ok wow. That was a doozy. There's a lot there and stuff we haven't covered but remember; this is an intro series!! (go easy on me!)
I highly recommend you play around with it yourself if anything didn't seem clear. It's easy enough and by now you should have the chops for it!
Alright we went over fundamental CLI commands for NATS Jetstream KV Store. I know, I know --you want to make something.
Ok I get it. Let's make something in Part 5!
