golang/golang-github-twin-gocache
1
0
Fork 0
Code Issues Activity

Adding upstream version 2.2.2.

Browse source 
Signed-off-by: Daniel Baumann <daniel@debian.org>
This commit is contained in:
Daniel Baumann 2025-05-16 22:45:16 +02:00
parent a6909e3829
commit 02ef45af86
Signed by: daniel
GPG key ID: FBB4F0E80A80222F
19 changed files with 3024 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

1
.gitattributes vendored Normal file
View file

@ -0,0 +1 @@
* text=lf

7
.github/codecov.yml vendored Normal file
View file

@ -0,0 +1,7 @@
coverage:
status:
patch: off
project:
default:
target: 75%
threshold: null

7
.github/dependabot.yml vendored Normal file
View file

@ -0,0 +1,7 @@
version: 2
updates:
- package-ecosystem: "github-actions"
directory: "/"
labels: ["dependencies"]
schedule:
interval: "daily"

25
.github/workflows/test.yml vendored Normal file
View file

@ -0,0 +1,25 @@
name: test
on:
pull_request:
paths-ignore:
- '*.md'
push:
branches:
- master
paths-ignore:
- '*.md'
jobs:
test:
name: test
runs-on: ubuntu-latest
timeout-minutes: 3
steps:
- uses: actions/setup-go@v5
with:
go-version: 1.21
- uses: actions/checkout@v4
- run: go test ./... -race -coverprofile=coverage.txt -covermode=atomic
- name: Codecov
uses: codecov/codecov-action@v4.1.0
with:
files: ./coverage.txt

1
.gitignore vendored Normal file
View file

@ -0,0 +1 @@
.idea

9
LICENSE Normal file
View file

@ -0,0 +1,9 @@
MIT License
Copyright (c) 2023 TwiN
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

361
README.md Normal file
View file

@ -0,0 +1,361 @@
# gocache
![test](https://github.com/TwiN/gocache/workflows/test/badge.svg?branch=master)
[![Go Report Card](https://goreportcard.com/badge/github.com/TwiN/gocache)](https://goreportcard.com/report/github.com/TwiN/gocache)
[![codecov](https://codecov.io/gh/TwiN/gocache/branch/master/graph/badge.svg)](https://codecov.io/gh/TwiN/gocache)
[![Go version](https://img.shields.io/github/go-mod/go-version/TwiN/gocache.svg)](https://github.com/TwiN/gocache)
[![Go Reference](https://pkg.go.dev/badge/github.com/TwiN/gocache.svg)](https://pkg.go.dev/github.com/TwiN/gocache/v2)
[![Follow TwiN](https://img.shields.io/github/followers/TwiN?label=Follow&style=social)](https://github.com/TwiN)
gocache is an easy-to-use, high-performance, lightweight and thread-safe (goroutine-safe) in-memory key-value cache
with support for LRU and FIFO eviction policies as well as expiration, bulk operations and even retrieval of keys by pattern.
## Table of Contents
- [Features](#features)
- [Usage](#usage)
- [Initializing the cache](#initializing-the-cache)
- [Functions](#functions)
- [Examples](#examples)
- [Creating or updating an entry](#creating-or-updating-an-entry)
- [Getting an entry](#getting-an-entry)
- [Deleting an entry](#deleting-an-entry)
- [Complex example](#complex-example)
- [Persistence](#persistence)
- [Eviction](#eviction)
- [MaxSize](#maxsize)
- [MaxMemoryUsage](#maxmemoryusage)
- [Expiration](#expiration)
- [Performance](#performance)
- [Summary](#summary)
- [Results](#results)
- [FAQ](#faq)
- [How can I persist the data on application termination?](#how-can-i-persist-the-data-on-application-termination)
## Features
gocache supports the following cache eviction policies:
- First in first out (FIFO)
- Least recently used (LRU)
It also supports cache entry TTL, which is both active and passive. Active expiration means that if you attempt
to retrieve a cache key that has already expired, it will delete it on the spot and the behavior will be as if
the cache key didn't exist. As for passive expiration, there's a background task that will take care of deleting
expired keys.
It also includes what you'd expect from a cache, like GET/SET, bulk operations and get by pattern.
## Usage
```
go get -u github.com/TwiN/gocache/v2
```
### Initializing the cache
```go
cache := gocache.NewCache().WithMaxSize(1000).WithEvictionPolicy(gocache.LeastRecentlyUsed)
```
If you're planning on using expiration (`SetWithTTL` or `Expire`) and you want expired entries to be automatically deleted
in the background, make sure to start the janitor when you instantiate the cache:
```go
cache.StartJanitor()
```
### Functions
| Function | Description |
|-----------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| WithMaxSize | Sets the max size of the cache. `gocache.NoMaxSize` means there is no limit. If not set, the default max size is `gocache.DefaultMaxSize`. |
| WithMaxMemoryUsage | Sets the max memory usage of the cache. `gocache.NoMaxMemoryUsage` means there is no limit. The default behavior is to not evict based on memory usage. |
| WithEvictionPolicy | Sets the eviction algorithm to be used when the cache reaches the max size. If not set, the default eviction policy is `gocache.FirstInFirstOut` (FIFO). |
| WithDefaultTTL | Sets the default TTL for each entry. |
| WithForceNilInterfaceOnNilPointer | Configures whether values with a nil pointer passed to write functions should be forcefully set to nil. Defaults to true. |
| StartJanitor | Starts the janitor, which is in charge of deleting expired cache entries in the background. |
| StopJanitor | Stops the janitor. |
| Set | Same as `SetWithTTL`, but using the default TTL (which is `gocache.NoExpiration`, unless configured otherwise). |
| SetWithTTL | Creates or updates a cache entry with the given key, value and expiration time. If the max size after the aforementioned operation is above the configured max size, the tail will be evicted. Depending on the eviction policy, the tail is defined as the oldest |
| SetAll | Same as `Set`, but in bulk. |
| SetAllWithTTL | Same as `SetWithTTL`, but in bulk. |
| Get | Gets a cache entry by its key. |
| GetByKeys | Gets a map of entries by their keys. The resulting map will contain all keys, even if some of the keys in the slice passed as parameter were not present in the cache. |
| GetAll | Gets all cache entries. |
| GetKeysByPattern | Retrieves a slice of keys that matches a given pattern. |
| Delete | Removes a key from the cache. |
| DeleteAll | Removes multiple keys from the cache. |
| DeleteKeysByPattern | Removes all keys that that matches a given pattern. |
| Count | Gets the size of the cache. This includes cache keys which may have already expired, but have not been removed yet. |
| Clear | Wipes the cache. |
| TTL | Gets the time until a cache key expires. |
| Expire | Sets the expiration time of an existing cache key. |
For further documentation, please refer to [Go Reference](https://pkg.go.dev/github.com/TwiN/gocache)
### Examples
#### Creating or updating an entry
```go
cache.Set("key", "value")
cache.Set("key", 1)
cache.Set("key", struct{ Text string }{Test: "value"})
cache.SetWithTTL("key", []byte("value"), 24*time.Hour)
```
#### Getting an entry
```go
value, exists := cache.Get("key")
```
You can also get multiple entries by using `cache.GetByKeys([]string{"key1", "key2"})`
#### Deleting an entry
```go
cache.Delete("key")
```
You can also delete multiple entries by using `cache.DeleteAll([]string{"key1", "key2"})`
#### Complex example
```go
package main
import (
"fmt"
"time"
"github.com/TwiN/gocache/v2"
)
func main() {
cache := gocache.NewCache().WithEvictionPolicy(gocache.LeastRecentlyUsed).WithMaxSize(10000)
cache.StartJanitor() // Passively manages expired entries
defer cache.StopJanitor()
cache.Set("key", "value")
cache.SetWithTTL("key-with-ttl", "value", 60*time.Minute)
cache.SetAll(map[string]any{"k1": "v1", "k2": "v2", "k3": "v3"})
fmt.Println("[Count] Cache size:", cache.Count())
value, exists := cache.Get("key")
fmt.Printf("[Get] key=key; value=%s; exists=%v\n", value, exists)
for key, value := range cache.GetByKeys([]string{"k1", "k2", "k3"}) {
fmt.Printf("[GetByKeys] key=%s; value=%s\n", key, value)
}
for _, key := range cache.GetKeysByPattern("key*", 0) {
fmt.Printf("[GetKeysByPattern] pattern=key*; key=%s\n", key)
}
cache.Expire("key", time.Hour)
time.Sleep(500*time.Millisecond)
timeUntilExpiration, _ := cache.TTL("key")
fmt.Println("[TTL] Number of minutes before 'key' expires:", int(timeUntilExpiration.Seconds()))
cache.Delete("key")
cache.DeleteAll([]string{"k1", "k2", "k3"})
cache.Clear()
fmt.Println("[Count] Cache size after clearing the cache:", cache.Count())
}
```
<details>
<summary>Output</summary>
```
[Count] Cache size: 5
[Get] key=key; value=value; exists=true
[GetByKeys] key=k1; value=v1
[GetByKeys] key=k2; value=v2
[GetByKeys] key=k3; value=v3
[GetKeysByPattern] pattern=key*; key=key-with-ttl
[GetKeysByPattern] pattern=key*; key=key
[TTL] Number of minutes before 'key' expires: 3599
[Count] Cache size after clearing the cache: 0
```
</details>
## Persistence
Prior to v2, gocache supported persistence out of the box.
After some thinking, I decided that persistence added too many dependencies, and given than this is a cache library
and most people wouldn't be interested in persistence, I decided to get rid of it.
That being said, you can use the `GetAll` and `SetAll` methods of `gocache.Cache` to implement persistence yourself.
## Eviction
### MaxSize
Eviction by MaxSize is the default behavior, and is also the most efficient.
The code below will create a cache that has a maximum size of 1000:
```go
cache := gocache.NewCache().WithMaxSize(1000)
```
This means that whenever an operation causes the total size of the cache to go above 1000, the tail will be evicted.
### MaxMemoryUsage
Eviction by MaxMemoryUsage is **disabled by default**, and is in alpha.
The code below will create a cache that has a maximum memory usage of 50MB:
```go
cache := gocache.NewCache().WithMaxSize(0).WithMaxMemoryUsage(50*gocache.Megabyte)
```
This means that whenever an operation causes the total memory usage of the cache to go above 50MB, one or more tails
will be evicted.
Unlike evictions caused by reaching the MaxSize, evictions triggered by MaxMemoryUsage may lead to multiple entries
being evicted in a row. The reason for this is that if, for instance, you had 100 entries of 0.1MB each and you suddenly added
a single entry of 10MB, 100 entries would need to be evicted to make enough space for that new big entry.
It's very important to keep in mind that eviction by MaxMemoryUsage is approximate.
**The only memory taken into consideration is the size of the cache, not the size of the entire application.**
If you pass along 100MB worth of data in a matter of seconds, even though the cache's memory usage will remain
under 50MB (or whatever you configure the MaxMemoryUsage to), the memory footprint generated by that 100MB will
still exist until the next GC cycle.
As previously mentioned, this is a work in progress, and here's a list of the things you should keep in mind:
- The memory usage of structs are a gross estimation and may not reflect the actual memory usage.
- Native types (string, int, bool, []byte, etc.) are the most accurate for calculating the memory usage.
- Adding an entry bigger than the configured MaxMemoryUsage will work, but it will evict all other entries.
## Expiration
There are two ways that the deletion of expired keys can take place:
- Active
- Passive
**Active deletion of expired keys** happens when an attempt is made to access the value of a cache entry that expired.
`Get`, `GetByKeys` and `GetAll` are the only functions that can trigger active deletion of expired keys.
**Passive deletion of expired keys** runs in the background and is managed by the janitor.
If you do not start the janitor, there will be no passive deletion of expired keys.
## Performance
### Summary
- **Set**: Both map and gocache have the same performance.
- **Get**: Map is faster than gocache.
This is because gocache keeps track of the head and the tail for eviction and expiration/TTL.
Ultimately, the difference is negligible.
We could add a way to disable eviction or disable expiration altogether just to match the map's performance,
but if you're looking into using a library like gocache, odds are, you want more than just a map.
### Results
| key | value |
|:-------|:---------|
| goos | windows |
| goarch | amd64 |
| cpu | i7-9700K |
| mem | 32G DDR4 |
```
// Normal map
BenchmarkMap_Get-8 49944228 24.2 ns/op 7 B/op 0 allocs/op
BenchmarkMap_Set/small_value-8 3939964 394.1 ns/op 188 B/op 2 allocs/op
BenchmarkMap_Set/medium_value-8 3868586 395.5 ns/op 191 B/op 2 allocs/op
BenchmarkMap_Set/large_value-8 3992138 385.3 ns/op 186 B/op 2 allocs/op
// Gocache
BenchmarkCache_Get/FirstInFirstOut-8 27907950 44.3 ns/op 7 B/op 0 allocs/op
BenchmarkCache_Get/LeastRecentlyUsed-8 28211396 44.2 ns/op 7 B/op 0 allocs/op
BenchmarkCache_Set/FirstInFirstOut_small_value-8 3139538 373.5 ns/op 185 B/op 3 allocs/op
BenchmarkCache_Set/FirstInFirstOut_medium_value-8 3099516 378.6 ns/op 186 B/op 3 allocs/op
BenchmarkCache_Set/FirstInFirstOut_large_value-8 3086776 386.7 ns/op 186 B/op 3 allocs/op
BenchmarkCache_Set/LeastRecentlyUsed_small_value-8 3070555 379.0 ns/op 187 B/op 3 allocs/op
BenchmarkCache_Set/LeastRecentlyUsed_medium_value-8 3056928 383.8 ns/op 187 B/op 3 allocs/op
BenchmarkCache_Set/LeastRecentlyUsed_large_value-8 3108250 383.8 ns/op 186 B/op 3 allocs/op
BenchmarkCache_SetUsingMaxMemoryUsage/medium_value-8 2773315 449.0 ns/op 210 B/op 4 allocs/op
BenchmarkCache_SetUsingMaxMemoryUsage/large_value-8 2731818 440.0 ns/op 211 B/op 4 allocs/op
BenchmarkCache_SetUsingMaxMemoryUsage/small_value-8 2659296 446.8 ns/op 213 B/op 4 allocs/op
BenchmarkCache_SetWithMaxSize/100_small_value-8 4848658 248.8 ns/op 114 B/op 3 allocs/op
BenchmarkCache_SetWithMaxSize/10000_small_value-8 4117632 293.7 ns/op 106 B/op 3 allocs/op
BenchmarkCache_SetWithMaxSize/100000_small_value-8 3867402 313.0 ns/op 110 B/op 3 allocs/op
BenchmarkCache_SetWithMaxSize/100_medium_value-8 4750057 250.1 ns/op 113 B/op 3 allocs/op
BenchmarkCache_SetWithMaxSize/10000_medium_value-8 4143772 294.5 ns/op 106 B/op 3 allocs/op
BenchmarkCache_SetWithMaxSize/100000_medium_value-8 3768883 313.2 ns/op 111 B/op 3 allocs/op
BenchmarkCache_SetWithMaxSize/100_large_value-8 4822646 251.1 ns/op 114 B/op 3 allocs/op
BenchmarkCache_SetWithMaxSize/10000_large_value-8 4154428 291.6 ns/op 106 B/op 3 allocs/op
BenchmarkCache_SetWithMaxSize/100000_large_value-8 3897358 313.7 ns/op 110 B/op 3 allocs/op
BenchmarkCache_SetWithMaxSizeAndLRU/100_small_value-8 4784180 254.2 ns/op 114 B/op 3 allocs/op
BenchmarkCache_SetWithMaxSizeAndLRU/10000_small_value-8 4067042 292.0 ns/op 106 B/op 3 allocs/op
BenchmarkCache_SetWithMaxSizeAndLRU/100000_small_value-8 3832760 313.8 ns/op 111 B/op 3 allocs/op
BenchmarkCache_SetWithMaxSizeAndLRU/100_medium_value-8 4846706 252.2 ns/op 114 B/op 3 allocs/op
BenchmarkCache_SetWithMaxSizeAndLRU/10000_medium_value-8 4103817 292.5 ns/op 106 B/op 3 allocs/op
BenchmarkCache_SetWithMaxSizeAndLRU/100000_medium_value-8 3845623 315.1 ns/op 111 B/op 3 allocs/op
BenchmarkCache_SetWithMaxSizeAndLRU/100_large_value-8 4744513 257.9 ns/op 114 B/op 3 allocs/op
BenchmarkCache_SetWithMaxSizeAndLRU/10000_large_value-8 3956316 299.5 ns/op 106 B/op 3 allocs/op
BenchmarkCache_SetWithMaxSizeAndLRU/100000_large_value-8 3876843 351.3 ns/op 110 B/op 3 allocs/op
BenchmarkCache_GetSetMultipleConcurrent-8 750088 1566.0 ns/op 128 B/op 8 allocs/op
BenchmarkCache_GetSetConcurrentWithFrequentEviction/FirstInFirstOut-8 3836961 316.2 ns/op 80 B/op 1 allocs/op
BenchmarkCache_GetSetConcurrentWithFrequentEviction/LeastRecentlyUsed-8 3846165 315.6 ns/op 80 B/op 1 allocs/op
BenchmarkCache_GetConcurrently/FirstInFirstOut-8 4830342 239.8 ns/op 8 B/op 1 allocs/op
BenchmarkCache_GetConcurrently/LeastRecentlyUsed-8 4895587 243.2 ns/op 8 B/op 1 allocs/op
(Trimmed "BenchmarkCache_" for readability)
WithForceNilInterfaceOnNilPointer/true_with_nil_struct_pointer-8 6901461 178.5 ns/op 7 B/op 1 allocs/op
WithForceNilInterfaceOnNilPointer/true-8 6629566 180.7 ns/op 7 B/op 1 allocs/op
WithForceNilInterfaceOnNilPointer/false_with_nil_struct_pointer-8 6282798 170.1 ns/op 7 B/op 1 allocs/op
WithForceNilInterfaceOnNilPointer/false-8 6741382 172.6 ns/op 7 B/op 1 allocs/op
WithForceNilInterfaceOnNilPointerWithConcurrency/true_with_nil_struct_pointer-8 4432951 258.0 ns/op 8 B/op 1 allocs/op
WithForceNilInterfaceOnNilPointerWithConcurrency/true-8 4676943 244.4 ns/op 8 B/op 1 allocs/op
WithForceNilInterfaceOnNilPointerWithConcurrency/false_with_nil_struct_pointer-8 4818418 239.6 ns/op 8 B/op 1 allocs/op
WithForceNilInterfaceOnNilPointerWithConcurrency/false-8 5025937 238.2 ns/op 8 B/op 1 allocs/op
```
## FAQ
### How can I persist the data on application termination?
While creating your own auto save feature might come in handy, it may still lead to loss of data if the application
automatically saves every 10 minutes and your application crashes 9 minutes after the previous save.
To increase your odds of not losing any data, you can use Go's `signal` package, more specifically its `Notify` function
which allows listening for termination signals like SIGTERM and SIGINT. Once a termination signal is caught, you can
add the necessary logic for a graceful shutdown.
In the following example, the code that would usually be present in the `main` function is moved to a different function
named `Start` which is launched on a different goroutine so that listening for a termination signals is what blocks the
main goroutine instead:
```go
package main
import (
"log"
"os"
"os/signal"
"syscall"
"github.com/TwiN/gocache/v2"
)
var cache = gocache.NewCache()
func main() {
data := retrieveCacheEntriesUsingWhateverMeanYouUsedToPersistIt()
cache.SetAll(data)
// Start everything else on another goroutine to prevent blocking the main goroutine
go Start()
// Wait for termination signal
sig := make(chan os.Signal, 1)
done := make(chan bool, 1)
signal.Notify(sig, os.Interrupt, syscall.SIGTERM)
go func() {
<-sig
log.Println("Received termination signal, attempting to gracefully shut down")
// Persist the cache entries
cacheEntries := cache.GetAll()
persistCacheEntriesHoweverYouWant(cacheEntries)
// Tell the main goroutine that we're done
done <- true
}()
<-done
log.Println("Shutting down")
}
```
Note that this won't protect you from a SIGKILL, as this signal cannot be caught.

108
entry.go Normal file
View file

@ -0,0 +1,108 @@
package gocache
import (
"fmt"
"time"
"unsafe"
)
// Entry is a cache entry
type Entry struct {
// Key is the name of the cache entry
Key string
// Value is the value of the cache entry
Value any
// RelevantTimestamp is the variable used to store either:
// - creation timestamp, if the Cache's EvictionPolicy is FirstInFirstOut
// - last access timestamp, if the Cache's EvictionPolicy is LeastRecentlyUsed
//
// Note that updating an existing entry will also update this value
RelevantTimestamp time.Time
// Expiration is the unix time in nanoseconds at which the entry will expire (-1 means no expiration)
Expiration int64
next *Entry
previous *Entry
}
// Accessed updates the Entry's RelevantTimestamp to now
func (entry *Entry) Accessed() {
entry.RelevantTimestamp = time.Now()
}
// Expired returns whether the Entry has expired
func (entry Entry) Expired() bool {
if entry.Expiration > 0 {
if time.Now().UnixNano() > entry.Expiration {
return true
}
}
return false
}
// SizeInBytes returns the size of an entry in bytes, approximately.
func (entry *Entry) SizeInBytes() int {
return toBytes(entry.Key) + toBytes(entry.Value) + 32
}
func toBytes(value any) int {
switch value.(type) {
case string:
return int(unsafe.Sizeof(value)) + len(value.(string))
case int8, uint8, bool:
return int(unsafe.Sizeof(value)) + 1
case int16, uint16:
return int(unsafe.Sizeof(value)) + 2
case int32, uint32, float32, complex64:
return int(unsafe.Sizeof(value)) + 4
case int64, uint64, int, uint, float64, complex128:
return int(unsafe.Sizeof(value)) + 8
case []any:
size := 0
for _, v := range value.([]any) {
size += toBytes(v)
}
return int(unsafe.Sizeof(value)) + size
case []string:
size := 0
for _, v := range value.([]string) {
size += toBytes(v)
}
return int(unsafe.Sizeof(value)) + size
case []int8:
return int(unsafe.Sizeof(value)) + len(value.([]int8))
case []uint8:
return int(unsafe.Sizeof(value)) + len(value.([]uint8))
case []bool:
return int(unsafe.Sizeof(value)) + len(value.([]bool))
case []int16:
return int(unsafe.Sizeof(value)) + (len(value.([]int16)) * 2)
case []uint16:
return int(unsafe.Sizeof(value)) + (len(value.([]uint16)) * 2)
case []int32:
return int(unsafe.Sizeof(value)) + (len(value.([]int32)) * 4)
case []uint32:
return int(unsafe.Sizeof(value)) + (len(value.([]uint32)) * 4)
case []float32:
return int(unsafe.Sizeof(value)) + (len(value.([]float32)) * 4)
case []complex64:
return int(unsafe.Sizeof(value)) + (len(value.([]complex64)) * 4)
case []int64:
return int(unsafe.Sizeof(value)) + (len(value.([]int64)) * 8)
case []uint64:
return int(unsafe.Sizeof(value)) + (len(value.([]uint64)) * 8)
case []int:
return int(unsafe.Sizeof(value)) + (len(value.([]int)) * 8)
case []uint:
return int(unsafe.Sizeof(value)) + (len(value.([]uint)) * 8)
case []float64:
return int(unsafe.Sizeof(value)) + (len(value.([]float64)) * 8)
case []complex128:
return int(unsafe.Sizeof(value)) + (len(value.([]complex128)) * 8)
default:
return int(unsafe.Sizeof(value)) + len(fmt.Sprintf("%v", value))
}
}

74
entry_test.go Normal file
View file

@ -0,0 +1,74 @@
package gocache
import (
"fmt"
"testing"
)
func TestEntry_SizeInBytes(t *testing.T) {
testSizeInBytes(t, "key", 0, 75)
testSizeInBytes(t, "k", 0, 73)
testSizeInBytes(t, "k", "v", 66)
testSizeInBytes(t, "k", true, 66)
testSizeInBytes(t, "k", int8(1), 66)
testSizeInBytes(t, "k", uint8(1), 66)
testSizeInBytes(t, "k", true, 66)
testSizeInBytes(t, "k", int16(1), 67)
testSizeInBytes(t, "k", uint16(1), 67)
testSizeInBytes(t, "k", int32(1), 69)
testSizeInBytes(t, "k", uint32(1), 69)
testSizeInBytes(t, "k", float32(1), 69)
testSizeInBytes(t, "k", complex64(1), 69)
testSizeInBytes(t, "k", int64(1), 73)
testSizeInBytes(t, "k", uint64(1), 73)
testSizeInBytes(t, "k", 1, 73)
testSizeInBytes(t, "k", uint(1), 73)
testSizeInBytes(t, "k", float64(1), 73)
testSizeInBytes(t, "k", complex128(1), 73)
testSizeInBytes(t, "k", []string{}, 65)
testSizeInBytes(t, "k", []string{"what"}, 85)
testSizeInBytes(t, "k", []string{"what", "the"}, 104)
testSizeInBytes(t, "k", []int8{}, 65)
testSizeInBytes(t, "k", []int8{1}, 66)
testSizeInBytes(t, "k", []int8{1, 2}, 67)
testSizeInBytes(t, "k", []uint8{1}, 66)
testSizeInBytes(t, "k", []uint8{1, 2}, 67)
testSizeInBytes(t, "k", []bool{true}, 66)
testSizeInBytes(t, "k", []bool{true, false}, 67)
testSizeInBytes(t, "k", []int16{1}, 67)
testSizeInBytes(t, "k", []int16{1, 2}, 69)
testSizeInBytes(t, "k", []uint16{1}, 67)
testSizeInBytes(t, "k", []int32{1}, 69)
testSizeInBytes(t, "k", []int32{1, 2}, 73)
testSizeInBytes(t, "k", []uint32{1}, 69)
testSizeInBytes(t, "k", []uint32{1, 2}, 73)
testSizeInBytes(t, "k", []float32{1}, 69)
testSizeInBytes(t, "k", []float32{1, 2}, 73)
testSizeInBytes(t, "k", []complex64{1}, 69)
testSizeInBytes(t, "k", []complex64{1, 2}, 73)
testSizeInBytes(t, "k", []int64{1}, 73)
testSizeInBytes(t, "k", []int64{1, 2}, 81)
testSizeInBytes(t, "k", []uint64{1}, 73)
testSizeInBytes(t, "k", []uint64{1, 2}, 81)
testSizeInBytes(t, "k", []int{1}, 73)
testSizeInBytes(t, "k", []int{1, 2}, 81)
testSizeInBytes(t, "k", []uint{1}, 73)
testSizeInBytes(t, "k", []uint{1, 2}, 81)
testSizeInBytes(t, "k", []float64{1}, 73)
testSizeInBytes(t, "k", []float64{1, 2}, 81)
testSizeInBytes(t, "k", []complex128{1}, 73)
testSizeInBytes(t, "k", []complex128{1, 2}, 81)
testSizeInBytes(t, "k", struct{}{}, 67)
testSizeInBytes(t, "k", struct{ A string }{A: "hello"}, 72)
testSizeInBytes(t, "k", struct{ A, B string }{A: "hello", B: "world"}, 78)
testSizeInBytes(t, "k", nil, 70)
testSizeInBytes(t, "k", make([]any, 5), 170)
}
func testSizeInBytes(t *testing.T, key string, value any, expectedSize int) {
t.Run(fmt.Sprintf("%T_%d", value, expectedSize), func(t *testing.T) {
if size := (&Entry{Key: key, Value: value}).SizeInBytes(); size != expectedSize {
t.Errorf("expected size of entry with key '%v' and value '%v' (%T) to be %d, got %d", key, value, value, expectedSize, size)
}
})
}

3
go.mod Normal file
View file

@ -0,0 +1,3 @@
module github.com/TwiN/gocache/v2
go 1.21

599
gocache.go Normal file
View file

@ -0,0 +1,599 @@
package gocache
import (
"errors"
"reflect"
"sync"
"time"
)
var (
Debug = false
)
const (
// NoMaxSize means that the cache has no maximum number of entries in the cache
// Setting Cache.maxSize to this value also means there will be no eviction
NoMaxSize = 0
// NoMaxMemoryUsage means that the cache has no maximum number of entries in the cache
NoMaxMemoryUsage = 0
// DefaultMaxSize is the max size set if no max size is specified
DefaultMaxSize = 100000
// NoExpiration is the value that must be used as TTL to specify that the given key should never expire
NoExpiration = -1
Kilobyte = 1024
Megabyte = 1024 * Kilobyte
Gigabyte = 1024 * Megabyte
)
var (
ErrKeyDoesNotExist = errors.New("key does not exist") // Returned when a cache key does not exist
ErrKeyHasNoExpiration = errors.New("key has no expiration") // Returned when a cache key has no expiration
ErrJanitorAlreadyRunning = errors.New("janitor is already running") // Returned when the janitor has already been started
)
// Cache is the core struct of gocache which contains the data as well as all relevant configuration fields
//
// Do not instantiate this struct directly, use NewCache instead
type Cache struct {
// maxSize is the maximum amount of entries that can be in the cache at any given time
// By default, this is set to DefaultMaxSize
maxSize int
// maxMemoryUsage is the maximum amount of memory that can be taken up by the cache at any time
// By default, this is set to NoMaxMemoryUsage, meaning that the default behavior is to not evict
// based on maximum memory usage
maxMemoryUsage int
// evictionPolicy is the eviction policy
evictionPolicy EvictionPolicy
// defaultTTL is the default TTL for each entry
// Defaults to NoExpiration
defaultTTL time.Duration
// stats is the object that contains cache statistics/metrics
stats *Statistics
// entries is the content of the cache
entries map[string]*Entry
// mutex is the lock for making concurrent operations on the cache
mutex sync.RWMutex
// head is the cache entry at the head of the cache
head *Entry
// tail is the last cache node and also the next entry that will be evicted
tail *Entry
// stopJanitor is the channel used to stop the janitor
stopJanitor chan bool
// memoryUsage is the approximate memory usage of the cache (dataset only) in bytes
memoryUsage int
// forceNilInterfaceOnNilPointer determines whether all Set-like functions should set a value as nil if the
// interface passed has a nil value but not a nil type.
//
// By default, interfaces are only nil when both their type and value is nil.
// This means that when you pass a pointer to a nil value, the type of the interface
// will still show as nil, which means that if you don't cast the interface after
// retrieving it, a nil check will return that the value is not false.
forceNilInterfaceOnNilPointer bool
}
// MaxSize returns the maximum amount of keys that can be present in the cache before
// new entries trigger the eviction of the tail
func (cache *Cache) MaxSize() int {
return cache.maxSize
}
// MaxMemoryUsage returns the configured maxMemoryUsage of the cache
func (cache *Cache) MaxMemoryUsage() int {
return cache.maxMemoryUsage
}
// EvictionPolicy returns the EvictionPolicy of the Cache
func (cache *Cache) EvictionPolicy() EvictionPolicy {
return cache.evictionPolicy
}
// Stats returns statistics from the cache
func (cache *Cache) Stats() Statistics {
cache.mutex.RLock()
stats := Statistics{
EvictedKeys: cache.stats.EvictedKeys,
ExpiredKeys: cache.stats.ExpiredKeys,
Hits: cache.stats.Hits,
Misses: cache.stats.Misses,
}
cache.mutex.RUnlock()
return stats
}
// MemoryUsage returns the current memory usage of the cache's dataset in bytes
// If MaxMemoryUsage is set to NoMaxMemoryUsage, this will return 0
func (cache *Cache) MemoryUsage() int {
return cache.memoryUsage
}
// WithMaxSize sets the maximum amount of entries that can be in the cache at any given time
// A maxSize of 0 or less means infinite
func (cache *Cache) WithMaxSize(maxSize int) *Cache {
if maxSize < 0 {
maxSize = NoMaxSize
}
if maxSize != NoMaxSize && cache.Count() == 0 {
cache.entries = make(map[string]*Entry, maxSize)
}
cache.maxSize = maxSize
return cache
}
// WithMaxMemoryUsage sets the maximum amount of memory that can be used by the cache at any given time
//
// NOTE: This is approximate.
//
// Setting this to NoMaxMemoryUsage will disable eviction by memory usage
func (cache *Cache) WithMaxMemoryUsage(maxMemoryUsageInBytes int) *Cache {
if maxMemoryUsageInBytes < 0 {
maxMemoryUsageInBytes = NoMaxMemoryUsage
}
cache.maxMemoryUsage = maxMemoryUsageInBytes
return cache
}
// WithEvictionPolicy sets eviction algorithm.
//
// Defaults to FirstInFirstOut (FIFO)
func (cache *Cache) WithEvictionPolicy(policy EvictionPolicy) *Cache {
cache.evictionPolicy = policy
return cache
}
// WithDefaultTTL sets the default TTL for each entry (unless a different TTL is specified using SetWithTTL or SetAllWithTTL)
//
// Defaults to NoExpiration (-1)
func (cache *Cache) WithDefaultTTL(ttl time.Duration) *Cache {
if ttl > 1 {
cache.defaultTTL = ttl
}
return cache
}
// WithForceNilInterfaceOnNilPointer sets whether all Set-like functions should set a value as nil if the
// interface passed has a nil value but not a nil type.
//
// In Go, an interface is only nil if both its type and value are nil, which means that a nil pointer
// (e.g. (*Struct)(nil)) will retain its attribution to the type, and the unmodified value returned from
// Cache.Get, for instance, would return false when compared with nil if this option is set to false.
//
// We can bypass this by detecting if the interface's value is nil and setting it to nil rather than
// a nil pointer, which will make the value returned from Cache.Get return true when compared with nil.
// This is exactly what passing true to WithForceNilInterfaceOnNilPointer does, and it's also the default behavior.
//
// Alternatively, you may pass false to WithForceNilInterfaceOnNilPointer, which will mean that you'll have
// to cast the value returned from Cache.Get to its original type to check for whether the pointer returned
// is nil or not.
//
// If set to true (default):
//
// cache := gocache.NewCache().WithForceNilInterfaceOnNilPointer(true)
// cache.Set("key", (*Struct)(nil))
// value, _ := cache.Get("key")
// // the following returns true, because the interface{} (any) was forcefully set to nil
// if value == nil {}
// // the following will panic, because the value has been casted to its type (which is nil)
// if value.(*Struct) == nil {}
//
// If set to false:
//
// cache := gocache.NewCache().WithForceNilInterfaceOnNilPointer(false)
// cache.Set("key", (*Struct)(nil))
// value, _ := cache.Get("key")
// // the following returns false, because the interface{} (any) returned has a non-nil type (*Struct)
// if value == nil {}
// // the following returns true, because the value has been cast to its type
// if value.(*Struct) == nil {}
//
// In other words, if set to true, you do not need to cast the value returned from the cache to
// check if the value is nil.
//
// Defaults to true
func (cache *Cache) WithForceNilInterfaceOnNilPointer(forceNilInterfaceOnNilPointer bool) *Cache {
cache.forceNilInterfaceOnNilPointer = forceNilInterfaceOnNilPointer
return cache
}
// NewCache creates a new Cache
//
// Should be used in conjunction with Cache.WithMaxSize, Cache.WithMaxMemoryUsage and/or Cache.WithEvictionPolicy
//
// gocache.NewCache().WithMaxSize(10000).WithEvictionPolicy(gocache.LeastRecentlyUsed)
func NewCache() *Cache {
return &Cache{
maxSize: DefaultMaxSize,
evictionPolicy: FirstInFirstOut,
defaultTTL: NoExpiration,
stats: &Statistics{},
entries: make(map[string]*Entry),
mutex: sync.RWMutex{},
stopJanitor: nil,
forceNilInterfaceOnNilPointer: true,
}
}
// Set creates or updates a key with a given value
func (cache *Cache) Set(key string, value any) {
cache.SetWithTTL(key, value, cache.defaultTTL)
}
// SetWithTTL creates or updates a key with a given value and sets an expiration time (-1 is NoExpiration)
//
// The TTL provided must be greater than 0, or NoExpiration (-1). If a negative value that isn't -1 (NoExpiration) is
// provided, the entry will not be created if the key doesn't exist
func (cache *Cache) SetWithTTL(key string, value any, ttl time.Duration) {
// An interface is only nil if both its value and its type are nil, however, passing a nil pointer as an interface{}
// means that the interface itself is not nil, because the interface value is nil but not the type.
if cache.forceNilInterfaceOnNilPointer {
if value != nil && (reflect.ValueOf(value).Kind() == reflect.Ptr && reflect.ValueOf(value).IsNil()) {
value = nil
}
}
cache.mutex.Lock()
entry, ok := cache.get(key)
if !ok {
// A negative TTL that isn't -1 (NoExpiration) or 0 is an entry that will expire instantly,
// so might as well just not create it in the first place
if ttl != NoExpiration && ttl < 1 {
cache.mutex.Unlock()
return
}
// Cache entry doesn't exist, so we have to create a new one
entry = &Entry{
Key: key,
Value: value,
RelevantTimestamp: time.Now(),
next: cache.head,
}
if cache.head == nil {
cache.tail = entry
} else {
cache.head.previous = entry
}
cache.head = entry
cache.entries[key] = entry
if cache.maxMemoryUsage != NoMaxMemoryUsage {
cache.memoryUsage += entry.SizeInBytes()
}
} else {
// A negative TTL that isn't -1 (NoExpiration) or 0 is an entry that will expire instantly,
// so might as well just delete it immediately instead of updating it
if ttl != NoExpiration && ttl < 1 {
cache.delete(key)
cache.mutex.Unlock()
return
}
if cache.maxMemoryUsage != NoMaxMemoryUsage {
// Subtract the old entry from the cache's memoryUsage
cache.memoryUsage -= entry.SizeInBytes()
}
// Update existing entry's value
entry.Value = value
entry.RelevantTimestamp = time.Now()
if cache.maxMemoryUsage != NoMaxMemoryUsage {
// Add the memory usage of the new entry to the cache's memoryUsage
cache.memoryUsage += entry.SizeInBytes()
}
// Because we just updated the entry, we need to move it back to HEAD
cache.moveExistingEntryToHead(entry)
}
if ttl != NoExpiration {
entry.Expiration = time.Now().Add(ttl).UnixNano()
} else {
entry.Expiration = NoExpiration
}
// If the cache doesn't have a maxSize/maxMemoryUsage, then there's no point
// checking if we need to evict an entry, so we'll just return now
if cache.maxSize == NoMaxSize && cache.maxMemoryUsage == NoMaxMemoryUsage {
cache.mutex.Unlock()
return
}
// If there's a maxSize and the cache has more entries than the maxSize, evict
if cache.maxSize != NoMaxSize && len(cache.entries) > cache.maxSize {
cache.evict()
}
// If there's a maxMemoryUsage and the memoryUsage is above the maxMemoryUsage, evict
if cache.maxMemoryUsage != NoMaxMemoryUsage && cache.memoryUsage > cache.maxMemoryUsage {
for cache.memoryUsage > cache.maxMemoryUsage && len(cache.entries) > 0 {
cache.evict()
}
}
cache.mutex.Unlock()
}
// SetAll creates or updates multiple values
func (cache *Cache) SetAll(entries map[string]any) {
cache.SetAllWithTTL(entries, cache.defaultTTL)
}
// SetAllWithTTL creates or updates multiple values
func (cache *Cache) SetAllWithTTL(entries map[string]any, ttl time.Duration) {
for key, value := range entries {
cache.SetWithTTL(key, value, ttl)
}
}
// Get retrieves an entry using the key passed as parameter
// If there is no such entry, the value returned will be nil and the boolean will be false
// If there is an entry, the value returned will be the value cached and the boolean will be true
func (cache *Cache) Get(key string) (any, bool) {
cache.mutex.Lock()
entry, ok := cache.get(key)
if !ok {
cache.stats.Misses++
cache.mutex.Unlock()
return nil, false
}
if entry.Expired() {
cache.stats.ExpiredKeys++
cache.delete(key)
cache.mutex.Unlock()
return nil, false
}
cache.stats.Hits++
if cache.evictionPolicy == LeastRecentlyUsed {
entry.Accessed()
if cache.head == entry {
cache.mutex.Unlock()
return entry.Value, true
}
// Because the eviction policy is LRU, we need to move the entry back to HEAD
cache.moveExistingEntryToHead(entry)
}
cache.mutex.Unlock()
return entry.Value, true
}
// GetValue retrieves an entry using the key passed as parameter
// Unlike Get, this function only returns the value
func (cache *Cache) GetValue(key string) any {
value, _ := cache.Get(key)
return value
}
// GetByKeys retrieves multiple entries using the keys passed as parameter
// All keys are returned in the map, regardless of whether they exist or not, however, entries that do not exist in the
// cache will return nil, meaning that there is no way of determining whether a key genuinely has the value nil, or
// whether it doesn't exist in the cache using only this function.
func (cache *Cache) GetByKeys(keys []string) map[string]any {
entries := make(map[string]any)
for _, key := range keys {
entries[key], _ = cache.Get(key)
}
return entries
}
// GetAll retrieves all cache entries
//
// If the eviction policy is LeastRecentlyUsed, note that unlike Get and GetByKeys, this does not update the last access
// timestamp. The reason for this is that since all cache entries will be accessed, updating the last access timestamp
// would provide very little benefit while harming the ability to accurately determine the next key that will be evicted
//
// You should probably avoid using this if you have a lot of entries.
//
// GetKeysByPattern is a good alternative if you want to retrieve entries that you do not have the key for, as it only
// retrieves the keys and does not trigger active eviction and has a parameter for setting a limit to the number of keys
// you wish to retrieve.
func (cache *Cache) GetAll() map[string]any {
entries := make(map[string]any)
cache.mutex.Lock()
for key, entry := range cache.entries {
if entry.Expired() {
cache.delete(key)
continue
}
entries[key] = entry.Value
}
cache.stats.Hits += uint64(len(entries))
cache.mutex.Unlock()
return entries
}
// GetKeysByPattern retrieves a slice of keys that match a given pattern
// If the limit is set to 0, the entire cache will be searched for matching keys.
// If the limit is above 0, the search will stop once the specified number of matching keys have been found.
//
// e.g.
//
// cache.GetKeysByPattern("*some*", 0) will return all keys containing "some" in them
// cache.GetKeysByPattern("*some*", 5) will return 5 keys (or less) containing "some" in them
//
// Note that GetKeysByPattern does not trigger active evictions, nor does it count as accessing the entry (if LRU).
// The reason for that behavior is that these two (active eviction and access) only applies when you access the value
// of the cache entry, and this function only returns the keys.
func (cache *Cache) GetKeysByPattern(pattern string, limit int) []string {
var matchingKeys []string
cache.mutex.Lock()
for key, value := range cache.entries {
if value.Expired() {
continue
}
if MatchPattern(pattern, key) {
matchingKeys = append(matchingKeys, key)
if limit > 0 && len(matchingKeys) >= limit {
break
}
}
}
cache.mutex.Unlock()
return matchingKeys
}
// Delete removes a key from the cache
//
// Returns false if the key did not exist.
func (cache *Cache) Delete(key string) bool {
cache.mutex.Lock()
ok := cache.delete(key)
cache.mutex.Unlock()
return ok
}
// DeleteAll deletes multiple entries based on the keys passed as parameter
//
// Returns the number of keys deleted
func (cache *Cache) DeleteAll(keys []string) int {
numberOfKeysDeleted := 0
cache.mutex.Lock()
for _, key := range keys {
if cache.delete(key) {
numberOfKeysDeleted++
}
}
cache.mutex.Unlock()
return numberOfKeysDeleted
}
// DeleteKeysByPattern deletes all entries matching a given key pattern and returns the number of entries deleted.
//
// Note that DeleteKeysByPattern does not trigger active evictions, nor does it count as accessing the entry (if LRU).
func (cache *Cache) DeleteKeysByPattern(pattern string) int {
return cache.DeleteAll(cache.GetKeysByPattern(pattern, 0))
}
// Count returns the total amount of entries in the cache, regardless of whether they're expired or not
func (cache *Cache) Count() int {
cache.mutex.RLock()
count := len(cache.entries)
cache.mutex.RUnlock()
return count
}
// Clear deletes all entries from the cache
func (cache *Cache) Clear() {
cache.mutex.Lock()
cache.entries = make(map[string]*Entry)
cache.memoryUsage = 0
cache.head = nil
cache.tail = nil
cache.mutex.Unlock()
}
// TTL returns the time until the cache entry specified by the key passed as parameter
// will be deleted.
func (cache *Cache) TTL(key string) (time.Duration, error) {
cache.mutex.RLock()
entry, ok := cache.get(key)
cache.mutex.RUnlock()
if !ok {
return 0, ErrKeyDoesNotExist
}
if entry.Expiration == NoExpiration {
return 0, ErrKeyHasNoExpiration
}
timeUntilExpiration := time.Until(time.Unix(0, entry.Expiration))
if timeUntilExpiration < 0 {
// The key has already expired but hasn't been deleted yet.
// From the client's perspective, this means that the cache entry doesn't exist
return 0, ErrKeyDoesNotExist
}
return timeUntilExpiration, nil
}
// Expire sets a key's expiration time
//
// A TTL of -1 means that the key will never expire
// A TTL of 0 means that the key will expire immediately
// If using LRU, note that this does not reset the position of the key
//
// Returns true if the cache key exists and has had its expiration time altered
func (cache *Cache) Expire(key string, ttl time.Duration) bool {
entry, ok := cache.get(key)
if !ok || entry.Expired() {
return false
}
if ttl != NoExpiration {
entry.Expiration = time.Now().Add(ttl).UnixNano()
} else {
entry.Expiration = NoExpiration
}
return true
}
// get retrieves an entry using the key passed as parameter, but unlike Get, it doesn't update the access time or
// move the position of the entry to the head
func (cache *Cache) get(key string) (*Entry, bool) {
entry, ok := cache.entries[key]
return entry, ok
}
func (cache *Cache) delete(key string) bool {
entry, ok := cache.entries[key]
if ok {
if cache.maxMemoryUsage != NoMaxMemoryUsage {
cache.memoryUsage -= entry.SizeInBytes()
}
cache.removeExistingEntryReferences(entry)
delete(cache.entries, key)
}
return ok
}
// moveExistingEntryToHead replaces the current cache head for an existing entry
func (cache *Cache) moveExistingEntryToHead(entry *Entry) {
if !(entry == cache.head && entry == cache.tail) {
cache.removeExistingEntryReferences(entry)
}
if entry != cache.head {
entry.next = cache.head
entry.previous = nil
if cache.head != nil {
cache.head.previous = entry
}
cache.head = entry
}
}
// removeExistingEntryReferences modifies the next and previous reference of an existing entry and re-links
// the next and previous entry accordingly, as well as the cache head or/and the cache tail if necessary.
// Note that it does not remove the entry from the cache, only the references.
func (cache *Cache) removeExistingEntryReferences(entry *Entry) {
if cache.tail == entry && cache.head == entry {
cache.tail = nil
cache.head = nil
} else if cache.tail == entry {
cache.tail = cache.tail.previous
} else if cache.head == entry {
cache.head = cache.head.next
}
if entry.previous != nil {
entry.previous.next = entry.next
}
if entry.next != nil {
entry.next.previous = entry.previous
}
entry.next = nil
entry.previous = nil
}
// evict removes the tail from the cache
func (cache *Cache) evict() {
if cache.tail == nil || len(cache.entries) == 0 {
return
}
if cache.tail != nil {
oldTail := cache.tail
cache.removeExistingEntryReferences(oldTail)
delete(cache.entries, oldTail.Key)
if cache.maxMemoryUsage != NoMaxMemoryUsage {
cache.memoryUsage -= oldTail.SizeInBytes()
}
cache.stats.EvictedKeys++
}
}

251
gocache_bench_test.go Normal file
View file

@ -0,0 +1,251 @@
package gocache
import (
"fmt"
"math/rand"
"strconv"
"strings"
"testing"
)
func BenchmarkMap_Get(b *testing.B) {
m := make(map[string]any)
for n := 0; n < b.N; n++ {
_, _ = m[strconv.Itoa(n)]
}
b.ReportAllocs()
}
func BenchmarkMap_Set(b *testing.B) {
values := map[string]string{
"small": "a",
"medium": strings.Repeat("a", 1024),
"large": strings.Repeat("a", 1024*100),
}
for name, value := range values {
b.Run(fmt.Sprintf("%s value", name), func(b *testing.B) {
m := make(map[string]any)
for n := 0; n < b.N; n++ {
m[strconv.Itoa(n)] = value
}
b.ReportAllocs()
})
}
}
func BenchmarkCache_Get(b *testing.B) {
evictionPolicies := []EvictionPolicy{FirstInFirstOut, LeastRecentlyUsed}
for _, evictionPolicy := range evictionPolicies {
cache := NewCache().WithMaxSize(NoMaxSize).WithMaxMemoryUsage(NoMaxMemoryUsage)
b.Run(string(evictionPolicy), func(b *testing.B) {
for n := 0; n < b.N; n++ {
cache.Get(strconv.Itoa(n))
}
b.ReportAllocs()
})
}
}
func BenchmarkCache_Set(b *testing.B) {
values := map[string]string{
"small": "a",
"medium": strings.Repeat("a", 1024),
"large": strings.Repeat("a", 1024*100),
}
evictionPolicies := []EvictionPolicy{FirstInFirstOut, LeastRecentlyUsed}
for _, evictionPolicy := range evictionPolicies {
for name, value := range values {
b.Run(fmt.Sprintf("%s %s value", evictionPolicy, name), func(b *testing.B) {
cache := NewCache().WithMaxSize(NoMaxSize).WithMaxMemoryUsage(NoMaxMemoryUsage).WithEvictionPolicy(evictionPolicy)
for n := 0; n < b.N; n++ {
cache.Set(strconv.Itoa(n), value)
}
b.ReportAllocs()
})
}
}
}
// BenchmarkCache_SetUsingMaxMemoryUsage does NOT test evictions, it tests the overhead of the extra work
// automatically performed when using MaxMemoryUsage
func BenchmarkCache_SetUsingMaxMemoryUsage(b *testing.B) {
values := map[string]string{
"small": "a",
"medium": strings.Repeat("a", 1024),
"large": strings.Repeat("a", 1024*100),
}
for name, value := range values {
b.Run(fmt.Sprintf("%s value", name), func(b *testing.B) {
cache := NewCache().WithMaxSize(NoMaxSize).WithMaxMemoryUsage(999 * Gigabyte)
for n := 0; n < b.N; n++ {
cache.Set(strconv.Itoa(n), value)
}
b.ReportAllocs()
})
}
}
func BenchmarkCache_SetWithMaxSize(b *testing.B) {
values := map[string]string{
"small": "a",
"medium": strings.Repeat("a", 1024),
"large": strings.Repeat("a", 1024*100),
}
maxSizes := []int{100, 10000, 100000}
for name, value := range values {
for _, maxSize := range maxSizes {
b.Run(fmt.Sprintf("%d %s value", maxSize, name), func(b *testing.B) {
cache := NewCache().WithMaxSize(maxSize)
for n := 0; n < b.N; n++ {
cache.Set(strconv.Itoa(n), value)
}
b.ReportAllocs()
})
}
}
}
func BenchmarkCache_SetWithMaxSizeAndLRU(b *testing.B) {
values := map[string]string{
"small": "a",
"medium": strings.Repeat("a", 1024),
"large": strings.Repeat("a", 1024*100),
}
maxSizes := []int{100, 10000, 100000}
for name, value := range values {
for _, maxSize := range maxSizes {
b.Run(fmt.Sprintf("%d %s value", maxSize, name), func(b *testing.B) {
cache := NewCache().WithMaxSize(maxSize).WithEvictionPolicy(LeastRecentlyUsed)
for n := 0; n < b.N; n++ {
cache.Set(strconv.Itoa(n), value)
}
b.ReportAllocs()
})
}
}
}
func BenchmarkCache_GetSetMultipleConcurrent(b *testing.B) {
data := map[string]string{
"k1": "v1",
"k2": "v2",
"k3": "v3",
"k4": "v4",
"k5": "v5",
"k6": "v6",
"k7": "v7",
"k8": "v8",
}
cache := NewCache().WithMaxSize(NoMaxSize)
b.RunParallel(func(pb *testing.PB) {
for pb.Next() {
for k, v := range data {
cache.Set(k, v)
cache.Get(k)
}
}
})
b.ReportAllocs()
}
func BenchmarkCache_GetSetConcurrentWithFrequentEviction(b *testing.B) {
value := strings.Repeat("a", 256)
evictionPolicies := []EvictionPolicy{FirstInFirstOut, LeastRecentlyUsed}
for _, evictionPolicy := range evictionPolicies {
b.Run(string(evictionPolicy), func(b *testing.B) {
cache := NewCache().WithEvictionPolicy(LeastRecentlyUsed).WithMaxSize(3).WithMaxMemoryUsage(NoMaxMemoryUsage)
b.RunParallel(func(pb *testing.PB) {
for pb.Next() {
k := strconv.Itoa(rand.Intn(15))
cache.Set(k, value)
_, _ = cache.Get(k)
}
})
b.ReportAllocs()
})
}
}
func BenchmarkCache_GetConcurrently(b *testing.B) {
value := strings.Repeat("a", 256)
for _, evictionPolicy := range []EvictionPolicy{FirstInFirstOut, LeastRecentlyUsed} {
b.Run(string(evictionPolicy), func(b *testing.B) {
cache := NewCache().WithMaxSize(100000)
for i := 0; i < 100000; i++ {
cache.Set(strconv.Itoa(i), value)
}
b.RunParallel(func(pb *testing.PB) {
for pb.Next() {
key := strconv.Itoa(rand.Intn(100000))
val, ok := cache.Get(key)
if !ok {
b.Errorf("key: %v; value: %v", key, val)
}
if val != value {
b.Errorf("expected: %v; got: %v", val, value)
}
}
})
b.ReportAllocs()
})
}
}
// Note: The default value for Cache.forceNilInterfaceOnNilPointer is true
func BenchmarkCache_WithForceNilInterfaceOnNilPointer(b *testing.B) {
const (
Min = 10000
Max = 99999
)
type Struct struct {
Value string
}
forceNilInterfaceOnNilPointerValues := []bool{true, false}
values := []*Struct{nil, {Value: "value"}}
for _, forceNilInterfaceOnNilPointer := range forceNilInterfaceOnNilPointerValues {
for _, value := range values {
name := fmt.Sprintf("%v", forceNilInterfaceOnNilPointer)
if value == nil {
name += " with nil struct pointer"
}
b.Run(name, func(b *testing.B) {
cache := NewCache().WithMaxSize(NoMaxSize).WithMaxMemoryUsage(NoMaxMemoryUsage).WithForceNilInterfaceOnNilPointer(forceNilInterfaceOnNilPointer)
for n := 0; n < b.N; n++ {
cache.Set(strconv.Itoa(rand.Intn(Max-Min)+Min), value)
}
b.ReportAllocs()
})
}
}
}
func BenchmarkCache_WithForceNilInterfaceOnNilPointerWithConcurrency(b *testing.B) {
const (
Min = 10000
Max = 99999
)
type Struct struct {
Value string
}
forceNilInterfaceOnNilPointerValues := []bool{true, false}
values := []*Struct{nil, {Value: "value"}}
for _, forceNilInterfaceOnNilPointer := range forceNilInterfaceOnNilPointerValues {
for _, value := range values {
name := fmt.Sprintf("%v", forceNilInterfaceOnNilPointer)
if value == nil {
name += " with nil struct pointer"
}
b.Run(name, func(b *testing.B) {
cache := NewCache().WithMaxSize(NoMaxSize).WithMaxMemoryUsage(NoMaxMemoryUsage).WithForceNilInterfaceOnNilPointer(forceNilInterfaceOnNilPointer)
b.RunParallel(func(pb *testing.PB) {
for pb.Next() {
cache.Set(strconv.Itoa(rand.Intn(Max-Min)+Min), value)
}
})
b.ReportAllocs()
})
}
}
}

1122
gocache_test.go Normal file
View file

File diff suppressed because it is too large Load diff

146
janitor.go Normal file
View file

@ -0,0 +1,146 @@
package gocache
import (
"log"
"time"
)
const (
// JanitorShiftTarget is the target number of expired keys to find during passive clean up duty
// before pausing the passive expired keys eviction process
JanitorShiftTarget = 25
// JanitorMaxIterationsPerShift is the maximum number of nodes to traverse before pausing
//
// This is to prevent the janitor from traversing the entire cache, which could take a long time
// to complete depending on the size of the cache.
//
// By limiting it to a small number, we are effectively reducing the impact of passive eviction.
JanitorMaxIterationsPerShift = 1000
// JanitorMinShiftBackOff is the minimum interval between each iteration of steps
// defined by JanitorMaxIterationsPerShift
JanitorMinShiftBackOff = 50 * time.Millisecond
// JanitorMaxShiftBackOff is the maximum interval between each iteration of steps
// defined by JanitorMaxIterationsPerShift
JanitorMaxShiftBackOff = 500 * time.Millisecond
)
// StartJanitor starts the janitor on a different goroutine
// The janitor's job is to delete expired keys in the background, in other words, it takes care of passive eviction.
// It can be stopped by calling Cache.StopJanitor.
// If you do not start the janitor, expired keys will only be deleted when they are accessed through Get, GetByKeys, or
// GetAll.
func (cache *Cache) StartJanitor() error {
if cache.stopJanitor != nil {
return ErrJanitorAlreadyRunning
}
cache.stopJanitor = make(chan bool)
go func() {
// rather than starting from the tail on every run, we can try to start from the last traversed entry
var lastTraversedNode *Entry
totalNumberOfExpiredKeysInPreviousRunFromTailToHead := 0
backOff := JanitorMinShiftBackOff
for {
select {
case <-time.After(backOff):
// Passive clean up duty
cache.mutex.Lock()
if cache.tail != nil {
start := time.Now()
steps := 0
expiredEntriesFound := 0
current := cache.tail
if lastTraversedNode != nil {
// Make sure the lastTraversedNode is still in the cache, otherwise we might be traversing nodes that were already deleted.
// Furthermore, we need to make sure that the entry from the cache has the same pointer as the lastTraversedNode
// to verify that there isn't just a new cache entry with the same key (i.e. in case lastTraversedNode got evicted)
if entryFromCache, isInCache := cache.get(lastTraversedNode.Key); isInCache && entryFromCache == lastTraversedNode {
current = lastTraversedNode
}
}
if current == cache.tail {
if Debug {
log.Printf("There are currently %d entries in the cache. The last walk resulted in finding %d expired keys", len(cache.entries), totalNumberOfExpiredKeysInPreviousRunFromTailToHead)
}
totalNumberOfExpiredKeysInPreviousRunFromTailToHead = 0
}
for current != nil {
// since we're walking from the tail to the head, we get the previous reference
var previous *Entry
steps++
if current.Expired() {
expiredEntriesFound++
// Because delete will remove the previous reference from the entry, we need to store the
// previous reference before we delete it
previous = current.previous
cache.delete(current.Key)
cache.stats.ExpiredKeys++
}
if current == cache.head {
lastTraversedNode = nil
break
}
// Travel to the current node's previous node only if no specific previous node has been specified
if previous != nil {
current = previous
} else {
current = current.previous
}
lastTraversedNode = current
if steps == JanitorMaxIterationsPerShift || expiredEntriesFound >= JanitorShiftTarget {
if expiredEntriesFound > 0 {
backOff = JanitorMinShiftBackOff
} else {
if backOff*2 <= JanitorMaxShiftBackOff {
backOff *= 2
} else {
backOff = JanitorMaxShiftBackOff
}
}
break
}
}
if Debug {
log.Printf("traversed %d nodes and found %d expired entries in %s before stopping\n", steps, expiredEntriesFound, time.Since(start))
}
totalNumberOfExpiredKeysInPreviousRunFromTailToHead += expiredEntriesFound
} else {
if backOff*2 < JanitorMaxShiftBackOff {
backOff *= 2
} else {
backOff = JanitorMaxShiftBackOff
}
}
cache.mutex.Unlock()
case <-cache.stopJanitor:
cache.stopJanitor <- true
return
}
}
}()
//if Debug {
// go func() {
// var m runtime.MemStats
// for {
// runtime.ReadMemStats(&m)
// log.Printf("Alloc=%vMB; HeapReleased=%vMB; Sys=%vMB; HeapInUse=%vMB; HeapObjects=%v; HeapObjectsFreed=%v; GC=%v; cache.memoryUsage=%vMB; cacheSize=%d\n", m.Alloc/1024/1024, m.HeapReleased/1024/1024, m.Sys/1024/1024, m.HeapInuse/1024/1024, m.HeapObjects, m.Frees, m.NumGC, cache.memoryUsage/1024/1024, cache.Count())
// time.Sleep(3 * time.Second)
// }
// }()
//}
return nil
}
// StopJanitor stops the janitor
func (cache *Cache) StopJanitor() {
if cache.stopJanitor != nil {
// Tell the janitor to stop, and then wait for the janitor to reply on the same channel that it's stopping
// This may seem a bit odd, but this allows us to avoid a data race condition when trying to set
// cache.stopJanitor to nil
cache.stopJanitor <- true
<-cache.stopJanitor
cache.stopJanitor = nil
}
}

129
janitor_test.go Normal file
View file

@ -0,0 +1,129 @@
package gocache
import (
"fmt"
"testing"
"time"
)
func TestCache_StartJanitor(t *testing.T) {
cache := NewCache()
cache.SetWithTTL("1", "1", time.Nanosecond)
if cacheSize := cache.Count(); cacheSize != 1 {
t.Errorf("expected cacheSize to be 1, but was %d", cacheSize)
}
err := cache.StartJanitor()
if err != nil {
t.Fatal(err)
}
defer cache.StopJanitor()
time.Sleep(JanitorMinShiftBackOff * 2)
if cacheSize := cache.Count(); cacheSize != 0 {
t.Errorf("expected cacheSize to be 0, but was %d", cacheSize)
}
}
func TestCache_StartJanitorWhenAlreadyStarted(t *testing.T) {
cache := NewCache()
if err := cache.StartJanitor(); err != nil {
t.Fatal(err)
}
if err := cache.StartJanitor(); err == nil {
t.Fatal("expected StartJanitor to return an error, because the janitor is already started")
}
cache.StopJanitor()
}
func TestCache_StopJanitor(t *testing.T) {
cache := NewCache()
_ = cache.StartJanitor()
if cache.stopJanitor == nil {
t.Error("starting the janitor should've initialized cache.stopJanitor")
}
cache.StopJanitor()
if cache.stopJanitor != nil {
t.Error("stopping the janitor should've set cache.stopJanitor to nil")
}
// Check if stopping the janitor even though it's already stopped causes a panic
cache.StopJanitor()
}
func TestJanitor(t *testing.T) {
cache := NewCache().WithMaxSize(3 * JanitorMaxIterationsPerShift)
defer cache.Clear()
for i := 0; i < 3*JanitorMaxIterationsPerShift; i++ {
if i < JanitorMaxIterationsPerShift && i%2 == 0 {
cache.SetWithTTL(fmt.Sprintf("%d", i), "value", time.Millisecond)
} else {
cache.SetWithTTL(fmt.Sprintf("%d", i), "value", time.Hour)
}
}
cacheSize := cache.Count()
err := cache.StartJanitor()
if err != nil {
t.Fatal(err)
}
defer cache.StopJanitor()
time.Sleep(JanitorMinShiftBackOff * 4)
if cacheSize <= cache.Count() {
t.Error("The janitor should be deleting expired cache entries")
}
cacheSize = cache.Count()
time.Sleep(JanitorMinShiftBackOff * 4)
if cacheSize <= cache.Count() {
t.Error("The janitor should be deleting expired cache entries")
}
cacheSize = cache.Count()
time.Sleep(JanitorMinShiftBackOff * 4)
if cacheSize <= cache.Count() {
t.Error("The janitor should be deleting expired cache entries")
}
}
func TestJanitorIsLoopingProperly(t *testing.T) {
cache := NewCache().WithMaxSize(JanitorMaxIterationsPerShift + 3)
defer cache.Clear()
for i := 0; i < JanitorMaxIterationsPerShift; i++ {
cache.SetWithTTL(fmt.Sprintf("%d", i), "value", time.Hour)
}
cache.SetWithTTL("key-to-expire-1", "value", JanitorMinShiftBackOff*2)
cache.SetWithTTL("key-to-expire-2", "value", JanitorMinShiftBackOff*2)
cache.SetWithTTL("key-to-expire-3", "value", JanitorMinShiftBackOff*2)
err := cache.StartJanitor()
if err != nil {
t.Fatal(err)
}
defer cache.StopJanitor()
if cache.Count() != JanitorMaxIterationsPerShift+3 {
t.Error("The janitor shouldn't have had enough time to remove anything from the cache yet", cache.Count())
}
const timeout = JanitorMinShiftBackOff * 20
threeKeysExpiredWithinOneSecond := false
for start := time.Now(); time.Since(start) < timeout; {
if cache.Stats().ExpiredKeys == 3 {
threeKeysExpiredWithinOneSecond = true
break
}
time.Sleep(JanitorMinShiftBackOff)
}
if !threeKeysExpiredWithinOneSecond {
t.Error("expected 3 keys to expire within 1 second")
}
if cache.Count() != JanitorMaxIterationsPerShift {
t.Error("The janitor should've deleted 3 entries")
}
}
func TestJanitorDoesNotThrowATantrumWhenThereIsNothingToClean(t *testing.T) {
cache := NewCache()
start := time.Now()
_ = cache.StartJanitor()
defer cache.StopJanitor()
time.Sleep(JanitorMaxShiftBackOff * 3)
// Technically, if the janitor doesn't backoff properly, the sleep above is likely to take more time than the sleep
// below because it would be eating up the CPU.
// This is a far-fetched test, but it's a good sanity check.
if time.Since(start) > JanitorMaxShiftBackOff*4 {
t.Error("The janitor should've backed off and prevented CPU usage from throttling the application")
}
}

12
pattern.go Normal file
View file

@ -0,0 +1,12 @@
package gocache
import "path/filepath"
// MatchPattern checks whether a string matches a pattern
func MatchPattern(pattern, s string) bool {
if pattern == "*" {
return true
}
matched, _ := filepath.Match(pattern, s)
return matched
}

121
pattern_test.go Normal file
View file

@ -0,0 +1,121 @@
package gocache
import "testing"
func TestMatchPattern(t *testing.T) {
scenarios := []struct {
pattern string
key string
expectedToMatch bool
}{
{
pattern: "*",
key: "livingroom_123",
expectedToMatch: true,
},
{
pattern: "*",
key: "livingroom_123",
expectedToMatch: true,
},
{
pattern: "**",
key: "livingroom_123",
expectedToMatch: true,
},
{
pattern: "living*",
key: "livingroom_123",
expectedToMatch: true,
},
{
pattern: "*living*",
key: "livingroom_123",
expectedToMatch: true,
},
{
pattern: "*123",
key: "livingroom_123",
expectedToMatch: true,
},
{
pattern: "*_*",
key: "livingroom_123",
expectedToMatch: true,
},
{
pattern: "living*_*3",
key: "livingroom_123",
expectedToMatch: true,
},
{
pattern: "living*room_*3",
key: "livingroom_123",
expectedToMatch: true,
},
{
pattern: "living*room_*3",
key: "livingroom_123",
expectedToMatch: true,
},
{
pattern: "*vin*om*2*",
key: "livingroom_123",
expectedToMatch: true,
},
{
pattern: "livingroom_123",
key: "livingroom_123",
expectedToMatch: true,
},
{
pattern: "*livingroom_123*",
key: "livingroom_123",
expectedToMatch: true,
},
{
pattern: "livingroom",
key: "livingroom_123",
expectedToMatch: false,
},
{
pattern: "livingroom123",
key: "livingroom_123",
expectedToMatch: false,
},
{
pattern: "what",
key: "livingroom_123",
expectedToMatch: false,
},
{
pattern: "*what*",
key: "livingroom_123",
expectedToMatch: false,
},
{
pattern: "*.*",
key: "livingroom_123",
expectedToMatch: false,
},
{
pattern: "room*123",
key: "livingroom_123",
expectedToMatch: false,
},
}
for _, scenario := range scenarios {
t.Run(scenario.pattern+"---"+scenario.key, func(t *testing.T) {
matched := MatchPattern(scenario.pattern, scenario.key)
if scenario.expectedToMatch {
if !matched {
t.Errorf("%s should've matched pattern '%s'", scenario.key, scenario.pattern)
}
} else {
if matched {
t.Errorf("%s shouldn't have matched pattern '%s'", scenario.key, scenario.pattern)
}
}
})
}
}

33
policy.go Normal file
View file

@ -0,0 +1,33 @@
package gocache
// EvictionPolicy is what dictates how evictions are handled
type EvictionPolicy string
const (
// LeastRecentlyUsed is an eviction policy that causes the most recently accessed cache entry to be moved to the
// head of the cache. Effectively, this causes the cache entries that have not been accessed for some time to
// gradually move closer and closer to the tail, and since the tail is the entry that gets deleted when an eviction
// is required, it allows less used cache entries to be evicted while keeping recently accessed entries at or close
// to the head.
//
// For instance, creating a Cache with a Cache.MaxSize of 3 and creating the entries 1, 2 and 3 in that order would
// put 3 at the head and 1 at the tail:
// 3 (head) -> 2 -> 1 (tail)
// If the cache entry 1 was then accessed, 1 would become the head and 2 the tail:
// 1 (head) -> 3 -> 2 (tail)
// If a cache entry 4 was then created, because the Cache.MaxSize is 3, the tail (2) would then be evicted:
// 4 (head) -> 1 -> 3 (tail)
LeastRecentlyUsed EvictionPolicy = "LeastRecentlyUsed"
// FirstInFirstOut is an eviction policy that causes cache entries to be evicted in the same order that they are
// created.
//
// For instance, creating a Cache with a Cache.MaxSize of 3 and creating the entries 1, 2 and 3 in that order would
// put 3 at the head and 1 at the tail:
// 3 (head) -> 2 -> 1 (tail)
// If the cache entry 1 was then accessed, unlike with LeastRecentlyUsed, nothing would change:
// 3 (head) -> 2 -> 1 (tail)
// If a cache entry 4 was then created, because the Cache.MaxSize is 3, the tail (1) would then be evicted:
// 4 (head) -> 3 -> 2 (tail)
FirstInFirstOut EvictionPolicy = "FirstInFirstOut"
)

15
statistics.go Normal file
View file

@ -0,0 +1,15 @@
package gocache
type Statistics struct {
// EvictedKeys is the number of keys that were evicted
EvictedKeys uint64
// ExpiredKeys is the number of keys that were automatically deleted as a result of expiring
ExpiredKeys uint64
// Hits is the number of cache hits
Hits uint64
// Misses is the number of cache misses
Misses uint64
}