Kelvin Mwinuka
93cfb9caeb
Renames Server struct and utils.Server interface to EchoVault and utils.EchoVault respectively
2024-03-25 16:23:16 +08:00
Kelvin Mwinuka
aea3e0675c
Updated snapshot engine to user options pattern. Updated SetValue and SetExpiry functions in snapshot and aof engine to SetKeyData function that handles setting both value and expiry of a key upon restore.
2024-03-25 11:29:49 +08:00
Kelvin Mwinuka
4bc1f44e1d
Added Apache 2.0 license header to all go files in src folder
2024-03-24 18:30:17 +08:00
Kelvin Mwinuka
dbfa398543
Implemented tests for SUBSCRIBE, PSUBSCRIBE, UNSUBSCRIBE, and PUNSUBSCRIBE command handlers
2024-03-17 02:41:49 +08:00
Kelvin Mwinuka
52646d1564
Take expiry into account when creating snashot and AOF preamble. If the key is already expired when snapshot is taken, it will not be persisted. If the key is expired when loading a snapshot/preamble, it will not be restored.
2024-03-12 21:35:39 +08:00
Kelvin Mwinuka
f27a0dda79
Implemented eviction algorithm that samples a configurable number of keys with TTLs (default 20) at a configurable interval (default 100ms) and deletes the keys that are expired.
2024-03-12 02:12:55 +08:00
Kelvin Mwinuka
52b39d5b0f
Re-enabled cluster nodes in docker-compose.yml.
...
Created flow to forward key deletion command from non-leader node to leader node.
Created flow for propagating key deletion accross the entire cluster to maintain consistency of the deleted keys.
2024-03-11 20:39:57 +08:00
Kelvin Mwinuka
c611dd60ee
Created flow for deleting key consistently within cluster.
2024-03-11 05:22:38 +08:00
Kelvin Mwinuka
8000cef72d
Added RWMutex to keysWithExpiry in order to improve thread safety
2024-03-11 02:10:23 +08:00
Kelvin Mwinuka
c414da16b4
KeyExists keyspace function now removes the key if the key expiry is in the past.
...
KeyData type moved to utils package to allow sharing between multiple packages.
Updated all commands and command tests to pass context object to KeyExists, KeyLock, keyUnlock, KeyRLock, and KeyRUnlock.
Create context object for each test in all test suites instead of just passing context.Background() to all functions that accept a context.
2024-03-10 23:19:05 +08:00
Kelvin Mwinuka
74c08a5b22
Created KeyData type to hold key data value and expireAt in the same map object. Created scaffolding for PERSIST, EXPIRETIME, PEXPIRETIME, TTL, PTTL, EXPIRE, PEXPIRE, EXPIREAT and PEXPIREAT commands
2024-03-09 00:18:58 +08:00
Kelvin Mwinuka
0b90c9343e
Added DEL command for deleting keys from the store
2024-03-08 22:49:03 +08:00
Kelvin Mwinuka
0b9fbb0132
Added mutexes for lfu and lru caches on server struct
2024-03-08 02:21:26 +08:00
Kelvin Mwinuka
82be1f6068
Added addedTime to EntryLFU object to control which entry will be removed if access count is the same. If two entries have the same access count, the older entry should be removed first.
...
SetKeyExpiry and GetValue keyspace receiver functions now require context object to be passed.
Created adjustMemoryUsage function for key eviction for LFU, LRU, and Random eviction policies.
Updated all modules to pass context to SetKeyExpirty and GetValue functions.
2024-03-08 00:26:49 +08:00
Kelvin Mwinuka
118e155144
Created IsMaxMemoryExceeded utility function that is used to check whether the server has exceeded the max memory.
...
Check whether max memory is exceeded before setting value or creating new key.
Return error from SetValue function when max memory is exceeded. This change has been propagated to the aof and snapshot engines as well.
Snapshot engine now only accepts SetValue option. No longer does it neet to use CreateKeyAndLock & KeyUnlock functions itself.
2024-03-06 15:20:45 +08:00
Kelvin Clement Mwinuka
5c347cb7de
Changed max-memory to uint64. Added TODO commends on how to handle memory checking for key eviction.
2024-03-05 22:45:05 +08:00
Kelvin Clement Mwinuka
28f97656c4
Removed etc and get modules and replaced them with generic module. Implemented functions to set and remove the expiry of a key. Implemented LRU and LFU caches using heap.
2024-03-03 16:21:12 +08:00
Kelvin Clement Mwinuka
97b41da729
Moved aof and snapshot packages from server folder to src folder. Created new package for evictions
2024-02-29 16:06:52 +08:00
Kelvin Clement Mwinuka
e18257e600
Added PUBSUB commands and made pubsub module more compatible with redis-cli client
2024-02-28 10:14:46 +08:00
Kelvin Clement Mwinuka
fc8d301525
Removed Consumer Group in PubSub module and made the module more compatible with redis client
2024-02-27 17:45:20 +08:00
Kelvin Clement Mwinuka
088e4c25f0
Made GET, MGET, SET, MSET, SETNX commands compatible with redis client
2024-02-27 15:41:22 +08:00
Kelvin Clement Mwinuka
b32292d8be
Implemented command that returns all the commands in the server
2024-02-26 05:31:38 +08:00
Kelvin Clement Mwinuka
2bdfc2e15c
Implemented options pattern for AOF engine
2024-02-25 16:15:27 +08:00
Kelvin Clement Mwinuka
83f070986d
Removed restore context parameters in AOFEngine
2024-02-25 15:19:16 +08:00
Kelvin Clement Mwinuka
8d7fe0225a
Created AppendReadWriter and PreambleReadWriter interfaces for file handling in the log and preamble packages respectively.
...
Seek to the beginning of the preamble file after truncating.
2024-02-25 15:16:27 +08:00
Kelvin Clement Mwinuka
ab366c4f73
Craeted AppendStore and PreambleStore engines
2024-02-25 11:39:51 +08:00
Kelvin Clement Mwinuka
ff2d579102
Added '\r\n\r\n' delimiter to server.go error returned to the client
2024-02-14 15:59:17 +08:00
Kelvin Clement Mwinuka
b1af2c907e
Updated server.go file to remove import from acl and pubsub packages. Use ACL and PubSub interfaces instead. Created NewServer function that returns and instance on server
2024-02-14 12:03:08 +08:00
Kelvin Clement Mwinuka
53ade871ad
Created ACL and PubSub interfaces to be used in Server struct definition to avoid future circular imports in acl and pubsub packages
2024-02-13 11:43:13 +08:00
Kelvin Clement Mwinuka
d44c76db4e
Server log differentiates between starting TLS and mTLS server. Removed TLS minimum version in tls config.
2024-02-11 16:37:58 +08:00
Kelvin Clement Mwinuka
69c9170f56
Implemented mTLS listener to verify client certificates.
2024-02-03 06:40:03 +08:00
Kelvin Clement Mwinuka
f9ae87862c
Implemented multiple server cert/key pairs. Implemented mTLS for verifying client using multiple pem files.
2024-02-03 00:24:59 +08:00
Kelvin Clement Mwinuka
019fb39b8b
Extracted command handler function for reuse in both the handleConnection function and the RestoreAOF function in the AOF engine.
2024-02-01 23:52:39 +08:00
Kelvin Clement Mwinuka
15d289c33d
Added config flags to choose between aof restore and snapshot restore in standalone mode.
2024-02-01 04:42:44 +08:00
Kelvin Clement Mwinuka
04ddd46d2b
Start goroutine in AOF engine to log queued commands
2024-02-01 04:08:34 +08:00
Kelvin Clement Mwinuka
ce0eabf865
GetState receiver function now checks wether there's an active copy or state mutation before proceeding with the copy.
...
Handling of write commands is delayed until state copy is complete.
2024-01-31 01:55:51 +08:00
Kelvin Clement Mwinuka
25b2cb7154
Implemented RewriteLog functionality and defined handler for REWRITE AOF command.
2024-01-30 23:33:24 +08:00
Kelvin Clement Mwinuka
dff80940cd
Changed bgsave command to save and bgrewriteaof to rewriteaof because these functions will only be ran concurrently and never on the main thread.
...
Implented AOF command logging in AOFEngine which creates a new log file if one does not exist and append to the file if one is found.
2024-01-30 21:16:37 +08:00
Kelvin Clement Mwinuka
a081971427
Implemented restore function in standalone snapshot engine
2024-01-29 00:22:47 +08:00
Kelvin Clement Mwinuka
d31acfbbdd
Updated Raft snapshot to also persist last saved unix timestamp in milliseconds. This timestamp is re-loaded when the server is re-started.
...
Created standalone snapshot flow including manifest file that is used to determine whether another snapshot should be taken based on the hashed contents of the snapshot.
2024-01-28 03:36:09 +08:00
Kelvin Clement Mwinuka
7e59de20a2
Implement checking wether snapshot is in progress before continuing
2024-01-27 01:46:16 +08:00
Kelvin Clement Mwinuka
3c5d6fef2c
Created basic handling for taking snapshots. Will implemented error when snapshot in progress and propagation of snapshot command to the rest of the cluster
2024-01-26 00:58:43 +08:00
Kelvin Clement Mwinuka
4e12064ce8
Implemented snapshot persistence on raft layer
2024-01-25 17:52:22 +08:00
Kelvin Clement Mwinuka
1c00706f88
Updated module name to echovault
2024-01-24 22:32:05 +08:00
Kelvin Clement Mwinuka
7e55fd469b
Code formatting: declare writer and reader on the same line
2024-01-23 23:58:09 +08:00
Kelvin Clement Mwinuka
4b980cc5fd
Remvoved response skipping for +ack message from client.
2024-01-23 23:54:39 +08:00
Kelvin Clement Mwinuka
ce14f59f41
Replaced all instances of utils.Contains with slices.Contains.
...
Deleted Contains function from the utils.go file.
Replaced bufio.ReadWriter with io.Reader and io.Writer respectively.
Updated PubSub module to remove posibility of not passing a channel name when subscribing or publishing.
2024-01-23 23:38:07 +08:00
Kelvin Clement Mwinuka
9017f12fa9
Removed http options flag as it will not be used for now.
...
Moved server cluster receiver functions into cluster.go.
Moved keyspace receiver functions into keyspace.go.
Moved module and command loading into modules.go.
Updated Dockerfile and docker-compose to remove http flag to server.
2024-01-22 17:38:44 +08:00
Kelvin Clement Mwinuka
78aa9a82c6
Renamed memebrlist_layer directory to memberlist.
...
Renamed raft_layer directory to raft.
Created snapshot and aof packages to handle snapshots in standalone mode and aof package to handle append-only function.
Created server package that will contain the server logic.
2024-01-21 15:03:36 +08:00
