It is important to verify that the data we got from Foundation DB matches with what we
want, since we are doing a range query.
for example, say we searched for ('db_name42', 10, 20). If this page does not exist, then
it could match with ('db_name41', 10, 20) or with ('db_name42', 9, 20) since it does
lexicographic search.
* Add basic tonic error handling
Added a simple error to handle `WriteConflict`s
* Improve the way we handle Foundation DB keys
Foundation DB keys support tuples. Earlier we used plain strings. For
example to store page 5, which had frame no 8, we did `ns/f/5/8`.
However this breaks lexicographic sorting. For e.g. we want `ns/f/5/8`
to be smaller than `ns/f/5/18`, but since we store as a plain string
the comparison breaks.
So we let FDB handle encoding with the `pack` API which encodes the
key properly as tuple `(ns, f, 5, 8)`
* Send `max_frame_no` while doing inserts and find frame
Whenever a txn is started, we keep the current `max_frame_no` in the
transaction's state. For the subsequent find and insert requests, we
send this value to storage server.
For a read txn, it should not read any frames greater than it's
max_frame_no. For a write txn, if the max_frame_no has changed
meanwhile, then txn should be aborted.
* Allow multiple writers at the storage server
This patch has two important changes:
- Storage server now uses `max_frame_no` from the txn to read frames
It will make sure to not read any frames beyond the `max_frame_no`
of the transaction. This fixes the bug in isolation level.
- Storage server now checks `max_frame_no` before inserting. If it
has changed meanwhile, then it rejects the insertion request.
This lets us have multiple (non concurrent) writers or even multiple
logical primary instances.
* minor cargo fmt fixes
* Update proto to send `max_frame_no` while inserting frames
We plan to allow multiple (non concurrent) writers. If a transaction's
`max_frame_no` does not match the server's `max_frame_no`, then it
has missed some new writes and we can abort the transaction
* Updates storage trait's insert_frames definition
- Change parameter type `FrameData` to `Frame` from rpc def, to
avoid unnecessary copying
- Take `max_frame_no` param
* Use `insert_frames` method
* cleanup: remove `insert_frame` method and impl
* Remove `FrameData`, use `Frame` from proto instead
* Add storage server RPC client
* Address review comments
- Use `#[tracing::instrument]` wherever appropriate
- Remove `Mutex` for `client` and `clone` it wherever required
- Avoid reading env variable inside the lib, rather take a config object
- Make the private methods of `DurableWal` async
- Don't try to create a runtime, instead assume it always exists. Let the caller create it
- Update proto:
- remove `max_frame_no` from `InsertFramesRequest`
- make `page_no` to `u32` in the proto
- Update storage server to have `page_no` as `u32`
* Add build and version files
* Add `FrameStore` trait
This trait provides the methods which can be implemented by different
storage backends like Redis, Foundation DB etc.
* Add a gRPC server which implements Storage Server proto
* Add storage server runner with pluggable storage backends
* Add an InMemory FrameStore
* Address review comments
- Remove unnecessary Arc/Mutex in the Service struct
- Chnage all `&self mut` to `&self` in the `FrameStore` trait, handle
locking internally
* Set cargo publish to false
