0
0
mirror of https://github.com/tursodatabase/libsql.git synced 2025-05-17 14:46:55 +00:00
Commit Graph

30599 Commits

Author SHA1 Message Date
8565938a72 segment list 2024-04-25 13:41:13 +02:00
f5b411524c silence warnings 2024-04-25 13:32:22 +02:00
00663a22b6 add missing deps 2024-04-25 13:32:21 +02:00
6d1aadfdf1 sealed log 2024-04-25 13:32:21 +02:00
4ed6abe722 segment module, header structs on helper functions 2024-04-25 13:32:20 +02:00
72501da8c1 add error module 2024-04-25 13:32:08 +02:00
89b92785d6 add libsql-wal crate 2024-04-25 13:31:52 +02:00
40ba51a28d libsql wal init ()
* add libsql-wal crate

* add filesystem abstraction

* add libsql-wal to workspace
2024-04-25 10:31:52 +00:00
777012123d Stop holding lock during query execution ()
Holding a lock during query execution can lead to a deadlock if
the transaction holding it is blocked by another transaction and
that other transaction can't proceed because it can't get the lock
that is held by the first transaction.

Fixes 

Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>
2024-04-24 13:03:44 +00:00
d0a0d8a414 fix link to client libraries () 2024-04-23 15:21:36 +00:00
8f96340eb0 server: parallelize namespace shutdown () 2024-04-23 06:35:51 +00:00
ca8834c053 Namespace passing in http header ()
* wip - passing userauthcontext instead of http headers

* revert comment out

* fixed temp change

* Merge branch 'main' into jw/changing-how-jwt-is-passed-around

* fixed failing tests

* next iteration of unit test fixing

* fixed remaining unit tests

* reverted debug

* removed accidentally added file

* moved str to userauthcontext conversion to from trait

* remove vague comment

* cargo fmt

* cleaned up mod reimportss

* refactored cryptic matching in replica_proxy

* marked potentially duplicate code with // todo dupe #auth

* refactored context to custome errors

* added a factory to produce empty UserAuthContext

* added constructors for UserAuthContext

* switched from try_into to using constructors

* cargo fmt

* added tests for failing cases in parsers

* adding mamespace as param wip

* cargo fmt

* added test for non-asci error

* incremental changes to make namespace as param work

* fixed failing test

* fixed log message

* removed unnecessary error mapping

* turned context to result

* removing dummy tokens from tests

* cargo fmt + cleanup

* namespace passing exammple

* added namespace config for the example

* remove unnecessary dummy token

* reverting accidental commit

* deduplicated handling of hrana hello and repreated hello. Code is fully equivalent to the previous form. No change in logic.

* fixed early return

* lazy unwrapping

* made session fields private again

* fmt

* cleaned up nesting in conn

* fmt

* refactored ns selection to be more expressive

* narrowed the type constraints for namespace arg

* restructured namespace_from_headers

* rearranged the logic of selecting namespace passing method

* fmt

* simplified type

* further refinement of namespace selection logic

* reverted unnecessary change

* reverted unnecessary change

* fmt

* added documentation

* Fix simple queries for schema databases ()

* server: release v0.24.5 ()

* libsql: fix embedded replica multiple txn ()

This commit fixes an issue where new connections for embedded replica's
with write delegation would not create a new "connection" on the server
causing weird issues with transactions and state. The fix here is to
create a new `client_id` when creating a new writer for a new connection
locally. This forces the server to use separate connections to execute
the writes.

Closes 

* libsql: prepare v0.3.3 release ()

* Go updates ()

* Go bindings: Improve imports

Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>

* Go bindings: continue Next() after a time.Time

Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>

* Go bindings: remove unused err check

Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>

---------

Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>

* Remove in-tree version of go-libsql ()

* Go bindings: build only sql-experimental for CI

Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>

* Go bindings: Run go-libsql tests in CI

Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>

* Go bindings: Remove in-tree version of go-libsql

go-libsql is maintained in https://github.com/tursodatabase/go-libsql

Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>

* Go bindings: Run more tests in CI

Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>

---------

Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>

* don't regen log on sqld version mismatch ()

* add debug info to release build ()

* Add 'aarch64-unknown-linux-gnu' to dist targets ()

* bump sqld v0.24.6 ()

* Bump arm builder machine ()

Builds of arm images are hanging/crashing

* Revert "add debug info to release build ()" ()

This reverts commit 342e4c368b.

* bottomless: upgrade s3 sdk to 1.0 ()

* bottomless: upgrade s3 sdk to 1.0

* fix behavior version

* libsql: fix embedded replica example ()

* server: add query stats to hrana responses ()

* server: add query stats to hrana responses

* update spec with stats

* remove unused import

* fix snapshot

* more snapshots

* add stats to legacy api

* address comments

* update tonic to 0.11 ()

* Add workflow_dispatch event trigger for PR images ()

* Fix sync ()

Old implementation could never finish if the write is faster than
time needed to do two round-trips to primary.

For example it's enough to have a write every 200ms to make sync
never finish if it's done in embedded replica in Sydney
if the primary and the writer are in the US.

Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>

* server: add `query_duration_ms` stats to query res ()

* server: add `query_duration_ms` stats to query res

* hrana: default new stats fields

* libsql: fix unwraps in local replicas ()

* server: release v0.24.7 ()

* cargo lock update

* Revert "cargo lock update"

This reverts commit a86b89d8bc.

---------

Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>
Co-authored-by: Julian <julian@Julians-MacBook-Pro.local>
Co-authored-by: Athos <athos@turso.tech>
Co-authored-by: Lucio Franco <luciofranco14@gmail.com>
Co-authored-by: Piotr Jastrzębski <haaawk@gmail.com>
Co-authored-by: ad hoc <postma.marin@protonmail.com>
Co-authored-by: Pekka Enberg <penberg@iki.fi>
Co-authored-by: Avinash Sajjanshetty <640792+avinassh@users.noreply.github.com>
2024-04-22 18:47:26 +00:00
237b0f959d Update DOCKER.md ()
Change libsql-server:main to libsql-server:latest as latest is more up to date and main wasn't updated for a long time
2024-04-22 18:25:11 +00:00
f26baff7b4 use user query string for queries rather than generated () 2024-04-19 17:26:30 +00:00
60df94b2cd Prepare libsql crate release ()
Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>
2024-04-18 15:34:14 +00:00
3368a2a204 Merge pull request from tursodatabase/embedded_replicas_logging
Log errors and long operations in remote_client
2024-04-18 15:56:19 +02:00
8037153d4c Log long operations in remote_client
Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>
2024-04-18 15:05:26 +02:00
b849a34542 Add error logging to remote_client
Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>
2024-04-18 08:14:36 +02:00
85775d4595 fix unused blocking rt () 2024-04-17 15:35:27 +00:00
49e28c6911 Prepare release ()
Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>
2024-04-17 15:00:11 +00:00
1869f70f5c Release libsql-sys ()
Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>
2024-04-17 14:15:45 +00:00
d9182c5946 Preparing a rust sdk release ()
Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>
2024-04-17 13:27:30 +00:00
65a8bfb6ab Prefetch frames during handshake ()
* Prefetch frames in handshake

Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>

* Remove unwraps from replication client creation

Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>

* remove secondary client

---------

Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>
Co-authored-by: ad hoc <postma.marin@protonmail.com>
2024-04-17 09:02:34 +00:00
20f56ad4a2 server: release v0.24.7 () libsql-server-v0.24.7 2024-04-16 18:27:37 +00:00
d24435e564 libsql: fix unwraps in local replicas () 2024-04-15 17:38:28 +00:00
7708a43772 server: add query_duration_ms stats to query res ()
* server: add `query_duration_ms` stats to query res

* hrana: default new stats fields
2024-04-15 16:37:13 +00:00
505b21f536 Fix sync ()
Old implementation could never finish if the write is faster than
time needed to do two round-trips to primary.

For example it's enough to have a write every 200ms to make sync
never finish if it's done in embedded replica in Sydney
if the primary and the writer are in the US.

Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>
2024-04-15 14:31:15 +00:00
7badc6d35d Add workflow_dispatch event trigger for PR images () 2024-04-12 06:16:34 +00:00
15fe600c40 update tonic to 0.11 () 2024-04-11 17:54:36 +00:00
dac888bbb7 server: add query stats to hrana responses ()
* server: add query stats to hrana responses

* update spec with stats

* remove unused import

* fix snapshot

* more snapshots

* add stats to legacy api

* address comments
2024-04-11 16:23:21 +00:00
4534dd9901 libsql: fix embedded replica example () 2024-04-11 13:20:29 +00:00
67eae4c461 bottomless: upgrade s3 sdk to 1.0 ()
* bottomless: upgrade s3 sdk to 1.0

* fix behavior version
2024-04-10 23:13:15 +00:00
4771fa7134 Revert "add debug info to release build ()" ()
This reverts commit 342e4c368b.
2024-04-10 22:12:18 +00:00
218bdc72c3 Bump arm builder machine ()
Builds of arm images are hanging/crashing
2024-04-10 21:09:24 +00:00
38da8f7d8c bump sqld v0.24.6 () 2024-04-10 10:41:35 +00:00
f3f742fd6c Add 'aarch64-unknown-linux-gnu' to dist targets () 2024-04-10 06:16:04 +00:00
342e4c368b add debug info to release build () 2024-04-09 20:14:54 +00:00
87d0145408 don't regen log on sqld version mismatch () 2024-04-09 17:48:48 +00:00
ba96a2d77b Remove in-tree version of go-libsql ()
* Go bindings: build only sql-experimental for CI

Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>

* Go bindings: Run go-libsql tests in CI

Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>

* Go bindings: Remove in-tree version of go-libsql

go-libsql is maintained in https://github.com/tursodatabase/go-libsql

Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>

* Go bindings: Run more tests in CI

Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>

---------

Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>
2024-04-07 09:24:08 +00:00
f7c5bfb7bd Go updates ()
* Go bindings: Improve imports

Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>

* Go bindings: continue Next() after a time.Time

Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>

* Go bindings: remove unused err check

Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>

---------

Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>
2024-04-06 14:43:48 +00:00
e9d637e051 libsql: prepare v0.3.3 release () libsql-rs-v0.3.3 2024-04-05 20:44:41 +00:00
958f5613cc libsql: fix embedded replica multiple txn ()
This commit fixes an issue where new connections for embedded replica's
with write delegation would not create a new "connection" on the server
causing weird issues with transactions and state. The fix here is to
create a new `client_id` when creating a new writer for a new connection
locally. This forces the server to use separate connections to execute
the writes.

Closes 
2024-04-05 20:44:31 +00:00
fa98db3c04 server: release v0.24.5 () libsql-server-v0.24.5 2024-04-05 20:14:29 +00:00
59b922b294 Fix simple queries for schema databases () 2024-04-05 19:57:06 +00:00
724f6db925 libsql: fix malformed db and add test ()
* add malformed db test

* continue to debug output sql

* reduce sql

* reset connection on injection

* remove println in test

* address comments
2024-04-04 18:20:17 +00:00
897ae8beac libsql-ffi: Make CC and CXX configurable ()
Make CC and CXX configurable for the build process. Allows us to, for
example, make sure we're building with just clang on macOS.
2024-04-02 14:40:30 +00:00
00080c5983 server: add embedded replica stats test for replica () 2024-04-01 20:22:31 +00:00
9bb9cd4753 Add more logging in case snapshots fail ()
We have a user with a failing snapshot, but we don't know why.
We know rename returns EINVAL, which could happen for a variety
of reasons, all of them related to the paths themselves.

By printing the path, we will be able to figure out the actual reason.
Right now we're shooting in the dark.
2024-04-01 16:13:14 +00:00
49ccba7d80 Stop expiring connections in replication hyper client ()
* Stop expiring connections in replication hyper client

Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>

* Remove keep-alive

Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>

---------

Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>
2024-04-01 12:51:11 +00:00
6fc29fcc5c Dedupe Hrana hello and repeated hello ()
* deduplicated handling of hrana hello and repreated hello. Code is fully equivalent to the previous form. No change in logic.

* fixed early return

* lazy unwrapping

* made session fields private again

* fmt

* cleaned up nesting in conn

* fmt
2024-03-30 13:41:20 +00:00