0
0
mirror of https://github.com/tursodatabase/libsql.git synced 2025-06-15 12:53:06 +00:00
Commit Graph

150 Commits

Author SHA1 Message Date
4c3f5fc7fa allow reindex statement in libsql () 2024-07-24 20:30:25 +00:00
376a89d25e test: add hrana intp_stream test ()
* test: add hrana intp_stream test

* fix test
2024-07-23 22:08:08 +00:00
35d3fb6702 server: clean up auth tests () 2024-07-22 15:53:02 +00:00
3e219998ce tests: gen jwt token for auth test () 2024-07-22 15:07:47 +00:00
de6dc81a1e server: fix auth and add ws/http/grpc auth tests ()
* server: fix auth and add ws/http/grpc auth tests

* fix grpc parser
2024-07-19 18:43:17 +00:00
a566230b8f Fix sqld test ()
* consume stream completely in order to avoid test deadlock

- reproduced pretty stabily with rr record --chaos
- slack thread: https://turso.slack.com/archives/C044XT6KK46/p1721393397388469

* moved nextest config to the default location and fix toml
2024-07-19 17:17:11 +00:00
f80d113adf test: enable txn timeout test () 2024-07-17 19:37:24 +00:00
a69b6772c2 server: add replica frame counter test check () 2024-07-10 10:03:10 +00:00
ebf9e0f504 fix bad autocommit ()
* add test

* fix bug when updating autocommit status in write proxy

* fmt
2024-07-09 18:06:57 +00:00
8ec7db15e6 libsql: add for remote conn ()
* libsql: add  for remote conn

* feature flag new/new_skip_last

* clean up rows usage

* add allow unused for BatchRows::empty
2024-06-19 15:36:45 +00:00
5d6e743403 libsql: add stream test () 2024-06-07 13:57:35 +00:00
7668db8be4 server: fix hrana error classification ()
* server: fix hrana error classification

* add new test snapshots

* more snapshots
2024-06-05 16:34:12 +00:00
e451079c18 Allow single transaction in schema migration ()
* Return also Program from Connection::run

Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>

* Extract LibSqlConnection::execute

Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>

* Use LibSqlConnection::execute in write_proxy

Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>

* Remove Clone from Program

Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>

* Remove Arc from Program

Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>

* Allow a single transaction in schema migration

Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>

* fix rustc-hash dep issue

Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>

---------

Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>
2024-05-28 15:05:45 +00:00
0c730c439a Propagate schema migration errors correctly ()
* Update hecrj/setup-rust-action to v2

Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>

* Update rust-toolchain.toml to 1.78

Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>

* Propagate schema migration errors correctly

Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>

---------

Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>
2024-05-21 11:53:42 +00:00
7708a43772 server: add query_duration_ms stats to query res ()
* server: add `query_duration_ms` stats to query res

* hrana: default new stats fields
2024-04-15 16:37:13 +00:00
505b21f536 Fix sync ()
Old implementation could never finish if the write is faster than
time needed to do two round-trips to primary.

For example it's enough to have a write every 200ms to make sync
never finish if it's done in embedded replica in Sydney
if the primary and the writer are in the US.

Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>
2024-04-15 14:31:15 +00:00
dac888bbb7 server: add query stats to hrana responses ()
* server: add query stats to hrana responses

* update spec with stats

* remove unused import

* fix snapshot

* more snapshots

* add stats to legacy api

* address comments
2024-04-11 16:23:21 +00:00
958f5613cc libsql: fix embedded replica multiple txn ()
This commit fixes an issue where new connections for embedded replica's
with write delegation would not create a new "connection" on the server
causing weird issues with transactions and state. The fix here is to
create a new `client_id` when creating a new writer for a new connection
locally. This forces the server to use separate connections to execute
the writes.

Closes 
2024-04-05 20:44:31 +00:00
724f6db925 libsql: fix malformed db and add test ()
* add malformed db test

* continue to debug output sql

* reduce sql

* reset connection on injection

* remove println in test

* address comments
2024-04-04 18:20:17 +00:00
00080c5983 server: add embedded replica stats test for replica () 2024-04-01 20:22:31 +00:00
4ecf2a5ecd server: fix conflict on attach update ()
* server: fix conflict on attach update

* add tests and add on conflict to other branch

* only error on server errors

* remove test query

* add more on conflicts fixes

* Revert "add more on conflicts fixes"

This reverts commit 9cef2d320e.

* remove replace for meta store inserts
2024-03-20 12:41:05 +00:00
1fc0550a65 harden attach ()
* pass namespace path resolver to VM instead of hacky dir walking

* test attach forbidden in migration

* fmt
2024-03-14 15:31:38 +00:00
a2077c54ab Fix namespace deletion ()
* fix schema deletion

* optional ns delete payload
2024-03-14 11:35:20 +00:00
0d5dc18ce1 rename migration tables ()
* rename __libsql_migration_tasks to sqlite3_libsql_tasks

* rename job tables
2024-03-13 11:03:55 +00:00
b9ab41c485 ignore legacy auth when new auth is present () 2024-03-13 06:30:53 +00:00
c79cec3ee2 move migration api to user api () 2024-03-12 20:09:07 +00:00
a95bdf4d35 job status notifier & wait for dry run success () 2024-03-12 19:55:41 +00:00
153c8382d8 remove broken test () 2024-03-12 18:00:30 +00:00
d945d41566 Hrana: don't close the stream at the end of prepared statement execution ()
* Hrana: don't close the stream at the end of prepared statement execution

* hrana stream and transaction: check for tokio runtime existence on drop

* standalone::execute_transaction test: raise simulation duration

* hrana: added comments to conditions for stream closing

* hrana: added comments to conditions for stream closing
2024-03-12 11:57:07 +00:00
fe2ebffe28 check schema migration auth () 2024-03-11 17:09:43 +00:00
5ece28f4fe gid tag token claim () 2024-03-11 16:14:52 +00:00
1528128fce allow ddl override () 2024-03-11 15:00:26 +00:00
998d330428 extend test timeout () 2024-03-09 11:39:39 +00:00
226fba9a12 Add tests for data migration in schema dbs ()
- Added a test which performs data migration including
  inserts, updates and deletes
- Added a test which shows that schema migration fails
  if there is any conflicting data in the linked dbs
2024-03-08 16:11:26 +00:00
ae708f8952 Fork a new DB when creating from shared schema ()
* Fork a new DB when creating from shared schema

* Use the correct db type to get the wal logger

* Add a test for a new db in shared schema

- When a new db is created with a shared schema, it should have
  the same schema and the schema version
- When a new migration is done, the new db and all the existing
  dbs should have the same schema and version

* Move schema db fork logic inside namespace mod

* clean up and remove unnecessary store call
2024-03-08 14:32:22 +00:00
c9ba69245e Handle dry run failure, and abort migration ()
* test snasphots

* add test

* handle dry run failure, and abort all tasks

* fmt
2024-03-08 11:01:32 +00:00
73909accf6 Update tests to check if all schema versions are same () 2024-03-08 08:58:27 +00:00
405f605705 libsql: prevent database overwrite misuse ()
Closes 
2024-03-07 11:59:46 +00:00
bf19f3e1d7 reject migration containing migration statements () 2024-03-06 18:12:45 +00:00
75321d56d3 use custom query builder to collect migration error ()
use custom query builder to collect migration error
2024-03-06 15:31:46 +00:00
8f897d510e Block writes during schema migration ()
* Add block_writes to Connection

Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>

* Add block_writes to PrimaryDatabase

Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>

* Use block_writes in Connection::run

Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>

* Unblock writes once migration is finished

Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>

* Block writes on startup if there are pending migration tasks

Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>

* Block writes when a migration task is enqueued

Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>

* twix

---------

Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>
Co-authored-by: ad hoc <postma.marin@protonmail.com>
2024-03-05 22:17:36 +00:00
4e0ce557a8 libsql: Rename periodic_sync to sync_interval ()
Rename the configuration option for periodic sync to `sync_interval` to
align with what we're doing with the rest of the SDKs.
2024-03-05 19:05:46 +00:00
bb1dd03748 Migrations status admin api ()
* added an endpoint to ask for a general migration status

* return detailed information about individual migration job progress

* integrated migrations API in tests

* post-rebase

* adjusted code after rebase
2024-03-05 16:20:30 +00:00
6b13d77519 libsql: Deasyncify some Conn trait methods ()
There's no reason to have `is_autocommit()`, `changes()` or
`last_insert_row()` methods async because they're all fully local and
cannot block. Deasyncify them.
2024-03-05 13:44:53 +00:00
50648a6b4b prevent task enqueue when pending task ()
check for already enqueued job for schema + migrate schema db ops to
schema module.
2024-03-05 10:31:06 +00:00
1826582133 Embedded replicas encryption test ()
* Embedded replicas encryption test

Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>

* Fix encryption embedded replica test

* ci: add encryption tests

* add clean cipher build dir

---------

Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>
Co-authored-by: Lucio Franco <luciofranco14@gmail.com>
2024-03-04 16:09:02 +00:00
2b06b6b7d8 schema migration ()
happy path schema migration
2024-03-04 13:15:29 +00:00
612aae4277 server: trim quotes in namespace string in attach statement ()
* Add a regression test when UUIDs are used in attach
statements

Also, added a test to check if the UUID is wrapped
in quotes

* bugfix: trim namespace string if it is wrapped in quotes

the client might send namespace wrapped in quotes in case of uuids,
so we will just trim the string.
2024-03-01 15:08:01 +00:00
adf36104ad fix bad attach name ()
* add test

* fix auth using attach alias instead of target
2024-02-29 14:15:43 +00:00
4b2732c5e2 Go: Reset connection when it's returned to a pool ()
* Make all methods of Conn trait async

Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>

* C bindings: Add libsql_reset

Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>

---------

Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>
2024-02-29 10:09:07 +00:00