0
0
mirror of https://github.com/tursodatabase/libsql.git synced 2025-06-13 13:03:03 +00:00

1598 Commits

Author SHA1 Message Date
b3c3fac14c fix ci again 2024-03-15 11:14:02 +01:00
fdcc696131 patch dist arm64 macos build && bump sqld 2024-03-15 11:08:35 +01:00
1196382b70 bump sqld version 0.24.0 2024-03-14 17:44:31 +01:00
1fc0550a65 harden attach ()
* pass namespace path resolver to VM instead of hacky dir walking

* test attach forbidden in migration

* fmt
2024-03-14 15:31:38 +00:00
a2077c54ab Fix namespace deletion ()
* fix schema deletion

* optional ns delete payload
2024-03-14 11:35:20 +00:00
3b46378935 catch parser panic () 2024-03-13 17:56:02 +00:00
965f3dc7d7 per-namespace sentinel () 2024-03-13 11:37:33 +00:00
3de73ef356 Namespace::destroy: add parameter to perform soft delete (preserve backup) ()
* Namespace::destroy: add parameter to perform soft delete (preserve backup)

* make sure that log statement won't panic
2024-03-13 11:14:45 +00:00
0d5dc18ce1 rename migration tables ()
* rename __libsql_migration_tasks to sqlite3_libsql_tasks

* rename job tables
2024-03-13 11:03:55 +00:00
b9ab41c485 ignore legacy auth when new auth is present () 2024-03-13 06:30:53 +00:00
c79cec3ee2 move migration api to user api () 2024-03-12 20:09:07 +00:00
a95bdf4d35 job status notifier & wait for dry run success () 2024-03-12 19:55:41 +00:00
153c8382d8 remove broken test () 2024-03-12 18:00:30 +00:00
d945d41566 Hrana: don't close the stream at the end of prepared statement execution ()
* Hrana: don't close the stream at the end of prepared statement execution

* hrana stream and transaction: check for tokio runtime existence on drop

* standalone::execute_transaction test: raise simulation duration

* hrana: added comments to conditions for stream closing

* hrana: added comments to conditions for stream closing
2024-03-12 11:57:07 +00:00
969560c6d4 libsql-server: s/conccurent/concurrent/ () 2024-03-12 11:01:55 +00:00
e388a72f2d implement missing function on schema db () 2024-03-12 09:47:02 +00:00
5bad81bc11 block migration during db operation ()
* introduce schema lock

* synchronize migration with db creation
2024-03-11 17:25:10 +00:00
fe2ebffe28 check schema migration auth () 2024-03-11 17:09:43 +00:00
5ece28f4fe gid tag token claim () 2024-03-11 16:14:52 +00:00
95cb5b0b5c prevent ns deletion during migration () 2024-03-11 15:42:41 +00:00
1528128fce allow ddl override () 2024-03-11 15:00:26 +00:00
833350dae5 handle scheduler errors ()
* handle scheduler errors

* fix query
2024-03-09 11:40:31 +00:00
998d330428 extend test timeout () 2024-03-09 11:39:39 +00:00
9092268a89 Report stats latencies with microseconds accuracy () 2024-03-08 17:04:26 +00:00
226fba9a12 Add tests for data migration in schema dbs ()
- Added a test which performs data migration including
  inserts, updates and deletes
- Added a test which shows that schema migration fails
  if there is any conflicting data in the linked dbs
2024-03-08 16:11:26 +00:00
ae708f8952 Fork a new DB when creating from shared schema ()
* Fork a new DB when creating from shared schema

* Use the correct db type to get the wal logger

* Add a test for a new db in shared schema

- When a new db is created with a shared schema, it should have
  the same schema and the schema version
- When a new migration is done, the new db and all the existing
  dbs should have the same schema and version

* Move schema db fork logic inside namespace mod

* clean up and remove unnecessary store call
2024-03-08 14:32:22 +00:00
21fa8076e9 fix flaky shared schema test () 2024-03-08 13:05:19 +00:00
c9ba69245e Handle dry run failure, and abort migration ()
* test snasphots

* add test

* handle dry run failure, and abort all tasks

* fmt
2024-03-08 11:01:32 +00:00
b67550ffea fix conccurency bug in test () 2024-03-08 09:16:02 +00:00
73909accf6 Update tests to check if all schema versions are same () 2024-03-08 08:58:27 +00:00
fef7689301 Make sure setup_migration_table does not change schema_version ()
Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>
2024-03-07 17:15:23 +00:00
02bd31c69d Disable forking for a shared schema database () 2024-03-07 14:48:59 +00:00
baecc81789 Add optional flush param when storing in MetaStore ()
Added a new method `store_and_maybe_flush` in MetaStore, which gives you an option to flush the config changes to disk.

The previous `store` method behaviour is not changed. It internally calls `store_and_maybe_flush` method, stores the config and flushes to disk.

This patch fixes a bug in fork. Earlier, we were flushing the target config before actually creating a fork. If the fork failed, MetaStore would have invalid data. Now, we store the config, without flush, create the fork. Iff successful, flush the target config to disk.
2024-03-07 15:21:26 +01:00
9b96dbc349 Cleanup errors ()
Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>
2024-03-07 12:29:53 +00:00
405f605705 libsql: prevent database overwrite misuse ()
Closes 
2024-03-07 11:59:46 +00:00
25bd951ee1 Schema migrations metastore backup ()
backup meta store on state change and schema on update

Co-authored-by: ad hoc <postma.marin@protonmail.com>
2024-03-07 11:35:27 +00:00
0c1c35f8bf Disallow creation of shared schema db with a dump ()
Disallow shared schema db with dump
2024-03-07 10:58:53 +00:00
bf19f3e1d7 reject migration containing migration statements () 2024-03-06 18:12:45 +00:00
266ab4f0e4 adding backup sync await for schema migrations ()
* adding backup sync await for schema migrations

* move backup_sync call after write_block was set

* perform backup in task

* save shared schema db backup once all databases confirm dry run

* added schema job backup sync

* adjusted tests

* ensure that there are no ongoing transactions before migrated db backup

* fixed flaky test

* tweaks

---------

Co-authored-by: ad hoc <postma.marin@protonmail.com>
2024-03-06 17:21:29 +00:00
75321d56d3 use custom query builder to collect migration error ()
use custom query builder to collect migration error
2024-03-06 15:31:46 +00:00
23f8fd2711 prevent namespace creation/update when there is a pending migration on shared schema () 2024-03-05 22:18:22 +00:00
8f897d510e Block writes during schema migration ()
* Add block_writes to Connection

Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>

* Add block_writes to PrimaryDatabase

Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>

* Use block_writes in Connection::run

Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>

* Unblock writes once migration is finished

Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>

* Block writes on startup if there are pending migration tasks

Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>

* Block writes when a migration task is enqueued

Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>

* twix

---------

Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>
Co-authored-by: ad hoc <postma.marin@protonmail.com>
2024-03-05 22:17:36 +00:00
4e0ce557a8 libsql: Rename periodic_sync to sync_interval ()
Rename the configuration option for periodic sync to `sync_interval` to
align with what we're doing with the rest of the SDKs.
2024-03-05 19:05:46 +00:00
2c1b2d0f48 handle job retry ()
do not re-perform task if already performed
2024-03-05 17:41:32 +00:00
bb1dd03748 Migrations status admin api ()
* added an endpoint to ask for a general migration status

* return detailed information about individual migration job progress

* integrated migrations API in tests

* post-rebase

* adjusted code after rebase
2024-03-05 16:20:30 +00:00
6b13d77519 libsql: Deasyncify some Conn trait methods ()
There's no reason to have `is_autocommit()`, `changes()` or
`last_insert_row()` methods async because they're all fully local and
cannot block. Deasyncify them.
2024-03-05 13:44:53 +00:00
50648a6b4b prevent task enqueue when pending task ()
check for already enqueued job for schema + migrate schema db ops to
schema module.
2024-03-05 10:31:06 +00:00
7de9b48110 trigger delete enqueued tasks ()
create a trigger to remove tasks from enqueued task table on update
2024-03-04 16:42:06 +00:00
1826582133 Embedded replicas encryption test ()
* Embedded replicas encryption test

Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>

* Fix encryption embedded replica test

* ci: add encryption tests

* add clean cipher build dir

---------

Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>
Co-authored-by: Lucio Franco <luciofranco14@gmail.com>
2024-03-04 16:09:02 +00:00
2b06b6b7d8 schema migration ()
happy path schema migration
2024-03-04 13:15:29 +00:00