0
0
mirror of https://github.com/tursodatabase/libsql.git synced 2025-06-03 14:15:44 +00:00
Commit Graph

30449 Commits

Author SHA1 Message Date
c611b3a63a libsql: Force AES256CBC for embedded replica's 2024-03-04 12:02:02 -05:00
1826582133 Embedded replicas encryption test ()
* Embedded replicas encryption test

Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>

* Fix encryption embedded replica test

* ci: add encryption tests

* add clean cipher build dir

---------

Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>
Co-authored-by: Lucio Franco <luciofranco14@gmail.com>
2024-03-04 16:09:02 +00:00
0d483ad479 Go binding: Change default encryption key ()
* Go binding: Change default encryption key

Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>

* Go bindings: Drop sqlite dependency

Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>

---------

Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>
2024-03-04 14:55:16 +00:00
2b06b6b7d8 schema migration ()
happy path schema migration
2024-03-04 13:15:29 +00:00
9dbf604843 libsql: prefix client_wal_index file ()
* libsql: prefix client_wal_index file

* remove create_dir_all
2024-03-04 12:43:39 +00:00
aa2490cd36 Bottomless savepoint ()
* bottomless: added ability to wait for uploaded frames without shutting down

* bottomless: savepoint tracker

* expose bottomless backup savepoint through PrimaryDatabase API

* tests for detached progression updates in bottomless progression tracker

* bottomless: reset savepoint tracker on new generation

* bottomless: expose savepoint tracker on PrimaryDatabase
2024-03-04 09:51:48 +00:00
7f9e289681 Aggregated stats for queries with most elapsed_time ()
* Aggregated stats for queries with most elapsed_time

* Add query percentile support to stats ()

* Record query latency percentiles with hdrhistogram

* Expose queries percentiles through stats response

* Simplify QueriesStats transformation to QueriesStatsResponse

* Reset query stats at the beginning of every hour ()

* Add expires_at to control when QueriesStats should be reset

* Add query count and elapsed sum to query stats response

Also group all latency aggregations together under the elapsed key

* Remove option sprawl on QueriesStats fields

By making the whole struct optional where it is used

* Add created_at to stats queries responde object

* Set queries stats to none when stats is created

This will make the API response the same when no queries
have been recorded on the queries stats struct. This can
happen right after:
- the stats struct initialization
- the stats queries object expiration

Example of stats API response when no queries have been
recorded:

{
  ...
  "queries": null
}

* Use if else instead of early return from stats to response
2024-03-01 21:43:50 +00:00
612aae4277 server: trim quotes in namespace string in attach statement ()
* Add a regression test when UUIDs are used in attach
statements

Also, added a test to check if the UUID is wrapped
in quotes

* bugfix: trim namespace string if it is wrapped in quotes

the client might send namespace wrapped in quotes in case of uuids,
so we will just trim the string.
2024-03-01 15:08:01 +00:00
169ef7fa11 Factorize program execution in VM object ()
factorize program Execution in Vm
2024-03-01 10:44:24 +00:00
f46d157fbc Tag docker builds with short sha () 2024-03-01 09:27:21 +00:00
adf36104ad fix bad attach name ()
* add test

* fix auth using attach alias instead of target
2024-02-29 14:15:43 +00:00
213c7dabf5 Go: Add support for bool type ()
Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>
2024-02-29 15:08:13 +01:00
db894cb68a Refactor Go API ()
* Go: refactor API

Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>

* Go: add read your writes test

Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>

---------

Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>
2024-02-29 12:14:42 +00:00
4b2732c5e2 Go: Reset connection when it's returned to a pool ()
* Make all methods of Conn trait async

Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>

* C bindings: Add libsql_reset

Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>

---------

Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>
2024-02-29 10:09:07 +00:00
be417ace83 register schema migrations ()
* introduce schema migration scheduler

* introduce schema database

* register schema migration job in meta store

* fix meta update error not being reported

* add meta store tests

* make program serializable

* export PrimaryConnectionMaker type

* add schema database variant

* handle migration error

* initialize migration scheduler

* instantiate schema database

* fmt

* add test snapshots

* fix tests

* fmt
2024-02-29 09:10:55 +00:00
c41e17a998 Encryption in Go SDK ()
* Go tests: Run less tests in parallel

Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>

* Go bindings: Read your writes

Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>

* Go bindings: encryption

Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>

* Don't panic when encryption fails

Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>

* Go bindings: Encryption test

Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>

---------

Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>
2024-02-29 07:55:49 +00:00
2ee5a4726e sys: feature flag pager creator ()
* sys: feature flag pager creator

* fix feature flag pghdr_creator
2024-02-29 00:07:17 +00:00
49db7d57b7 Make sure the C code doesn't segfault ()
* make sure pager is valid

We have had a segfault issue recently because the pager passed down
to the c code was null. That happened because libsql-server had
encryption disabled, while libsql had encryption enabled.

To prevent issues like that in the future, the C code should be checking
if the arguments it needs are valid.

Because this function returned a pointer, we now change it so that it
can return an integer, with the pointer as a return argument.

* update generated code

We have generated code in-tree. We should either remove it, or keep it
in sync. For now, just keeping it in sync.

This patch is just the result of running tests, and seeing what changes.
2024-02-28 20:05:50 +00:00
6db9996199 libsql: read_your_writes to true by default () 2024-02-28 19:40:59 +00:00
16c6e40335 fix auth malformed token ()
* add tests

* allow legacy tokens with no perm claims

* fmt
2024-02-28 14:16:27 +00:00
0cb88082ef Fix build ()
Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>
2024-02-28 14:08:03 +00:00
2593ee9585 fix docker-entry point http basic auth () 2024-02-27 21:03:21 +00:00
de91aa2450 remove stray backtrace () 2024-02-27 20:54:30 +00:00
f938b84e62 tools: add rebuild-log tool ()
* tools: add rebuild-log tool

* fix unused mut

* mark rebuild-log as publish false

* dedup pages
2024-02-27 20:31:47 +00:00
4cc113e190 ci: fix merge job docker metadata () 2024-02-27 18:01:20 +00:00
350a70cf2e ci: fix multi-arch docker builds ()
* chore: fix multi-arch docker builds

* fix perms

* fix metadata

* rename publish-server2 -> publish-server

* use buildjet

* add test branch check

* remove test branch
2024-02-27 14:54:11 +00:00
9aa0a15754 Update README.md ()
minor fix
2024-02-27 15:54:57 +02:00
c63fe26117 Allow multiple namespaces types to be stored in the same NamespaceStore ()
* introduce `Database` enum

stores different kinds of databases int the same type, and multiplex the
connection type the same way.

* move NamespaceName to own module

* move NamespaceStore to own module

- Namespace store is not generic over the Namespace anymore
- create method takes a config
- simplify loading namespaces

* remove NamespaceMaker, make generic NamespaceConfig

* unify Namespace kinds

Namespace is not generic on db type anymore, but used the multiplexed
database types instead.

* fix forking

* add `map` method to MakeConnection

allows mapping the connection type, when wrapping it in database/mod.rs

* move connection time tracking to the proxy module

where it's being used

* remove generics

* use NamespaceName fro share_schema_name

* simplify server kind configuration.

* remove leftover debug
2024-02-27 13:20:37 +00:00
29749f2506 ci: fix duplicated rust suites test () 2024-02-27 12:37:41 +00:00
e7a512ba1d server: disable always on http auth for docker () 2024-02-26 21:14:24 +00:00
9478edc51f server: fix stats report out of bounds () 2024-02-26 21:14:20 +00:00
ece9ebfcd1 chore: prepare v0.23 release, update dist to 0.11.1 () 2024-02-26 16:20:36 +00:00
52d8b8f1c4 Multi-scopes auth & Attach fix ()
* test snapshots

* allow multi-scope auth

* hors-sujet: fix C compile warning

* update jwt and add hashbrown feature

* replace Auth with RequestContext

* extract RequestContext

* fix auth check for multi-scopes

* pass NamespaceName in StmtKind::Attach

* fallback to global auth if key is not set in ns config

* add allow_attach to namespace creation

* return meta store from namespace store

* test attach

* fmt

* review edits
libsql-server-v0.23.0
2024-02-26 13:56:20 +00:00
82cd274805 Shared schema ()
* Rename SharedSchemaError to SharedSchemaCreationError

Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>

* Improve NamespaceStore::exists

Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>

* Metastore: Implement shared schema links

Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>

---------

Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com>
2024-02-26 07:30:50 +00:00
e60a4e33f8 Updated libsql_extensions.md fix broken link ()
Fixed broken link to `wal.h`
2024-02-23 05:15:27 +00:00
ae8a36a9dd server: reduce size of slow read/write log ()
* server: reduce size of slow read/write log

* bump limit to 512
2024-02-22 21:40:10 +00:00
bb8058b5a9 server: move auth log to debug ()
server: move auth log to trace
2024-02-22 16:35:27 +00:00
9de3ccc815 auth strategy2 ()
* Introduce UserAuthStrategy to allow third party authentication implementation

Co-authored-by: Jeremy Rowe <jeremy.rowe@shopify.com>

* PR feedback

* fix broken import

* fmt

---------

Co-authored-by: James Newton <hello@jamesnewton.com>
Co-authored-by: Jeremy Rowe <jeremy.rowe@shopify.com>
Co-authored-by: Jeremy W. Rowe <jeremywrowe@users.noreply.github.com>
2024-02-22 12:08:03 +00:00
807b45c88a Fix configuration not being synced with replicas () 2024-02-22 09:57:10 +00:00
b3c6863c26 Implemented todos in Hrana Statement::params related methods ()
implemented todos in Hrana Statement::params related methods
2024-02-22 08:22:23 +00:00
e1eaf3b74a libsql: fix replica example () 2024-02-21 22:09:24 +00:00
a046bac2ed chore: move arm docker images to its own tag () 2024-02-21 20:07:42 +00:00
08b27626c0 libsql: add http_request_callback for replicas ()
* libsql: add `http_request_callback` for replicas

* fix feature usage

* feature gate http request callback

* remove arc from import
2024-02-21 17:41:54 +00:00
52619b4deb fix bottomless bugs () 2024-02-21 14:33:06 +00:00
97efe98469 Build libsql-server images on PR ()
Build images on every PR
2024-02-21 14:00:15 +00:00
6a6b5dfc70 chore: remove filesystem clean c bindings ()
* chore: remove filesystem clean c bindings

* add env var in actions file
2024-02-20 20:10:39 +00:00
b9440fb38a chore: fix examples link () 2024-02-20 19:59:03 +00:00
8ef5868cc6 chore: remove amd64 from arm image build () 2024-02-20 19:38:15 +00:00
b769c6c557 Enable parsing ATTACH statement for any db ()
* Add tests for parsing `ATTACH` statements

* Enable parsing `ATTACH` statement for any db

Earlier, it allowed attaching same db.
2024-02-20 18:45:35 +00:00
169f0271af recovery panic fix ()
* add missing config default

* check dbs dir exist before restoring
2024-02-20 17:33:48 +00:00