0
0
mirror of https://github.com/tursodatabase/libsql.git synced 2025-06-27 17:19:29 +00:00

61 Commits

Author SHA1 Message Date
62f61dbf14 review edits 2024-01-19 12:16:25 +01:00
e83308c5e5 fix sqlite3 rust tests 2024-01-19 09:33:22 +01:00
ea46e792ca introduce wal checkpoint callback 2024-01-19 09:32:34 +01:00
3922cd9bae libsql-sqlite3: add temporary API for leaking database pager
Used for WAL encryption until we migrate that layer to virtual WAL.
2024-01-19 08:22:56 +01:00
87ebafccc3 Merge pull request from tursodatabase/penberg/wasm-autocommit
libsql-sqlite3: Export more API functions in Wasm
2024-01-16 10:39:54 +00:00
12cab8ae52 libsql-sqlite3: Export sqlite3_last_insert_rowid() in Wasm 2024-01-15 12:59:13 +02:00
566434c339 libsql-sqlite3: Export sqlite3_get_autocommit() in Wasm
We need the `sqlite3_get_autocomit()` helper in the TypeScript SDK, for
example. The fact that it's not exported seems to be just an omission so
let's add it.
2024-01-11 11:32:40 +02:00
441b283551 rename getMissingFrame to ignoreFrameIfNewerExist 2024-01-09 16:06:15 +01:00
26789ec455 review edits 2024-01-09 14:25:42 +01:00
f932572382 sqlite3 wal reverse iterator 2024-01-09 12:31:53 +01:00
461f5731b2 Merge pull request from vlcn-io/ci
add a ci task to build cr-sqlite binaries against libsql
2024-01-08 15:48:36 +00:00
450e9053b9 main,pager,pragma: forward-declare custom libsql functions 2024-01-03 13:00:12 +01:00
379b227587 pager,wal: first batch of hooks to integrate encryption
Based on https://gist.github.com/psarna/0dbd95424fb7121bb54bf5f961e1e117
from SQLite3MultipleCiphers, let's try and make libsql flexible enough
to not require patching the amalgamation files at all in order to integrate
with SQLite3Multiple ciphers.

For starters, their codec functions are instead declared as libsql_pager_codec
and libsql_pager_has_codec. The idea for future integration is that
if an appropriate compilation flag is on, we assume that the code will be compiled
with additional symbols that provide all the implementation we need, e.g.
libsql_pager_codec_impl() will just transiently call sqlite3mcPagerCodec().
2024-01-03 13:00:12 +01:00
52bb6cc706 libsql-sqlite3: add hooks for vfs, pragmas and uri
If the following directives are defined:
 - LIBSQL_PRE_VFS_HOOK
 - LIBSQL_EXTRA_PRAGMAS
 - LIBSQL_EXTRA_URI_PARAMS
, a few new symbols need to be compiled in, respectively:
 - libsql_pre_vfs_hook(const char *)
 - libsql_extra_pragma(sqlite3 *, const char *, void *)
 - libsql_handle_extra_uri_params(sqlite3 *, const char *),
   libsql_handle_extra_attach_params(sqlite3 *, const char *,
     const char *, sqlite3_value *, char **)

Those hooks, combined with virtual WAL, should be plenty enough to
integrate SQLite3MultipleCiphers without patching sqlite3.c code.
2024-01-03 13:00:12 +01:00
2f599e2621 libsql-sqlite3: restore sqlite3_filename_* API semantics
Those are really weird internally, and assume that the db path string
you pass as an argument resides in a very specific place in memory,
preceded by 4 zero bytes, and followed by journal and wal names.
The WALv2 rework broke this magic assumption, so we hereby restore it.

As follow-up, we should move this filename management to virtual WAL,
because with virtual WAL implementations you sometimes don't even have
a notion of "WAL file path".
2023-12-22 11:21:03 +01:00
682d42783d fix missing define for cases where we build a loadable rather than static libsql extension 2023-12-21 15:08:39 -05:00
f42bd234c6 pull in latest cr-sqlite changes 2023-12-21 15:05:06 -05:00
5a9c14fc26 libsql-sqlite3: initialize libsqlApi properly
... previously we somehow missed libsql_close_hook
2023-12-20 12:07:01 +01:00
e21cefe907 Merge pull request from vlcn-io/update-libsql
update cr-sqlite to v0.16.2
2023-12-15 17:00:49 +00:00
0e025264b2 fix missing commit frame on local_client inject from snapshot 2023-12-13 13:18:07 +01:00
7a3723663b tweaks to get compiling again after libsql updates 2023-12-12 09:58:03 -05:00
28bb1bc621 update cr-sqlite to v0.16.2 2023-12-12 09:46:49 -05:00
ae9daa8b74 Use NonZeroU32 for frame_no and page_no 2023-12-05 09:20:44 +01:00
4ae365b557 fix review nits 2023-11-30 11:26:49 +01:00
49b11abce0 homogenise rust wal naming 2023-11-30 11:26:49 +01:00
3d7c3c0a5f static initialization of sqlite3wal rc 2023-11-30 11:26:49 +01:00
05f6d47643 make pager's wal a pointer 2023-11-30 11:26:44 +01:00
b426b236b0 restore deprecated libsql open methods, introduce v3 2023-11-30 11:25:22 +01:00
2125e633b9 static sqlite3 wal 2023-11-30 11:25:22 +01:00
ff53bcbc57 rename libsql_create_wal to libsql_wal_manager 2023-11-30 11:25:22 +01:00
100c52cbb5 fix rust tests 2023-11-30 11:24:52 +01:00
217ecb84b5 sqlite3 virtual wal v2
Refactor of the virtual wal API. Drop global wals in favor of passing
the wal implementation as an argument to libsql_open.

The WAL interface is split in two sets of virtual methods:
- `create_wal` is passed when opening a sqlite connection. It's role is
  to instantiate a wal.
- `libql_wal` is the wal itself, created by the `create_wal`.

The sqlite3_wal and `sqlite3_create_wal` implementation are completely
decoupled from the wal implementation. They are the default
implementation when using the traditional open methods, and
sqlite3_create_wal is exposed as a global variable, and can be composed
with other wal implementations.
2023-11-30 11:24:47 +01:00
a56ee2f615 ext/wasm: fix SQLTester path
It was moved incorrectly due to the migration of the C API
to libsql-sqlite3/ subdirectory.
2023-11-29 16:53:59 +01:00
6b3430c10d windows,sqliteInt: skip SEH
We're not committing to SEH support due to virtual WAL
incompatibility reasons.
2023-11-29 15:17:40 +01:00
d89f0b4d8c wasi: add virtual WAL stub
On top of a special VFS, `make wasi` now also compiles-in
a specialized virtual WAL methods module. It doesn't do anything
right now (just falls back to the regular implementation),
but can be used to call back to the host, inject hooks, etc.
2023-11-29 09:27:24 +01:00
822b2e1b81 Merge pull request from psarna/extwasmwal
ext/wasm: allow WAL mode
2023-11-28 08:37:48 +00:00
dfa4ce43c0 ext/wasm: allow WAL mode
All the groundwork was there for WASI anyway.
2023-11-28 09:26:28 +01:00
1445e42483 wasi: add poor man's microbenchmark 2023-11-28 09:00:49 +01:00
a5cd4f3d81 wasi: refactor to libsql-wasi crate
The crate still has a main.rs file for demo purposes,
but the logic is extracted to a library.
2023-11-27 09:49:33 +01:00
3b0d30f7af wasi,demo: add creating a table 2023-11-26 14:19:50 +01:00
52bf745b32 apply clippy 2023-11-26 14:17:16 +01:00
7cb1eee19c wasi: implement the rest of VFS 2023-11-26 14:17:16 +01:00
d2c3b1eadc wasi: add close() 2023-11-26 14:17:16 +01:00
0bbf971c61 wasi: add write support 2023-11-26 14:17:16 +01:00
9974e2c745 wasi: implement stub for reading files 2023-11-26 14:17:16 +01:00
6113da3db9 wasi: ensure exclusive locking mode and WAL journaling 2023-11-26 14:17:16 +01:00
15a6e43c3c wasi: add WASI demo stub
It's far from working, but it already properly calls back
from a WebAssembly instance to the host for VFS operations.
We also want the same for virtual WAL and we'll have a solid foundation
for running libSQL in a Wasm runtime.
2023-11-26 14:17:16 +01:00
b1bbbf05c7 wasi: add vfs.c to the build path 2023-11-26 14:17:16 +01:00
dc75074a37 wasi: add WASI VFS stub 2023-11-26 14:17:16 +01:00
cc9bcb1e15 wasm: add make wasi command
It is not directly tested or usable yet, and depends on having
wasi-libc library available at /usr/share/wasi-sysroot.

Still, it compiled, so it's a nice start.

`make wasi` produces `libsql.wasm` module.
Future work includes using this module to run a WebAssembly app
in a runtime that supports WASI, using `libsql.wasm` as the database
implementation.

The patch is based on splitting SQLITE_OMIT_WAL into SQLITE_OMIT_SHARED_MEM,
so that we can still compile WAL mode, just without shared memory,
so still usable with exclusive mode or via libSQL's virtual WAL.
2023-11-23 19:55:56 +01:00