to_vector / from_vector.
Right now very little validation is done on the type being a blob.
TODO:
* Tests
* Use stable representation for the blob on disk
* make sure pager is valid
We have had a segfault issue recently because the pager passed down
to the c code was null. That happened because libsql-server had
encryption disabled, while libsql had encryption enabled.
To prevent issues like that in the future, the C code should be checking
if the arguments it needs are valid.
Because this function returned a pointer, we now change it so that it
can return an integer, with the pointer as a return argument.
* update generated code
We have generated code in-tree. We should either remove it, or keep it
in sync. For now, just keeping it in sync.
This patch is just the result of running tests, and seeing what changes.
* test snapshots
* allow multi-scope auth
* hors-sujet: fix C compile warning
* update jwt and add hashbrown feature
* replace Auth with RequestContext
* extract RequestContext
* fix auth check for multi-scopes
* pass NamespaceName in StmtKind::Attach
* fallback to global auth if key is not set in ns config
* add allow_attach to namespace creation
* return meta store from namespace store
* test attach
* fmt
* review edits
* Add regression tests for random rowid bugs
Tests added for:
1. VACUUM does not respect RANDOM ROWID - https://github.com/tursodatabase/libsql/issues/929
2. Bulk insert into a fresh table does not respect RANDOM ROWID - https://github.com/tursodatabase/libsql/issues/1046
In both cases, the table should end up random rowids, but they seem to have sequential ones.
* bugfix: pass `regNextRowid` to `OP_NewRowid` call in `xferOptimization`
The register `regNextRowid` contains the `LIBSQL_RANDOM_ROWID_MARKER` value
which tells the VDBE to use a random value for row ids. The method `xferOptimization`
is used in `VACUUM` and bulk insert routines where data from one table is
inserted in another. Since the data is inserted in a loop, it checks for
max rowid once and then keeps inserting it serially. Hence we pass the marker
in each call, so that random id is generated.
* Remove `libsql_module` field from `sqlite3_vtab`
* remove `libsql_module` struct
Also removed the associated functions `libsql_create_module_v2`, `libsql_create_module` functions'.
The `libsql_module` had a function `xPreparedSql` which is now moved to `sqlite_module`. The `sqlite_module` might get changed in the upstream, so added some padding space for our custom functions
* generate ffi bindings
* Add a github workflow to test crsqlite
* fix crsqlite: remove `pLibsqlModule` references
* Add tests for sqlite-vss extension
* wal.c: fix the function pointer type
A 100% benign error, because the function has the correct signature,
but gets rid of a compiler warning.
* regenerate sqlite3.c amalgamation
* libsql-ffi: regenerate SQLite3MultipleCiphers' sqlite3.c
... not needed later, because an existing patch series switches it
to use the sqlite3.c amalgamation file from bundled/src/,
but let's update it for now.
We need the `sqlite3_get_autocomit()` helper in the TypeScript SDK, for
example. The fact that it's not exported seems to be just an omission so
let's add it.
Based on https://gist.github.com/psarna/0dbd95424fb7121bb54bf5f961e1e117
from SQLite3MultipleCiphers, let's try and make libsql flexible enough
to not require patching the amalgamation files at all in order to integrate
with SQLite3Multiple ciphers.
For starters, their codec functions are instead declared as libsql_pager_codec
and libsql_pager_has_codec. The idea for future integration is that
if an appropriate compilation flag is on, we assume that the code will be compiled
with additional symbols that provide all the implementation we need, e.g.
libsql_pager_codec_impl() will just transiently call sqlite3mcPagerCodec().
If the following directives are defined:
- LIBSQL_PRE_VFS_HOOK
- LIBSQL_EXTRA_PRAGMAS
- LIBSQL_EXTRA_URI_PARAMS
, a few new symbols need to be compiled in, respectively:
- libsql_pre_vfs_hook(const char *)
- libsql_extra_pragma(sqlite3 *, const char *, void *)
- libsql_handle_extra_uri_params(sqlite3 *, const char *),
libsql_handle_extra_attach_params(sqlite3 *, const char *,
const char *, sqlite3_value *, char **)
Those hooks, combined with virtual WAL, should be plenty enough to
integrate SQLite3MultipleCiphers without patching sqlite3.c code.
Those are really weird internally, and assume that the db path string
you pass as an argument resides in a very specific place in memory,
preceded by 4 zero bytes, and followed by journal and wal names.
The WALv2 rework broke this magic assumption, so we hereby restore it.
As follow-up, we should move this filename management to virtual WAL,
because with virtual WAL implementations you sometimes don't even have
a notion of "WAL file path".
Refactor of the virtual wal API. Drop global wals in favor of passing
the wal implementation as an argument to libsql_open.
The WAL interface is split in two sets of virtual methods:
- `create_wal` is passed when opening a sqlite connection. It's role is
to instantiate a wal.
- `libql_wal` is the wal itself, created by the `create_wal`.
The sqlite3_wal and `sqlite3_create_wal` implementation are completely
decoupled from the wal implementation. They are the default
implementation when using the traditional open methods, and
sqlite3_create_wal is exposed as a global variable, and can be composed
with other wal implementations.
On top of a special VFS, `make wasi` now also compiles-in
a specialized virtual WAL methods module. It doesn't do anything
right now (just falls back to the regular implementation),
but can be used to call back to the host, inject hooks, etc.
