915ab1f4ba
configure yystackdepth
2025-03-22 14:00:36 +04:00
3ad67a8ecf
fix: only write to OUT_DIR
2025-03-17 09:11:08 -03:00
da54c1bf97
fix: try fixing copy, again
2025-03-17 09:11:08 -03:00
5e20bd300c
fix: use copy_with_cp
everywhere
2025-03-17 09:11:08 -03:00
7a941c6746
fix: add --no-preserve
for nix on macos
2025-03-17 09:11:08 -03:00
8408283b37
feat: add support for windows cross-compilation
2025-03-17 09:11:08 -03:00
c74c8eef99
refactor+fix: trying to fix sqlite3mc cross compilation
2025-03-17 09:11:08 -03:00
d09ed8230d
fix: cross compilation from aarch64 to x86_64 MacOS
2025-03-17 09:11:08 -03:00
e14762daf5
add libsql-disable-checkpoint-downgrade cargo feature in libsql-ffi crate
2025-02-27 13:08:34 +04:00
a8c11db984
fix: add sqlite3 includes
2025-01-28 13:01:16 -03:00
d3a156caf5
bundle SQLean extensions
...
A common complain with libSQL is how to run extensions. The main
mechanism, with a .so, has a lot of issues around how those .so are
distributed.
The most common extensions are the ones in the sqlean package. We can
improve this experience by bundling them in our sqlite build.
Not all SQLean extensions are kosher: some of them, like fileio, use
the vfs. Others, are deemed too complex.
The extensions included here are a subset that we deem important enough,
and low risk enough, to just be a part of the main bundle.
2025-01-16 22:25:16 -05:00
13385969d1
libsql-ffi: Fix build on macOS
...
The `cp` command does not support `--no-preserve` on macOS.
2024-12-04 17:44:43 +02:00
38cfab7d5d
Revert "Revert "Merge pull request #1791 from awakecoding/libsql-ffi-build-cp-fix""
...
This reverts commit 9499c3c40a
. The real
problem is not Windows, but build on macOS. Let's revert this revert and
fix macOS instead.
2024-12-04 17:44:22 +02:00
9499c3c40a
Revert "Merge pull request #1791 from awakecoding/libsql-ffi-build-cp-fix"
...
This reverts commit 5ee4723699
, reversing
changes made to cc1d9c5afd
because it breaks `libsql-js` build on macOS:
```
penberg@vonneumann libsql-js % cargo check
Updating git repository `https://github.com/tursodatabase/libsql/ `
Compiling libsql-ffi v0.5.0 (https://github.com/tursodatabase/libsql/?rev=9c73c68f0b034c5e01a51d9dc93a61936a71c801#9c73c68f )
Compiling libsql-sqlite3-parser v0.13.0 (https://github.com/tursodatabase/libsql/?rev=9c73c68f0b034c5e01a51d9dc93a61936a71c801#9c73c68f )
Checking reqwest v0.12.9
Checking libsql-hrana v0.2.0 (https://github.com/tursodatabase/libsql/?rev=9c73c68f0b034c5e01a51d9dc93a61936a71c801#9c73c68f )
error: failed to run custom build command for `libsql-ffi v0.5.0 (https://github.com/tursodatabase/libsql/?rev=9c73c68f0b034c5e01a51d9dc93a61936a71c801#9c73c68f )`
Caused by:
process didn't exit successfully: `/Users/penberg/src/tursodatabase/libsql-js/target/debug/build/libsql-ffi-755a07789947d931/build-script-build` (exit status: 101)
--- stdout
cargo:rustc-link-lib=framework=Security
cargo:rerun-if-changed=bundled/src/sqlite3.c
cargo:rerun-if-changed=/Users/penberg/src/tursodatabase/libsql-js/target/debug/build/libsql-ffi-e660048e9fb948a4/out/sqlite3mc/libsqlite3mc_static.a
--- stderr
cp: illegal option -- -
usage: cp [-R [-H | -L | -P]] [-fi | -n] [-aclpSsvXx] source_file target_file
cp [-R [-H | -L | -P]] [-fi | -n] [-aclpSsvXx] source_file ... target_directory
thread 'main' panicked at /Users/penberg/.cargo/git/checkouts/libsql-311658d335deb3b1/9c73c68/libsql-ffi/build.rs:43:44:
called `Result::unwrap()` on an `Err` value: Custom { kind: Other, error: "Failed to copy using cp" }
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
warning: build failed, waiting for other jobs to finish...
```
2024-11-15 10:55:30 +02:00
6a7a3e50a1
fix libsql-ffi build.rs usage of cp command on Windows
2024-10-22 16:15:29 -04:00
e83b13b1d2
libsql: fix musl builds failing due to fcntl64
...
This fixes a build issue with musl based systems that do not have
fcntl64 in their libc. This is done by setting `FILE_OFFSET_BITS=32`
only for musl targets. This forces the compiler to not convert `fcntl`
to `fcntl64`.
2024-09-09 13:40:44 -04:00
e349a06961
Make libsql-ffi's build more friendly to sandboxed environments. ( #1478 )
...
fix: prevent out_path from inheriting the mode of source bindgen.rs
2024-06-18 12:55:36 +00:00
cb89097a57
ffi, build: Don't use UNC paths on Windows ( #1451 )
...
Windows build fails with "The network name cannot be found" error when
built with encryption. That's because the std::fs::canocalize() function
returns UNC paths on Windows, which confuses CMake to fail.
Switch to std::env::current_dir() instead, which returns paths that
CMake can deal with.
2024-06-07 18:02:29 +00:00
17a687b0bb
Fix cross build ( #1453 )
...
* Revert "libsql-ffi: Make CC and CXX configurable (#1257 )"
This reverts commit 897ae8beac
, because
it's bogus and completely ignores the environment variables in a cross
environment.
* ffi: Fix cross builds
Let's use the `TARGET` environment variable to determine what the
build target is to fix cross build falures in libsql-js.git:
```
error: linking with `aarch64-linux-gnu-gcc` failed: exit status: 1
|
= note: LC_ALL="C" PATH="/rust/lib/rustlib/x86_64-unknown-linux-gnu/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/rust/bin" VSLANG="1033" "aarch64-linux-gnu-gcc" "-Wl,--version-script=/tmp/rustcualN23/list" "-Wl,--no-undefined-version" "/tmp/rustcualN23/symbols.o" "/target/aarch64-unknown-linux-gnu/release/deps/libsql_js.libsql_js.daf3718404b8afc6-cgu.0.rcgu.o" "-Wl,--as-needed" "-L" "/target/aarch64-unknown-linux-gnu/release/deps" "-L" "/target/release/deps" "-L" "/target/aarch64-unknown-linux-gnu/release/build/ring-87b9427ec69caec7/out" "-L" "/target/aarch64-unknown-linux-gnu/release/build/libsql-ffi-f019579845159c8e/out" "-L" "/rust/lib/rustlib/aarch64-unknown-linux-gnu/lib" "-Wl,-Bstatic" "/tmp/rustcualN23/libring-d3d989c5f3051615.rlib" "/tmp/rustcualN23/liblibsql_ffi-54e0e58458c35432.rlib" "/rust/lib/rustlib/aarch64-unknown-linux-gnu/lib/libcompiler_builtins-6043da066acc1466.rlib" "-Wl,-Bdynamic" "-ldl" "-lgcc_s" "-lutil" "-lrt" "-lpthread" "-lm" "-ldl" "-lc" "-Wl,--eh-frame-hdr" "-Wl,-z,noexecstack" "-L" "/rust/lib/rustlib/aarch64-unknown-linux-gnu/lib" "-o" "/target/aarch64-unknown-linux-gnu/release/deps/liblibsql_js.so" "-Wl,--gc-sections" "-shared" "-Wl,-z,relro,-z,now" "-Wl,-O1" "-Wl,--strip-all" "-nodefaultlibs"
= note: /usr/lib/gcc-cross/aarch64-linux-gnu/5/../../../../aarch64-linux-gnu/bin/ld: /tmp/rustcualN23/liblibsql_ffi-54e0e58458c35432.rlib(sqlite3mc.c.o): Relocations in generic ELF (EM: 62)
/usr/lib/gcc-cross/aarch64-linux-gnu/5/../../../../aarch64-linux-gnu/bin/ld: /tmp/rustcualN23/liblibsql_ffi-54e0e58458c35432.rlib(sqlite3mc.c.o): Relocations in generic ELF (EM: 62)
/usr/lib/gcc-cross/aarch64-linux-gnu/5/../../../../aarch64-linux-gnu/bin/ld: /tmp/rustcualN23/liblibsql_ffi-54e0e58458c35432.rlib(sqlite3mc.c.o): Relocations in generic ELF (EM: 62)
/usr/lib/gcc-cross/aarch64-linux-gnu/5/../../../../aarch64-linux-gnu/bin/ld: /tmp/rustcualN23/liblibsql_ffi-54e0e58458c35432.rlib(sqlite3mc.c.o): Relocations in generic ELF (EM: 62)
/usr/lib/gcc-cross/aarch64-linux-gnu/5/../../../../aarch64-linux-gnu/bin/ld: /tmp/rustcualN23/liblibsql_ffi-54e0e58458c35432.rlib(sqlite3mc.c.o): Relocations in generic ELF (EM: 62)
/usr/lib/gcc-cross/aarch64-linux-gnu/5/../../../../aarch64-linux-gnu/bin/ld: /tmp/rustcualN23/liblibsql_ffi-54e0e58458c35432.rlib(sqlite3mc.c.o): Relocations in generic ELF (EM: 62)
/usr/lib/gcc-cross/aarch64-linux-gnu/5/../../../../aarch64-linux-gnu/bin/ld: /tmp/rustcualN23/liblibsql_ffi-54e0e58458c35432.rlib(sqlite3mc.c.o): Relocations in generic ELF (EM: 62)
/usr/lib/gcc-cross/aarch64-linux-gnu/5/../../../../aarch64-linux-gnu/bin/ld: /tmp/rustcualN23/liblibsql_ffi-54e0e58458c35432.rlib(sqlite3mc.c.o): Relocations in generic ELF (EM: 62)
/usr/lib/gcc-cross/aarch64-linux-gnu/5/../../../../aarch64-linux-gnu/bin/ld: /tmp/rustcualN23/liblibsql_ffi-54e0e58458c35432.rlib(sqlite3mc.c.o): Relocations in generic ELF (EM: 62)
/tmp/rustcualN23/liblibsql_ffi-54e0e58458c35432.rlib: error adding symbols: File in wrong format
collect2: error: ld returned 1 exit status
```
2024-06-07 16:39:39 +00:00
5129bdb2a3
libsq wal integration ( #1427 )
...
* integrate libsql-wal
* fix wal bug
* shutdown registry on shutdown
* fmt
2024-06-03 09:30:40 +00:00
897ae8beac
libsql-ffi: Make CC and CXX configurable ( #1257 )
...
Make CC and CXX configurable for the build process. Allows us to, for
example, make sure we're building with just clang on macOS.
2024-04-02 14:40:30 +00:00
c45e72d0a6
libsql-ffi: Fix sqlite3mc build output directory ( #1234 )
...
Don't build inside the source tree because `cargo clean` won't clean up
after it...
2024-03-25 16:45:26 +00:00
234e5d4871
Fixed function make_amalgamation
naming typo ( #1168 )
...
Use `amalgamation` instead of `amalgation`.
Ref: https://www.merriam-webster.com/dictionary/amalgamation
2024-03-12 10:27:42 +00:00
47fef88f19
ffi: blocklist wasm fns from bindgen ( #1166 )
2024-03-07 20:13:39 +00:00
fd6afb784f
libsql-ffi: Disable stack protector for sqlite3mc on musl ( #1157 )
...
The stack protector links to symbols like `strcpy_chk`, which are not
available on musl. Therefore, disable it.
Fixes #1155
2024-03-07 11:59:26 +00:00
0cb88082ef
Fix build ( #1101 )
...
Signed-off-by: Piotr Jastrzebski <piotr@chiselstrike.com >
2024-02-28 14:08:03 +00:00
8fd841d87d
libsq-ffi: Drop git submodule trickery from sqlite3mc build ( #1031 )
...
As Lucio points out, we no longer do submodules for sqlite3mc so let's
just drop the bogus `git submodule update` invocation.
2024-02-16 15:31:52 +00:00
86172b548e
Fix sqlite3mc build ( #1024 )
...
* libsql-ffi: Enable PIC for sqlite3mc build
I am seeing the following build error when building the libsql-ffi crate
on Linux/x86-64:
```
/usr/bin/ld:
/tmp/rustcYVhHYo/liblibsql_ffi-20adf4ca11d8e922.rlib(sqlite3mc.c.o):
relocation R_X86_64_32S against `.rodata' can not be used when making a
shared object; recompile with -fPIC
```
Let's turn on PIC for sqlite3mc to fix the problem.
* libsql-ffi: Fix sqlite3mc build under `cross`
We need to use the C compiler configured by `cross` with sqlite3mc;
otherwise we end up building with the host compiler, resulting in errors
like this:
```
= note: /usr/lib/gcc-cross/aarch64-linux-gnu/5/../../../../aarch64-linux-gnu/bin/ld: /tmp/rustcJ70WRo/liblibsql_ffi-5d7e6e8037dbda37.rlib(sqlite3mc.c.o): Relocations in generic ELF (EM: 62)
```
* libsql-ffi: Rewrite build_libsqlite3mc.sh in Rust
The bash shell script is turning out to be painful to run on Windows so
let's just rewrite the logic in Rust and place it in `build.rs`.
2024-02-14 15:26:16 +00:00
02b9e82945
chore: don't rebuild ffi without ciphers ( #1022 )
2024-02-13 16:31:11 +00:00
37fb629f50
run clippy ( #971 )
2024-01-31 08:42:20 +00:00
f05f349c06
chore: fix ffi build with all features ( #950 )
...
* chore: fix ffi build with all features
* set flags only for clang
* Update libsql-ffi/build.rs
2024-01-25 21:03:11 +00:00
58eda02bb3
chore: fix ffi build script again ( #942 )
...
* chore: fix ffi build script again
* ffi: separate building and copying for cipher code
* build dylib if it doesn't exist
* fix wasm builds with ciphers
2024-01-24 22:02:51 +00:00
975646e034
libsql: add local sync write delegation ( #940 )
...
* libsql: add local sync write delegation
* add additional query check to local sync test
2024-01-24 20:39:04 +00:00
dd0e23c81c
fix docker build
2024-01-24 11:03:03 +01:00
d9a9f44ecd
libsql-ffi: add c11 flag to make build compatible with old compilers
2024-01-22 21:43:07 -03:00
ad5ff8d171
libsql-ffi: deduplicate sqlite3.c amalgamations
...
From now on, SQLite3MultipleCiphers will use the amalgamation
file kept in libsql-ffi/bundled/src.
2024-01-19 08:22:57 +01:00
d149ecebd8
libsql-ffi: compile-in Wasm conditionally with encryption enabled
2024-01-19 08:22:57 +01:00
dedbdd03b3
Merge pull request #890 from tursodatabase/sqlite3mc_part0
...
Encryption at rest preparations: integrate with SQLite3MultipleCiphers
2024-01-11 18:23:01 +00:00
f0929037d1
chore: fix ffi build script
2024-01-10 09:53:19 -05:00
c504f8311b
libsql-ffi: move multiple-ciphers support to a feature
...
Enabled with `cargo build -F multiple-ciphers`
2024-01-09 18:12:02 +01:00
c74fb49f5c
still hacky, but compiles lib instead of shipping precompiled
2024-01-09 18:12:02 +01:00
253a295865
encryption demo: based on precompiled libsqlite3mc.so
...
** DEEP, DEEP DRAFT **
The prebuilt library is from my fork:
https://github.com/psarna/SQLite3MultipleCiphers
The key is hardcoded to "heyhey".
After you run sqld with this patch, all data is encoded on disk
with a "heyhey" key. You can't read it directly from the file,
unless you use sqlite3mc's shell and start with
> PRAGMA KEY=heyhey;
, and then it gets properly decrypted.
** TODO **
1. We need to adopt SQLite3MultipleCiphers source code and integrate
with our build system, if we want to use it.
2. Pretty sure the hardcoded "heyhey" passphrase won't pass SOC2,
but I need to consult that with a lawyer.
2024-01-09 18:12:02 +01:00
30f998cac9
sqld: small debug additions and ignore clean up
2023-12-11 15:39:49 -05:00
425b370fb1
fix ffi build.rs
2023-12-08 17:57:34 +01:00
176d6c4376
add wasmtime-bindings feature to libsql-sys
2023-11-30 11:24:51 +01:00
477af9bef6
introduce libsql-ffi
2023-11-15 11:08:51 +01:00