Developer Guide
Building Locally
Prerequisites:
- Docker or Podman (note: unit tests run with Podman by default)
- If using
docker
- make sure it’s usable withoutsudo
(guidelines) - If using
podman
- make sure it’s setup to run root-less containers (guidelines)
- If using
- Rust toolset
- Install
rustup
- The correct toolchain version will be automatically installed based on the
rust-toolchain
file in the repository
- Install
- Tools used by tests
- Code generation tools (optional - needed if you will be updating schemas)
- Install
flatc
- Install
protoc
followed by:cargo install protoc-gen-prost
- to install prost protobuf plugincargo install protoc-gen-tonic
- to install tonic protobuf plugin
- Install
- Cargo toolbelt (optional - if you will be doing releases)
cargo install cargo-update
- to easily keep your tools up-to-datecargo install cargo-binstall
- to install binaries without compilingcargo binstall cargo-binstall --force
- make future updates to use precompiled versioncargo binstall cargo-edit
- for setting crate versions during releasecargo binstall cargo-update
- for keeping up with major dependency updatescargo binstall cargo-llvm-cov
- for coverage
Clone the repository:
git clone git@github.com:kamu-data/kamu-cli.git
Build it:
cd kamu-cli
cargo build
To use your locally-built kamu
executable link it as so:
ln -s $PWD/target/debug/kamu-cli ~/.local/bin/kamu
When needing to test against a specific official release you can install it under a different alias:
curl -s "https://get.kamu.dev" | KAMU_ALIAS=kamu-release sh
Run Tests with Podman (Recommended)
Prepare test containers once before running tests:
cargo run --bin kamu-cli -- config set --user engine.runtime podman
kamu init --pull-test-images
Then run tests:
cargo test
Run Tests with Docker (Alternative)
Prepare test containers once before running tests:
kamu init --pull-test-images
Then run tests:
KAMU_CONTAINER_RUNTIME_TYPE=docker cargo test
Using Nextest test runner (Optional)
Nextest is a better test runner.
To use it run:
cargo binstall cargo-nextest
cargo nextest run
Build Speed Tweaks (Optional)
Consider configuring Rust to use lld
linker, which is much faster than the default ld
(may improve link times by ~10-20x).
To do so install lld
, then create ~/.cargo/config.toml
file with the following contents:
[build]
rustflags = ["-C", "link-arg=-fuse-ld=lld"]
One more alternative is to use mold
linker, which is also much faster than the default ld
.
To do so install mold
or build it with clang++
compiler from mold sources then create ~/.cargo/config.toml
file with the following contents:
[build]
linker = "clang"
rustflags = ["-C", "link-arg=-fuse-ld=mold"]
Building with Web UI (Optional)
To build the tool with embedded Web UI you will need to clone and build kamu-web-ui repo or use pre-built release. Now build the tool while enabling the optional feature and passing the location of the web root directory:
KAMU_WEB_UI_DIR=`pwd`/../kamu-web-ui/dist/kamu-platform/ cargo build --features kamu-cli/web-ui
Note: KAMU_WEB_UI_DIR
requires absolute path
Note: in debug mode the directory content is not actually being embedded into the executable but accessed from the specified directory.
Code Generation
Many core types in kamu
are generated from schemas and IDLs in the open-data-fabric repository. If your work involves making changes to those - you will need to re-run the code generation tasks using:
make codegen
Make sure you have all related dependencies installed (see above) and that ODF repo is checked out in the same directory as kamu-cli
repo.
Release Procedure
- While on the feature branch, bump the crates versions using
release
tool, e.g.cargo run --bin release -- --major / --minor / --patch
- We try to stay up-to-date with all dependencies, so:
- Run
cargo update
to pull in any minor releases - Run
cargo upgrade --dry-run
and see which packages have major upgrades - either perform them or ticket them up
- Run
- Create a
CHANGELOG
entry for the new version - Create PR, wait for tests, then merge
- Checkout and pull
master
- Tag the latest commit with a new version:
git tag vX.Y.Z
- Push the tag to repo:
git push origin tag vX.Y.Z
- Github Actions will pick up the new tag and create a new GitHub release from it