Skip to main content

​
Supported Repository Types

Data in kamu is shared via . There are multiple types of repositories that differ by the kinds of services they provide. The most basic repository allows you to upload (“push”) and download (“pull”) data, while more advanced can allow searching datasets, query data in-place, or subscribing for updates.
TypeDescriptionCapabilitiesURL Examples
Local FSA basic repository that uses the local file system folder. Mainly used for examples and testing.pull
push		file:///home/me/example/repository
file:///c:/Users/me/example/repository
HTTP(s)A basic repository that provides read-only access to data.pullhttps://example.org/dataset
S3A basic repository that stores data in Amazon S3 bucket. Can be used with any S3-compatible storage APIpull
push		s3://bucket.my-company.example
s3+http://my-minio-server:9000/bucket
s3+https://my-minio-server:9000/bucket
IPFSUses IPFS HTTP Gateway for reading. Push is only possible via IPNS (see details).pull
push		ipfs://bafy...vqhe
ipns://k51q...v7mn
ODFODF-native repositories types that support very fast data transfer and querying data in-placepull
push
query
search		odf+http://odf.acme.com
odf+https://kamu.dev

​
Push / Pull Aliases

Starting out, you can always use explicit URLs when syncing data from and to repositories. If some remote source contains a dataset you’re interested in you can download it using the pull command: 
kamu pull https://example.org/dataset --as example-dataset
If you have a dataset you would like to share, or made some changes to a dataset you are sharing with your friends - you can upload the new data using the push command: 
kamu push example-dataset --to s3://my-bucket.example.org/example-dataset
When you run these commands, kamu will automatically create “pull” and “push aliases” for this dataset: 
kamu repo alias list example-dataset

┌─────────────────┬──────┬────────────────────────────────────────────┐
│     Dataset     │ Kind │                Alias                       │
├─────────────────┼──────┼────────────────────────────────────────────┤
│ example-dataset │ Pull │ https://example.org/dataset                │
│ example-dataset │ Push │ s3://my-bucket.example.org/example-dataset │
└─────────────────┴──────┴────────────────────────────────────────────┘
With aliases in place, next time you want to pull or push data you can simply do: 
kamu pull example-dataset  # Will pull from https://example.org/dataset
kamu push example-dataset  # Will push to s3://my-bucket.example.org/example-dataset
Both pull and push commands will analyze the state of the dataset in the and at the repository and will only upload data and metadata that wasn’t previously seen (a minimal update). These commands are also “safe”. They will detect all types of concurrent changes and history divergence and prevent you from overwriting someone else’s changes.
When you pull a Root dataset from a remote source running kamu pull on it will attempt to sync it from the repository, and NOT execute a polling or other ingest action. In kamu list datasets with pull aliases will be showing up as Remote(...).

​
Repositories

If you store multiple datasets side by side in some locations you can add it as a repository. Repositories are configured per using the kamu repo command group. To add new repo use: 
kamu repo add my-s3-bucket s3://my-bucket.example.org/
This repo will now be visible in kamu repo list. To pull/push a dataset from/to this repo you can now use remote references like <repo-name>/<dataset-name>: 
kamu pull my-s3-bucket/example-dataset
kamu push example-dataset --to my-s3-bucket

​
Searching for Data

To look for datasets in the repos use: 
kamu search covid19

# Searching with empty query lists (a limited number of) datasets in all repos
kamu search

# You can specify the search term and specific repo to look in
kamu search covid19 --repo my-s3-bucket
Search is delegated to the repository implementations and its capabilities depend on the type of the repo. Whereas smart repos may support advanced full-text search, simple storage-only repos may be limited to a substring search by dataset name.
Last modified on March 16, 2026