> ## Documentation Index
> Fetch the complete documentation index at: https://docs.kamu.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# Push Sources

export const Diagram = ({src, alt}) => {
  return <div style={{
    display: "flex",
    "flex-direction": "column",
    "align-items": "center"
  }}>
    <img src={src} alt={alt} style={{
    background: "#dddddd",
    "margin-bottom": 0
  }} />
    <span>{alt}</span>
  </div>;
};

export const YouTubeList = ({id}) => {
  const src = `https://www.youtube.com/embed/videoseries?list=${id}`;
  return <iframe className="w-full aspect-video rounded-xl" src={src} allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowFullScreen></iframe>;
};

export const YouTube = ({id, width}) => {
  const src = `https://www.youtube.com/embed/${id}`;
  return <iframe className="w-full aspect-video rounded-xl" src={src} allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowFullScreen width={width}></iframe>;
};

export const Schema = ({t, id}) => {
  const anchor = id ? id : t.toLowerCase().replace(/\s+/g, "-");
  const link = `/odf/schemas#${anchor}`;
  return <a class="schema-object" href={link}>{t}</a>;
};

export const Term = ({t, id}) => {
  const anchor = id ? id : t.toLowerCase().replace(/\s+/g, "-");
  const link = `/general/glossary#${anchor}`;
  return <a class="glossary-term" href={link}>{t}</a>;
};

Push sources are used in cases when some external actor actively sends data into a dataset.

Push sources are suitable for:

* IoT devices that periodically send data
* Business processes that report events directly into ODF dataset
* For ingesting data from streaming data APIs and event queues like [Apache Kafka](https://kafka.apache.org/).

## Source Metadata

Push sources are added via <Schema t="AddPushSource" /> metadata event:

```yaml theme={null}
kind: DatasetSnapshot
version: 1
content:
  name: gps
  kind: Root
  metadata:
    - kind: AddPushSource
      sourceName: default
      read:
        kind: NdJson
        schema:
          - t TIMESTAMP
          - long DOUBLE
          - lat DOUBLE
      merge:
        kind: Append
    - kind: SetVocab
      eventTimeColumn: t
```

Note that push sources are **named**:

```yaml theme={null}
sourceName: default
```

This allows you to declare **multiple push sources per dataset**, which may be useful for situations where several devices from different manufacturers produce slightly different data, but you want them to be able to write directly to one dataset.

An existing push source can be deactivated using <Schema t="DisablePushSource" /> event.

Also note that, just like polling sources, push sources pass data through a [Merge Strategy](/cli/ingest/merge-strategies), so they too can benefit, for example, from **event deduplication** of the <Schema t="Ledger" id="MergeStrategy::Ledger" /> merge strategy.

## Pushing Data via CLI

To push data into dataset via `kamu` use [`kamu ingest`](/cli/commands#kamu-ingest) command.

You can push data from files:

```sh theme={null}
echo '{"t": "2020-01-01T00:00:00Z", "long": -123.12, "lat": 49.28}' > data.json
kamu ingest gps data.json
```

Or from directly from STDIN:

```sh theme={null}
echo '{"t": "2020-01-01T00:00:00Z", "long": -123.12, "lat": 49.28}' | kamu ingest gps --stdin
```

Result:

```sh theme={null}
kamu tail gps
```

```sh theme={null}
┌────────┬────┬──────────────────────────┬──────────────────────┬─────────┬───────┐
│ offset │ op │       system_time        │          t           │  long   │  lat  │
├────────┼────┼──────────────────────────┼──────────────────────┼─────────┼───────┤
│      0 │ +A │ 2024-01-31T03:51:03.618Z │ 2020-01-01T00:00:00Z │ -123.12 │ 49.28 │
└────────┴────┴──────────────────────────┴──────────────────────┴─────────┴───────┘
```

## Pushing Data via API

See [APIs documentation](/api) for various options of pushing data programmatically via APIs.

## Format Translation

While push sources define a specific <Schema t="read format" id="ReadStep" /> for their data - there is some tolerance. For example you can push <Schema t="Csv" id="ReadStep::Csv" /> data to the source that declares <Schema t="NdJson" id="ReadStep::NdJson" /> format and `kamu` will attempt to read it using the same schema.

This behavior works both in CLI commands and in push APIs.