REST

Use the REST Connector to send data to Decodable over HTTPs. After you’ve created a REST connection, you’re able to send data to Decodable using the /v1alpha2/connections/<connection-id>/eventsendpoint.

Overview

Connector name

rest

Type

source

Delivery guarantee

at least once

Steps: Create a REST connection

Follow these steps to get data into Decodable using the REST Connector.

  1. From the Connections page, select the REST Connector and complete the following fields.

    UI Field Property Name in the Decodable CLI Description

    Endpoint

    endpoint

    The location where your application or service will need to send HTTP POST requests to. The complete URL will be in the format https://<your_account>.api.decodable.co/v1alpha2/connections/<connection_id>/events.

    Once the connection has been created, you can find the connection id by double-clicking on the name of the connector. The id is shown directly below the connector name.

    Value Format

    format

    Must be set to JSON.

    Parse error policy

    parse-error-policy

    Optional. Select the error handling policy. Must be one of the following:

    - FAIL: When set to FAIL, Decodable stops the connection if any validation errors are encountered in the incoming data stream. - IGNORE: When set to IGNORE, Decodable ignores all invalid records. All validated records are sent. With this policy, the connection is not stopped nor will any errors or warnings be shown if there is an invalid record. A total successful count will be returned int he response body.

    Defaults to FAIL.

  2. Select the stream that you’d like to connect to this connector. This will be the stream that receives events using REST. Then, select Next.

  3. Define the connection’s schema. Select New Schema to manually enter the fields and field types present or Import Schema if you want to paste the schema in the form of an Avro or JSON array.

    1. The stream’s schema must match the schema of the data that you plan on sending through this connection.

    2. For more information about creating a stream or defining the stream schema, see Create and Manage Streams.

  4. Select Next when you are finished defining the connection’s schema.

  5. Give the newly created connection a Name and Description and select Save.

Your Decodable stream can now receive data from HTTP clients using the REST Connector. Next, configure your HTTP clients to send data to Decodable by following the instructions in the next section.

Send data to Decodable using the REST APIs

To send data through the REST connection you created earlier, follow these steps.

  1. Generate the initial access token by logging into the Decodable CLI. See The Decodable CLI for Decodable CLI installation instructions.

  2. Locate and export the access token. After logging in to the Decodable CLI, find the access token in ~/.decodable/auth and put the token value into an environment variable in your shell.

    export DECODABLE_TOKEN="<access_token value>"
  3. Send your records through the REST connection. Use the following example as a template for your request, where payload.jsonis a file containing the record(s) you want to submit.

    curl -X POST https://<your_account>.api.decodable.co/v1alpha2/connections/<connection_id>/events \
      -H "Content-Type: application/json" \
      -H "Accept: application/json" \
      -H "Authorization: Bearer ${DECODABLE_TOKEN}" \
      -d @payload.json

    Notes:

    • Replace <your_account> with the name of your Decodable account and <connection_id> with the ID of your REST connection.

    • The payload.json file must be in the following format, and the records must have the same schema as the schema that you defined in the connection.

      {
        "events": [
          {
            "<fieldName>": <fieldValue>
          }
          ...
        ]
      }
  4. (Optional) Refresh the access token by logging into the Decodable CLI again. The access token expires after 24 hours.

A response displaying how many records were written is shown. Records are written atomically to the stream, meaning all records in the POST body are written or none are. Although rare, if an error occurs after successful writing (e.g., a network partition or failure of the client or server), the client will need to retry, possibly resulting in duplicates. Records are guaranteed to be written in the order they appear in the events element of the POST body.