Avro File Output
Options
Option | Description |
---|---|
Transform name | Name of the transform. Note: This name has to be unique in a single pipeline. |
Output Type | The type of output you want, this can be binary file, binary message or a JSON message. When selecting message it will be sent to the Output Field. |
Filename | The filepath where to save the avro data file. |
Output Field | When not writing to a file this is the fieldname used to output the message to. |
Automatically create schema | You can either select an existing schema or create a schema based on the fields in the fields tab. |
Write schema to file | Select this to write the schema to a separate file. |
Avro namespace | Namespace used in the Avro file. |
Avro record name | Name used for the Avro record. |
Avro documentation | |
Schema filename | Filepath to write or read the schema from. |
Create parent folder | When creating a file, select this if you want to create the parent folder. |
Compression codec | Compression used for the avro messages when writing to a file. |
Include transform nr in filename | Add the Transform instance number to the filename. |
Include partition nr in filename | Add the Partition number to the filename. |
Include date in filename | Include the date when the file was created in the filename |
Include time in filename | Include the time when the file was created in the filename |
Specify date format | Specify a custom date and format to attach to the filename |
Date time format | Date time specification used for the formatting |
Add filenames to result | Add the resulting filenames that are created to the filename results |
Fields tab:
Option | Description |
---|---|
Name | The source field in the stream. |
Avro Schema Path | The destination field, when an existing schema is specified this is populated with the fields from the schema to map the input fields to. |
Avro Type | The Avro type of the field in the Avro destination file. |
Nullable | Is the field nullable? |