Creates a Magpie table that references a topic or set of topics on a Kafka data source.
Syntax
create [temp | temporary] table { <table spec> | <table name> } [in schema <schema reference>] from kafka data source <data source reference> { topic "<topic name>" | topics "<topic list>" | topic pattern "<topic pattern>" | topic partitions <topic partitions spec> } [with key type <key type>] [with value type <value type>] [with starting offsets <starting offsets spec>] [with ending offsets <ending offsets spec>]
Parameters
table spec
JSON. A specification for saving the table. Note that persistence mapping and fields are not used when creating a table from Kafka.
table name
String. The name of the table to save the data as.
schema reference
String. The name of the schema to create the table in. Defaults to the current schema.
data source reference
String. The name of the kafka data source to reference.
topic name
String. The name of a single topic to load as a table.
topic list
String. A comma separated list of topics to load as a single table.
topic pattern
String. A java-style regular expression to use to search for topics to load.
topic partitions spec
JSON. A specification of specific topic partitions to load. For example: {"topicA":[0,1],"topicB":[2,4]}
.
key type
String. The data type of keys in the loaded topic or topics. Must be either string
or binary
. Default is string
.
value type
String. The data type of message values in the loaded topic or topics. Must be either string
or binary
. Default is string
.
starting offsets spec
JSON. The start point for loading messages from the configured topic or topics. Must be either "earliest"
or a specification or which offsets to use by topic partition, for example: {"topicA":{"0":23,"1":-2},"topicB":{"0":-2}}
. In the JSON, -2 refers to earliest. Default is "earliest"
.
ending offsets spec
JSON. The end point for loading messages from the configured topic or topics. Must be either "latest"
or a specification or which offsets to use by topic partition, for example: {"topicA":{"0":23,"1":-1},"topicB":{"0":-1}}
. In the JSON, -1 refers to latest. Default is "latest"
.