Announcement
From Embulk v0.9.16 (Released on 2019/03/11) has started to support built-in json parser.
- The embulk built-in json parser has same behavior for the columns option.
- The embulk built-in json parser does not support automatic type conversion with the default_typecast option.
For more detail you can visit this article
Jsonl parser plugin for Embulk
JSONL (JSON Lines) parser plugin for Embulk
Overview
- Plugin type: parser
- Guess supported: yes
Configuration
- type: Specify this parser as jsonl
- columns: Specify column name and type. See below (array, required)
- stop_on_invalid_record: Stop bulk load transaction if a file includes invalid record (such as invalid timestamp) (boolean, default: false)
- default_timezone: Default timezone of the timestamp (string, default: UTC)
-
default_timestamp_format: Default timestamp format of the timestamp (string, default:
%Y-%m-%d %H:%M:%S.%N %z
) - newline: Newline character (CRLF, LF or CR) (string, default: CRLF)
- charset: Character encoding (eg. ISO-8859-1, UTF-8) (string, default: UTF-8)
- default_typecast: Specify whether to cast values automatically to the specified types or not (boolean, default: true)
- min_rows_for_guess: Minimum rows for guess (integer, default: 4)
columns
- name: Name of the column (string, required)
- type: Type of the column (string, required)
- timezone: Timezone of the timestamp if type is timestamp (string, default: default_timestamp)
- format: Format of the timestamp if type is timestamp (string, default: default_format)
- typecast: Whether cast values or not (boolean, default: default_typecast)
Guess
(If guess supported) you don't have to write parser:
section in the configuration file. After writing in:
section, you can let embulk guess parser:
section using this command:
$ embulk gem install embulk-parser-jsonl
$ embulk guess -g jsonl config.yml -o guessed.yml
Build
$ ./gradlew gem classpath