BigQuery Schema Generator
GitHub Toggle Dark/Light/Auto mode Toggle Dark/Light/Auto mode Toggle Dark/Light/Auto mode Back to homepage
  1. BigQuery Schema Generator
  2. /
  3. BQ Protobuf Options
Edit page

BQ Protobuf Options

Protocol Documentation

Table of Contents

Top

api/bq_field.proto

Copyright 2014 Google LLC

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

BigQueryFieldOptions

Message containing options related to BigQuery schema generation and management via Protobuf.

Field Type Label Description
require bool Flag to specify that a field should be marked as 'REQUIRED' when used to generate schema for BigQuery.
type_override string
ignore bool Optionally omit a field from BigQuery schema.
description string Set the description for a field in BigQuery schema.
name string Customize the name of the field in the BigQuery schema.
policy_tags string Optionally add PolicyTag for a field in BigQuery schema.

File-level Extensions

Extension Type Base Number Description
bigquery BigQueryFieldOptions .google.protobuf.FieldOptions 1021 BigQuery field schema generation options.

Top

api/bq_table.proto

BigQueryMessageOptions

Field Type Label Description
table_name string Specifies a name of table in BigQuery for the message.

If not blank, indicates the message is a type of record to be stored into BigQuery. | | use_json_names | bool | | If true, BigQuery field names will default to a field's JSON name, not its original/proto field name. | | extra_fields | string | repeated | If set, adds defined extra fields to a JSON representation of the message. Value format: "<field name>:<BigQuery field type>" for basic types or "<field name>:RECORD:<protobuf type>" for message types. "NULLABLE" by default, different mode may be set via optional suffix ":<mode>" |

File-level Extensions

Extension Type Base Number Description
bigquery_opts BigQueryMessageOptions .google.protobuf.MessageOptions 1021 NB: There used to be a custom option named "table_name". But only one tag is registered for this project: 1021. So when adding other options, the only solution was to change tag 1021 to be a message, with all the various options as fields therein.

If you are upgrading this library and find that protoc begins to reject your proto files, you'll need to change all lines that look like so: option (gen_bq_schema.table_name) = "foo"; to instead look like this: option (gen_bq_schema.bigquery_opts).table_name = "foo"; There was no backwards compatible way to make this change. Sorry for the inconvenience. BigQuery message schema generation options.

The field number is a globally unique id for this option, assigned by protobuf-global-extension-registry@google.com |

Scalar Value Types

.proto Type Notes C++ Java Python Go C# PHP Ruby
double double double float float64 double float Float
float float float float float32 float float Float
int32 Uses variable-length encoding. Inefficient for encoding negative numbers – if your field is likely to have negative values, use sint32 instead. int32 int int int32 int integer Bignum or Fixnum (as required)
int64 Uses variable-length encoding. Inefficient for encoding negative numbers – if your field is likely to have negative values, use sint64 instead. int64 long int/long int64 long integer/string Bignum
uint32 Uses variable-length encoding. uint32 int int/long uint32 uint integer Bignum or Fixnum (as required)
uint64 Uses variable-length encoding. uint64 long int/long uint64 ulong integer/string Bignum or Fixnum (as required)
sint32 Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int32s. int32 int int int32 int integer Bignum or Fixnum (as required)
sint64 Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int64s. int64 long int/long int64 long integer/string Bignum
fixed32 Always four bytes. More efficient than uint32 if values are often greater than 2^28. uint32 int int uint32 uint integer Bignum or Fixnum (as required)
fixed64 Always eight bytes. More efficient than uint64 if values are often greater than 2^56. uint64 long int/long uint64 ulong integer/string Bignum
sfixed32 Always four bytes. int32 int int int32 int integer Bignum or Fixnum (as required)
sfixed64 Always eight bytes. int64 long int/long int64 long integer/string Bignum
bool bool boolean boolean bool bool boolean TrueClass/FalseClass
string A string must always contain UTF-8 encoded or 7-bit ASCII text. string String str/unicode string string string String (UTF-8)
bytes May contain any arbitrary sequence of bytes. string ByteString str []byte ByteString string String (ASCII-8BIT)