Key Default Type Description
format-table.repair.collect-statistics
false Boolean Whether MSCK REPAIR TABLE on a Format Table also measures the partitions it finds. Off by default: measuring lists the files inside every partition, not only the partition directories.
format-table.statistics.parallelism
8 Integer How many requests MSCK REPAIR TABLE and ANALYZE TABLE use at once to measure Format Table partitions, so that a large table does not burst them at storage.
legacy-timestamp-mapping.enabled
false Boolean If true, map Paimon TIMESTAMP to Spark TIMESTAMP instead of TIMESTAMP_NTZ.
read.allow.fullScan
true Boolean Whether to allow full scan when reading a partitioned table.
read.changelog
false Boolean Whether to read row in the form of changelog (add rowkind column in row to represent its change type).
read.stream.maxBytesPerTrigger
(none) Long The maximum number of bytes returned in a single batch.
read.stream.maxFilesPerTrigger
(none) Integer The maximum number of files returned in a single batch.
read.stream.maxRowsPerTrigger
(none) Long The maximum number of rows returned in a single batch.
read.stream.maxTriggerDelayMs
(none) Long The maximum delay between two adjacent batches, which used to create MinRowsReadLimit with read.stream.minRowsPerTrigger together.
read.stream.minRowsPerTrigger
(none) Long The minimum number of rows returned in a single batch, which used to create MinRowsReadLimit with read.stream.maxTriggerDelayMs together.
requiredSparkConfsCheck.enabled
true Boolean Whether to verify SparkSession is initialized with required configurations.
scan.preserve-data-grouping
false Boolean Whether batch scans preserve bucket grouping for Spark to use the table's distribution and ordering. Requires Spark V2 bucketing to be enabled. If false, scans use regular split packing and report no bucket distribution or ordering. If true, Spark may group multiple read units into one task per bucket, or use them separately for a partially clustered join. Set as a table/read option or with spark.paimon.scan.preserve-data-grouping. The choice is fixed when the scan is created.
source.split.target-size-with-column-pruning
false Boolean Whether to adjust the target split size based on pruned (projected) columns. If enabled, split size estimation uses only the columns actually being read.
vector-search.lateral-join.parallelism
16 Integer Parallelism used to repartition a single-partition LIMIT input before executing a lateral vector search.
write.data-evolution.update-conflict-retry.max-attempts
20 Integer Maximum attempts for Spark V1 UPDATE on data-evolution tables when concurrent partial-column updates conflict on the same row-id range and update columns. Values less than 2 disable retry.
write.data-evolution.update-conflict-retry.wait-ms
10 Long Wait time in milliseconds between retry attempts for Spark V1 UPDATE on data-evolution tables after row-id range update conflicts.
write.hive-style-dynamic-partition.enabled
false Boolean If true, positional SQL inserts with explicit dynamic partitions use Hive's column order, with non-dynamic columns followed by dynamic partition columns. If false, the query output follows the table schema order.
write.merge-schema
false Boolean If true, evolve the table schema to accept new columns from the incoming data. Existing column types are preserved and incoming values are cast to them; to also widen existing types, enable 'write.merge-schema.type-widening'.
write.merge-schema.explicit-cast
false Boolean Only effective when 'write.merge-schema.type-widening' is true. If true, also allow lossy type changes between compatible types (e.g. BIGINT -> INT, STRING -> DATE).
write.merge-schema.type-widening
false Boolean Only effective when 'write.merge-schema' is true. If true, widen an existing column type when the incoming data has a wider compatible type (e.g. INT -> BIGINT, DECIMAL precision increase). Lossy changes are still rejected unless 'write.merge-schema.explicit-cast' is also true.
write.use-v2-write
false Boolean If true, v2 write will be used. Currently, only HASH_FIXED and BUCKET_UNAWARE bucket modes are supported. Will fall back to v1 write for other bucket modes. Currently, Spark V2 write does not support TableCapability.STREAMING_WRITE.