Set parquet writer page size and row group size in ParquetTester
Set both setting to smaller value could force writer to write multiple
row groups and multiple pages in each row group, which could better
simulate real world examples.
@@ -703,7 +703,10 @@ public class ParquetTester
new FileOutputStream(outputFile),
columnNames,
types,
- ParquetWriterOptions.builder().build(),
+ ParquetWriterOptions.builder()
+ .setMaxPageSize(DataSize.succinctBytes(100))
+ .setMaxBlockSize(DataSize.succinctBytes(100000))
+ .build(),
compressionCodecName.getHadoopCompressionCodecClassName());
PageBuilder pageBuilder = new PageBuilder(types);
