MINOR: changes to the production broker configuration docs.
Author: Alex Loddengaard
Reviewers: Jun Rao
Closes from alexlod/production-config-docs
<h3><a id="config" href="#config">6.3 Kafka Configuration</a></h3>
<h4><a id="clientconfig" href="#clientconfig">Important Client Configurations</a></h4>
- The most important producer configurations control
+ The most important old Scala producer configurations control
<ul>
+ <li>acks</li>
<li>compression</li>
<li>sync vs async production</li>
<li>batch size (for async producers)</li>
</ul>
+ The most important new Java producer configurations control
+ <ul>
+ <li>acks</li>
+ <li>compression</li>
+ <li>batch size</li>
+ </ul>
The most important consumer configuration is the fetch size.
<p>
All configurations are documented in the <a href="#configuration">configuration</a> section.
<p>
<h4><a id="prodconfig" href="#prodconfig">A Production Server Config</a></h4>
- Here is our production server configuration:
+ Here is an example production server configuration:
<pre>
- # Replication configurations
- replica.fetch.max.bytes=1048576
- replica.fetch.wait.max.ms=500
- replica.high.watermark.checkpoint.interval.ms=5000
- replica.socket.timeout.ms=30000
- replica.socket.receive.buffer.bytes=65536
- replica.lag.time.max.ms=10000
-
- controller.socket.timeout.ms=30000
+ # ZooKeeper
+ zookeeper.connect=[list of ZooKeeper servers]
# Log configuration
num.partitions=8
- message.max.bytes=1000000
- auto.create.topics.enable=true
- log.index.interval.bytes=4096
- log.index.size.max.bytes=10485760
- log.retention.hours=168
- log.roll.hours=168
- log.retention.check.interval.ms=300000
- log.segment.bytes=1073741824
-
- # ZK configuration
- zookeeper.connection.timeout.ms=6000
- zookeeper.sync.time.ms=2000
-
- # Socket server configuration
- num.io.threads=8
- num.network.threads=8
- socket.request.max.bytes=104857600
- socket.receive.buffer.bytes=1048576
- socket.send.buffer.bytes=1048576
- queued.max.requests=500
- fetch.purgatory.purge.interval.requests=100
- producer.purgatory.purge.interval.requests=100
+ default.replication.factor=3
+ log.dir=[List of directories. Kafka should have its own dedicated disk(s) or SSD(s).]
+
+ # Other configurations
+ broker.id=[An integer. Start with 0 and increment by 1 for each new broker.]
+ listeners=[list of listeners]
+ auto.create.topics.enable=false
+ min.insync.replicas=2
+ queued.max.requests=[number of concurrent requests]
</pre>
Our client configuration varies a fair amount between different use cases.
