If you need to create a single node Kafka server quickly for development or testing and need to be repeatable, the following are the steps. This uses Chef-Solo and community cookbooks.
Pre-req:
A linux server machine or VM with just the OS installed
Connection to internet
git, curl and wget tools available for the user
If the tools are not available, as root or using sudo install these tools using apt-get or yum
Installing Chef:
1
curl -L https://www.opscode.com/chef/install.sh | bash
1
wget http://github.com/opscode/chef-repo/tarball/master
1
mv opscode-chef-repo* chef-repo
cd to chef-repo; if you do a ls there will be a list of directories
1
echo "cookbook_path [ '/path-to/chef-repo/cookbooks' ]" > .chef/knife.rb
1
2
git clone https://github.com/socrata-cookbooks/java.git
git clone https://github.com/bijugs/kafka-cookbook.git -b kafka_enhancement kafka
1
vi attributes/zookeeper.rb
and change the default[:zookeeper][:data_dir] value if required
Under chef-solo directory: vi solo.rb and add the following details
1
2
file_cache_path "/where-ever/chef-solo"
cookbook_path "/where-ever/chef-repo/cookbooks"
Under chef-solo directory: vi kafka.json and include the following
1
2
3
4
5
6
7
8
9
10
11
12
13
14
{
"java": {
"jdk_version" : "7",
"accept_license_agreement": true,
"oracle": {
"accept_oracle_download_terms": true
}
},
"run_list": [
"recipe[java::oracle]",
"recipe[kafka::zookeeper]",
"recipe[kafka]"
]
}
Staying under the chef-solo directory run
1
sudo chef-solo -c solo.rb -j kafka.json
If you don’t have the time to go through these steps manually, check out a quick hack which automates these steps https://github.com/bijugs/simple-scripts/blob/master/install_kafka_using_chef_solo.sh