log4j - Logstash-Kafka integration not working -

we want logs using logstash , pass them kafka.

we have written following conf file logstash 1.5.0 beta 1 , kafka 2.9.2_0.8.1.1

**

input {              file {              type => "apache"              path => ["/var/log/apache2/access.log", "/var/log/apache2/error.log"]          } }     output {         kafka {             codec => plain {                 format => "%{message}"             }     topic_id => "example1"         }     } 

**

after running following command : bin/logstash agent -f test.conf --log ex.log

test.conf our conf file. ex.log blank file have created logs stored.

we getting following output

sending logstash logs ex.log. using milestone 2 input plugin 'file'. plugin should stable, if see strange behavior, please let know! more information on plugin milestones, see http://logstash.net/docs/1.5.0.beta1/plugin-milestones {:level=>:warn} using milestone 1 output plugin 'kafka'. plugin should work, benefit use folks you. please let know if find bugs or have suggestions on how improve plugin. more information on plugin milestones, see http://logstash.net/docs/1.5.0.beta1/plugin-milestones {:level=>:warn} log4j:warn no appenders found logger (kafka.utils.verifiableproperties). log4j:warn please initialize log4j system properly. log4j:warn see http: //logging.apache.org/log4j/1.2/faq.html#noconfig more info. slf4j: failed load class "org.slf4j.impl.staticloggerbinder". slf4j: defaulting no-operation (nop) logger implementation slf4j: see http:/ /www.slf4j.org/codes.html#staticloggerbinder further details.

we tried setting classpath in bashrc. did not work. please tell going wrong. thankyou in advance!

some things can check: topic example1 available? if not did use auto create in kafka? check existing topics this:

bin/kafka-topics.sh --list --zookeeper localhost:2181 

example1 should between returned items, if not can create topic manually well.

bin/kafktopics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic example1 

to see if kafka receives message start consumer:

bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic example1 

i started simple logstash instance listens standard in , using consumer see if message arrives in kafka. logstash configuration:

input {         stdin { } }  output {     kafka {         codec => plain {             format => "%{message}"         }         topic_id => "example1"     } } 

hope helps