Skip to content
This repository has been archived by the owner on Apr 12, 2022. It is now read-only.

Logstash created pipeline in Kibana Management doesn't work, while same pipeline work as main #78

Open
Alex2357 opened this issue Feb 1, 2019 · 1 comment

Comments

@Alex2357
Copy link

Alex2357 commented Feb 1, 2019

Hi I added netflow pipeline in Kibana Management -> Logstash -> Pipelines

input {                                                                                                                                                                                                                                                                                               
  udp {                                                                                                                                               
    port => 40006                                                                                                                                      
    codec => netflow {                                                                                                                                
      versions => [5, 9]                                                                                                                              
    }                                                                                                                                                 
    type => netflow                                                                                                                                   
  }                                                                                                                                                   
}                                                                                                                                                                                                                                                                                                   

output {
        if ( [type] == "netflow" ) {
                elasticsearch {
                        
                        index => "logstash-netflow-%{host}-%{+YYYY.MM.dd}"
                        hosts    => [ 'elasticsearch' ]
                        user     => 'elastic'
                        password => "${ELASTIC_PASSWORD}" 
                        ssl => true
                        cacert => '/usr/share/logstash/config/certs/ca/ca.crt'

                }
        } else {
                elasticsearch {                        
                        index => "logstash-n-%{type}-%{+YYYY.MM.dd}"
                        hosts    => [ 'elasticsearch' ]
                        user     => 'elastic'
                        password => "${ELASTIC_PASSWORD}" 
                        ssl => true
                        cacert => '/usr/share/logstash/config/certs/ca/ca.crt'                        
                }
        }
}

and it did not work, but the main

input {
  heartbeat {
    interval => 5
    message  => 'Hello from Logstash 💓'
  }
}

output {
  elasticsearch {
    hosts    => [ 'elasticsearch' ]
    user     => 'elastic'
    password => "${ELASTIC_PASSWORD}"  # read password from logstash.keystore
    ssl => true
    cacert => '/usr/share/logstash/config/certs/ca/ca.crt'
  }
}

pipeline did work perfectly. Then I copied content of my pipeline
to ./config/logstash/pipeline/logstash.conf. Deleted the pipeline I created in Kibana. Did docker-compose up -d
and my pipeline started to work perfectly fine.

What do I need to do to make additional pipelines to work except the main one? Ideally I need to make multiple files ./config/logstash/pipeline/logstash.conf as I want it to be configured just from the docker-compose and avoid as much as possible manual configuration.

@Alex2357 Alex2357 changed the title Logstash created pipeline in Kibana Management doesn't work Logstash created pipeline in Kibana Management doesn't work, while same pipeline work as main Feb 1, 2019
@Alex2357
Copy link
Author

Most likely a clone of #16, I just need a time to make sure if in there there's a solution for multiple pipelines in multiple files.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant