Welcome to the Linux Foundation Forum!

Lab4-4

Hello,
If I use as:
sudo docker run -d --name elasticsearch -p 9200:9200 -e "discovery.type=single-node"docker.elastic.co/elasticsearch/elasticsearch:7.12.0
curl 172.17.0.3:9200 ---> It's Ok

I got after fluentd --config ./fluent/fluentnginxelastic.conf :

2022-04-03 15:22:52 +0300 [error]: #0 config error file="./fluent/fluentnginxelastic.conf" error_class=Fluent::ConfigError error="Using Elasticsearch client 8.1.0 is not compatible for your Elasticsearch server. Please check your using elasticsearch gem version and Elasticsearch server."
2022-04-03 15:22:53 +0300 [error]: Worker 0 finished unexpectedly with status 2
2022-04-03 15:22:53 +0300 [info]: Received graceful stop

If I use as:
sudo docker run -d --name elasticsearch -p 9200:9200 -e "discovery.type=single-node" docker.elastic.co/elasticsearch/elasticsearch:8.1.0
curl 172.17.0.3:9200 --> curl: (52) Empty reply from server

I also got after fluentd --config ./fluent/fluentnginxelastic.conf
2022-04-03 15:58:04 +0300 [info]: starting fluentd-1.14.6 pid=15599 ruby="2.7.0"
2022-04-03 15:58:04 +0300 [info]: spawn command to main: cmdline=["/usr/bin/ruby2.7", "-Eascii-8bit:ascii-8bit", "/usr/local/bin/fluentd", "--config", "./fluent/fluentnginxelastic.conf", "--under-supervisor"]
2022-04-03 15:58:05 +0300 [info]: adding match pattern="nginx.**" type="elasticsearch"
The client is unable to verify that the server is Elasticsearch. Some functionality may not be compatible if the server is running an unsupported product.
2022-04-03 15:58:08 +0300 [warn]: #0 Could not communicate to Elasticsearch, resetting connection and trying again. Connection refused - connect(2) for 172.17.0.3:9200 (Errno::ECONNREFUSED)
2022-04-03 15:58:08 +0300 [warn]: #0 Remaining retry: 14. Retry to communicate after 2 second(s).
The client is unable to verify that the server is Elasticsearch. Some functionality may not be compatible if the server is running an unsupported product.
2022-04-03 15:58:12 +0300 [warn]: #0 Could not communicate to Elasticsearch, resetting connection and trying again. Connection refused - connect(2) for 172.17.0.3:9200 (Errno::ECONNREFUSED)
2022-04-03 15:58:12 +0300 [warn]: #0 Remaining retry: 13. Retry to communicate after 4 second(s).
The client is unable to verify that the server is Elasticsearch. Some functionality may not be compatible if the server is running an unsupported product.
2022-04-03 15:58:20 +0300 [warn]: #0 Could not communicate to Elasticsearch, resetting connection and trying again. EOFError (EOFError)
2022-04-03 15:58:20 +0300 [warn]: #0 Remaining retry: 12. Retry to communicate after 8 second(s).

It seems like the version of plugin/elasticsearch/nginx incompatible.

BR,
Alexey

Comments

  • In this case you will have to install a specific version of the elasticsearch client plugin or try running a higher Elasticsearch version.

    The high level steps would be:

    • Remove the fluent-plugin-elasticsearch gem that was automatically installed
    • Install fluent-plugin-elasticsearch version 5.0.5 (that is what was used in the text)

    I will work out the specific commands and post it here within the day, thank you for the report.

  • joshl
    joshl Posts: 37

    I got a similar error:

    The client is unable to verify that the server is Elasticsearch. Some functionality may not be compatible if the server is running an unsupported product.
    2022-05-13 22:54:07 +0000 [error]: #0 config error file="/home/ubuntu/labsys/lab4/lab4.conf" error_class=Fluent::ConfigError error="Using Elasticsearch client 8.2.0 is not compatible for your Elasticsearch server. Please check your using elasticsearch gem version and Elasticsearch server."
    2022-05-13 22:54:07 +0000 [error]: Worker 0 finished unexpectedly with status 2
    

    Is there an update to this lab 4-4 yet?

  • Not yet, though it will be addressed in the upcoming update.

    For now, install version 5.0.5 with this command:

    sudo fluent-gem install -N fluent-plugin-elasticsearch -v '=5.0.5'

  • joshl
    joshl Posts: 37

    I've uninstalled 5.2.2 and installed 5.0.5 and got this:

    2022-05-14 08:59:47 +0000 [info]: parsing config file is succeeded path="/home/ubuntu/labsys/lab4/lab4.conf" 2022-05-14 08:59:47 +0000 [info]: gem 'fluent-plugin-elasticsearch' version '5.0.5' 2022-05-14 08:59:47 +0000 [info]: gem 'fluent-plugin-mongo' version '1.5.0' 2022-05-14 08:59:47 +0000 [info]: gem 'fluent-plugin-nginx-error-multiline' version '0.2.0' 2022-05-14 08:59:47 +0000 [info]: gem 'fluentd' version '1.14.6' Traceback (most recent call last): 15: from /usr/local/bin/fluentd:23:in

    '
    14: from /usr/local/bin/fluentd:23:in load' 13: from /var/lib/gems/2.7.0/gems/fluentd-1.14.6/bin/fluentd:15:in'
    12: from /usr/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:72:in require' 11: from /usr/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:72:inrequire'
    10: from /var/lib/gems/2.7.0/gems/fluentd-1.14.6/lib/fluent/command/fluentd.rb:356:in <top (required)>' 9: from /var/lib/gems/2.7.0/gems/fluentd-1.14.6/lib/fluent/supervisor.rb:668:inrun_supervisor'
    8: from /var/lib/gems/2.7.0/gems/fluentd-1.14.6/lib/fluent/engine.rb:80:in run_configure' 7: from /var/lib/gems/2.7.0/gems/fluentd-1.14.6/lib/fluent/engine.rb:105:inconfigure'
    6: from /var/lib/gems/2.7.0/gems/fluentd-1.14.6/lib/fluent/root_agent.rb:149:in configure' 5: from /var/lib/gems/2.7.0/gems/fluentd-1.14.6/lib/fluent/agent.rb:64:inconfigure'
    4: from /var/lib/gems/2.7.0/gems/fluentd-1.14.6/lib/fluent/agent.rb:64:in each' 3: from /var/lib/gems/2.7.0/gems/fluentd-1.14.6/lib/fluent/agent.rb:74:inblock in configure'
    2: from /var/lib/gems/2.7.0/gems/fluentd-1.14.6/lib/fluent/agent.rb:132:in add_match' 1: from /var/lib/gems/2.7.0/gems/fluentd-1.14.6/lib/fluent/plugin.rb:187:inconfigure'
    /var/lib/gems/2.7.0/gems/fluent-plugin-elasticsearch-5.0.5/lib/fluent/plugin/out_elasticsearch.rb:416:in configure': uninitialized constant Elasticsearch::Transport::VERSION (NameError) Did you mean? Elasticsearch::VERSION

    I've reinstalled 5.2.2 and got his.

    The client is unable to verify that the server is Elasticsearch. Some functionality may not be compatible if the server is running an unsupported product. 2022-05-14 09:43:32 +0000 [warn]: #0 Could not communicate to Elasticsearch, resetting connection and trying again. Connection refused - connect(2) for 172.17.0.3:9200 (Errno::ECONNREFUSED) 2022-05-14 09:43:32 +0000 [warn]: #0 Remaining retry: 14. Retry to communicate after 2 second(s). The client is unable to verify that the server is Elasticsearch. Some functionality may not be compatible if the server is running an unsupported product. 2022-05-14 09:43:36 +0000 [warn]: #0 Could not communicate to Elasticsearch, resetting connection and trying again. Connection refused - connect(2) for 172.17.0.3:9200 (Errno::ECONNREFUSED) 2022-05-14 09:43:36 +0000 [warn]: #0 Remaining retry: 13. Retry to communicate after 4 second(s).

  • Finally got it to work by installing the v7.13.3 elasticsearch and elasticsearch-transport gems alongside fluent-plugin-elasticsearch v5.0.5:

    ~$ sudo fluent-gem uninstall elasticsearch elasticsearch-transport fluent-plugin-elasticsearch
    
    ~$ sudo fluent-gem install elasticsearch -v 7.13.3
    
    ...
    
    ~$ sudo fluent-gem install elasticsearch-transport -v 7.13.3
    
    ...
    
    ~$ sudo fluent-gem install fluent-plugin-elasticsearch -v 5.0.5
    
    ...
    
    ~$
    

    With the following config:

    <source>
      @type tail
      <parse>
        @type nginx
      </parse>
      path /tmp/lab4/nginx/access.log
      pos_file /tmp/lab4/nginx/access.pos
      tag nginx.access
    </source>
    
    <source>
      @type tail
      path /tmp/lab4/nginx/error.log
      pos_file /tmp/lab4/nginx/error.pos
      tag nginx.error
      <parse>
        @type nginx_error_multiline
      </parse>
    </source>
    
    <match nginx.**>
      @type elasticsearch
      host 172.17.0.3
      port 9200
    </match>
    

    It ran successfully:

    ~$ fluentd -c ~/lab4/lab4.conf
    
    2022-05-14 16:26:05 +0000 [info]: parsing config file is succeeded path="/home/ubuntu/lab4.conf"
    2022-05-14 16:26:05 +0000 [info]: gem 'fluent-plugin-elasticsearch' version '5.0.5'
    2022-05-14 16:26:05 +0000 [info]: gem 'fluent-plugin-nginx-error-multiline' version '0.2.0'
    2022-05-14 16:26:05 +0000 [info]: gem 'fluentd' version '1.14.6'
    2022-05-14 16:26:06 +0000 [info]: using configuration file: <ROOT>
      <source>
        @type tail
        path "/tmp/lab4/nginx/access.log"
        pos_file "/tmp/lab4/nginx/access.pos"
        tag "nginx.access"
        <parse>
          @type "nginx"
          unmatched_lines 
        </parse>
      </source>
      <source>
        @type tail
        path "/tmp/lab4/nginx/error.log"
        pos_file "/tmp/lab4/nginx/error.pos"
        tag "nginx.error"
        <parse>
          @type "nginx_error_multiline"
          unmatched_lines 
          format1 /^(?<time>\d{4}\/\d{2}\/\d{2} \d{2}:\d{2}:\d{2}) \[(?<log_level>\w+)\] (?<pid>\d+).(?<tid>\d+): (?<message>.*)/
        </parse>
      </source>
      <match nginx.**>
        @type elasticsearch
        host "127.0.0.1"
        port 9200
      </match>
    </ROOT>
    2022-05-14 16:26:06 +0000 [info]: starting fluentd-1.14.6 pid=759509 ruby="2.7.0"
    2022-05-14 16:26:06 +0000 [info]: spawn command to main:  cmdline=["/usr/bin/ruby2.7", "-Eascii-8bit:ascii-8bit", "/usr/local/bin/fluentd", "-c", "/home/ubuntu/lab4.conf", "--under-supervisor"]
    2022-05-14 16:26:06 +0000 [info]: adding match pattern="nginx.**" type="elasticsearch"
    2022-05-14 16:26:07 +0000 [warn]: #0 Detected ES 7.x: `_doc` will be used as the document `_type`.
    2022-05-14 16:26:07 +0000 [info]: adding source type="tail"
    2022-05-14 16:26:07 +0000 [info]: adding source type="tail"
    2022-05-14 16:26:07 +0000 [info]: #0 starting fluentd worker pid=759514 ppid=759509 worker=0
    2022-05-14 16:26:07 +0000 [info]: #0 following tail of /tmp/lab4/nginx/error.log
    2022-05-14 16:26:07 +0000 [info]: #0 following tail of /tmp/lab4/nginx/access.log
    2022-05-14 16:26:07 +0000 [info]: #0 fluentd worker is now running worker=0
    
  • joshl
    joshl Posts: 37
    edited May 2022

    Thanks @ChristianLacsina964 ! That works. However, it came with this warning:

    warning: 299 Elasticsearch-7.13.3-5d21bea28db1e89ecc1f66311ebdec9dc3aa7d64 "Elasticsearch built-in security features are not enabled. Without authentication, your cluster could be accessible to anyone. See https://www.elastic.co/guide/en/elasticsearch/reference/7.13/security-minimal-setup.html to enable security."

Categories

Upcoming Training