Welcome to the Linux Foundation Forum!

Exercise 5.3: Cannot get example for fluentd working as described

bryonbakerbryonbaker Posts: 28
edited June 2019 in LFD259 Class Forum

Hi,
I followed the labs and the example to get fluentd forwarding the nginx logs would not work. The issue appeared to be with forwarding to the container's localhost.

There were a few of issues:
1. Fluentd does not appear to read the environment variable FLUENTD_ARGS so I had to overwrite the default path /fluentd/etc
2. The fluent.conf file match would not forward to localhost so I had to change it to stdout.
3. There is a mistake in the lab. The correct path for weblog-pv-storage should be /var/log/nginx, not /var/log.

Below is what I ended up with to get it going. Can you please help me understand why #1 and #2 didn't work as expected?

apiVersion: v1
kind: List
items:
- apiVersion: v1
  kind: PersistentVolume
  metadata:
    name: weblog-pv
    labels:
      type: local
  spec:
    storageClassName: manual
    accessModes: 
    - ReadWriteOnce
    hostPath:
      path: /tmp/weblog
      type: DirectoryOrCreate
    capacity: 
      storage: 500Mi

- apiVersion: v1
  kind: PersistentVolumeClaim
  metadata:
    name: weblog-pvc
  spec:
    storageClassName: manual
    accessModes:
    - ReadWriteOnce
    resources:
      requests:
        storage: 200Mi

- apiVersion: v1
  kind: ConfigMap
  metadata:
    name: fluentd-config
  data:
    fluent.conf: |
      <source>
        @type tail
        format none
        path /var/log/nginx/access.log
        tag count.format1
      </source>
      <match *.**>
        @type stdout
      </match>

- apiVersion: v1
  kind: Pod
  metadata:
    name: sidecar-example
    labels:
      app: webserver
  spec:
    containers:
    - name: nginx
      image: nginx:latest
      ports: 
      - containerPort: 80
      volumeMounts:
      - name: logging-vol
        mountPath: /var/log/nginx
    - name: fdlogger
      env:
        - name: FLUENTD_ARGS
          value: -c /etc/fluentd-config/fluent.conf
      image: fluent/fluentd
      volumeMounts:
      - name: logging-vol
        mountPath: /var/log/nginx
      - name: log-config
        # mountPath: /etc/fluentd-config
        mountPath: /fluentd/etc
    volumes:
      - name: logging-vol
        persistentVolumeClaim: 
          claimName: weblog-pvc
      - name: log-config
        configMap:
          name: fluentd-config

- apiVersion: v1
  kind: Service
  metadata:
    name: sidecar-svc
  spec:
    selector:
      app:  webserver
    type: NodePort
    ports:
    - name:  sidecar-port
      port:  80
      nodePort:  32000

Comments

  • NargiTNargiT Posts: 6

    Hello,

    For #1 I think it's better to not use env but instead pass arguments to the container

      - name: fdlogger
        image: fluent/fluentd:v1.6-1
        args: ['-c', '/etc/fluentd-config/fluentd.conf']
    

    For #2 I still didn't understand why they send the input data to the localhost

    4th point : There is also a bad link that lead no where for fluentd documenation, it has to be https://docs.fluentd.org/v/1.0/configuration/config-file (if v1) is the target

  • @NargiT
    arguments to the fdlogger container needs to be specified as:
    args: ['fluentd', '-c', '/etc/fluentd-config/fluentd.conf']

  • If you want to stick on environment variable configuration, use FLUENTD_OPT, instead of FLUENTD_ARGS; and it will works.

  • michele.santuarimichele.santuari Posts: 1
    edited July 4

    I agree with @bryonbaker
    The solution does not work and also the output shown in the solution is wrong:
    parsing config file is succeeded path="/fluentd/etc/fluent.conf" means that the config we want to set is not loaded.
    According to the configmap volume, fluentd should load the config placed in:
    /etc/fluentd-config/fluentd.conf

    The solution proposed by @rameshsingal works.
    Thanks!

  • serewiczserewicz Posts: 609

    Hello,

    Indeed Fluent continues to evolve, and the previous configuration does not work the same way anymore. Documentation does not indicate why, or really much on the better way to do the same. One option was mentioned by Bertrand, earlier in the string, by using the FLUENTD_OPT instead.

    We will update or remove this section for the next course release.

    Regards,

Sign In or Register to comment.