Welcome to the Linux Foundation Forum!

Lab 5.3 with fluentd is too confusing?

Posts: 2
edited February 2020 in LFD259 Class Forum

Hello,

While doing lab 5.3 I was disappointed to see how I found it confusing.
As a beginner it took me some time to understand the objective.
I think I saw several mistakes and I would propose following modifications:

FLUENT_CONF env var is not working

Better to use

  1. command: ["fluentd"]
  2. args: ["-c", "/etc/fluentd-config/fluentd.conf"]

We can see even coursebook is wrong
In the sample given in step 15, the config file path is incorrect!!

  1. student@ckad-1:~$ kubectl logs basicpod fdlogger
  2. 2019-03-06 18:55:33 +0000 [info]: parsing config file is succeeded path="/fluentd/etc/fluent.conf"

They should read the ConfigMap content at /etc/fluentd-config/fluentd.conf and not default config at /fluentd/etc/fluent.conf

ConfigMap is incorrect

  • Mountpoint given in step 11 is incorrect should be /var/log/access.log instead of /var/log/nginx/access.log
  • The @type forward type is not relevant and leads to error, use @type stdout instead?
  1. echo '
  2. apiVersion: v1
  3. kind: ConfigMap
  4. metadata:
  5. name: fluentd-config
  6. data:
  7. fluentd.conf: |
  8. <source>
  9. @type tail
  10. format none
  11. path /var/log/access.log # <- we were pointing to wrong mount point /var/log/nginx/access.log in step 11
  12. tag count.format1
  13. </source>
  14.  
  15. <match *.**> # <- we were doing a forward to server which shows nothing
  16. @type stdout
  17. </match>
  18. ' > weblog-configmap.yaml

I propose following modif from step 5.3.11 (I use deployment but pod is fine)

  1. echo '
  2. apiVersion: v1
  3. kind: ConfigMap
  4. metadata:
  5. name: fluentd-config
  6. data:
  7. fluentd.conf: |
  8. <source>
  9. @type tail
  10. format none
  11. path /var/log/access.log
  12. tag count.format1
  13. </source>
  14.  
  15. <match *.**>
  16. @type stdout
  17. </match>
  18. ' > weblog-configmap.yaml
  19.  
  20. k create -f weblog-configmap.yaml
  21.  

And modify deployment as follows

  1. echo '
  2. apiVersion: apps/v1
  3. kind: Deployment
  4. metadata:
  5. creationTimestamp: null
  6. labels:
  7. app: basic
  8. name: basic
  9. spec:
  10. replicas: 1
  11. selector:
  12. matchLabels:
  13. app: basic
  14. strategy: {}
  15. template:
  16. metadata:
  17. creationTimestamp: null
  18. labels:
  19. app: basic
  20. spec:
  21. volumes:
  22. - name: weblog-pv-storage
  23. persistentVolumeClaim:
  24. claimName: weblog-pv-claim
  25. - name: log-config
  26. configMap:
  27. name: fluentd-config
  28. containers:
  29. - image: nginx
  30. name: nginx
  31. ports:
  32. - containerPort: 80
  33. volumeMounts:
  34. - mountPath: "/var/log/nginx"
  35. name: weblog-pv-storage
  36. - image: fluent/fluentd
  37. command: ["fluentd"]
  38. args: ["-c", "/etc/fluentd-config/fluentd.conf", "--verbose"]
  39. name: fdlogger
  40. volumeMounts:
  41. - mountPath: "/var/log"
  42. name: weblog-pv-storage
  43. - mountPath: "/etc/fluentd-config"
  44. name: log-config
  45. ' > basic3.yaml
  46. k create -f basic3.yaml
  47.  

Then we can tail the logs

  1. export POD_NAME=$(k get pods -o wide | grep "basic-" | awk '{ print $1 }')
  2. k logs -f $POD_NAME fdlogger
  3.  
  4. vagrant@k8sMaster:~$ k logs $POD_NAME fdlogger | grep access
  5. path "/var/log/access.log"
  6. 2020-02-13 19:08:22 +0000 [info]: #0 following tail of /var/log/access.log
  7.  

Then in a separate window do:

  1. export POD_IP=$(k get pods -o wide | grep "basic-" | awk '{ print $6 }')
  2. echo $POD_IP
  3. watch -n1 curl $POD_IP

And it appears in first window:

  1. 2020-02-13 19:56:12.855010348 +0000 count.format1: {"message":"10.0.2.15 - - [13/Feb/2020:19:56:12 +0000] \"GET / HTTP/1.1\" 200 612 \"-\" \"curl/7.58.0\" \"-\""}
  2. 2020-02-13 19:56:13.856553408 +0000 count.format1: {"message":"10.0.2.15 - - [13/Feb/2020:19:56:13 +0000] \"GET / HTTP/1.1\" 200 612 \"-\" \"curl/7.58.0\" \"-\""}

Am I wrong, did I miss something? What do you think?

Thanks for the help,

Cheers,

Sylvain

Welcome!

It looks like you're new here. Sign in or register to get started.
Sign In

Comments

  • Posts: 1,000

    Hello,

    Thank you for the feedback. We'll investigate the matter and update the material as necessary.

    Thank you,

  • Posts: 2
    edited April 2020

    @serewicz thanks a lot. Were you able to investigate. Do you plan to update based on this feedback.

  • This still seems to be broken. Setting the FLUENTD_OPT variable does not have any effect, setting the command arguments as @scoulomb suggested works.
    Please update the material.

    Thanks to @scoulomb for giving a hint how to resolve this issue.

Welcome!

It looks like you're new here. Sign in or register to get started.
Sign In

Welcome!

It looks like you're new here. Sign in or register to get started.
Sign In

Categories

Upcoming Training