Error on running script aio-centos-gcp-deploy.sh

YuriiKantonistov

Hi,

hopefully you can help to troubleshoot this asap.
Tried aio-centos-gcp-deploy.sh script for LAB 4.1, step 4.1.5 several times -
but it fails each time in the same way.

nohup sudo bash -x aio-centos-gcp-deploy.sh 4 &

=>

--- import errors ---
Failed to import test module: neutron_tempest_plugin.api.admin.test_agent_availability_zone
Traceback (most recent call last):
File "/usr/lib64/python3.6/unittest/loader.py", line 428, in _find_test_path
module = self._get_module_from_name(name)
File "/usr/lib64/python3.6/unittest/loader.py", line 369, in _get_module_from_name
import(name)
File "/openstack/venvs/tempest-22.3.2/lib/python3.6/site-packages/neutron_tempest_plugin/api/admin/test_agent_availability_zone.py", line 16, in
from neutron_tempest_plugin.api import base
File "/openstack/venvs/tempest-22.3.2/lib/python3.6/site-packages/neutron_tempest_plugin/api/base.py", line 28, in
from neutron_tempest_plugin.api import clients
File "/openstack/venvs/tempest-22.3.2/lib/python3.6/site-packages/neutron_tempest_plugin/api/clients.py", line 25, in
from tempest.lib.services.network import qos_limit_bandwidth_rules_client
ImportError: cannot import name 'qos_limit_bandwidth_rules_client'
. . .
aio1_utility_container-702cc7d3 : ok=68 changed=39 unreachable=0 failed=1 skipped=19 rescued=0 ignored=0
. . .
EXIT NOTICE [Playbook execution failure] **************************************

How can I resolve this?

raghuramg

Hi @YuriiKantonistov ,

Could you please add my below public key to the /home/aarna/.ssh/authorized_keys and share the Public IP of OpenStack VM?. i will take a look and get back to you.

ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4wz7H/63GV4XmIFLpAh2VhFOuGDZhrABXThtQ80gpPunwI0zgtZJq15k77tdig3540l/DE2JU+QXQfHy38A7TZI9bcUixTbP8Qw4P5hY639dRyU5sBN/PFd1h9kSqZAmWuNXSHV2vdhUJDLn9idbPEJ1YdpwtfxF25ESi5RDvHiunZjZudecGwGlIL+pvu92+VR8j1itUl7aA1b6NCU6Z+Dj3fpWuHz+LkgFH4yaTH1tsPcq9GXPqhACver00INj8sVfSznQZ0cEsnFpr0HTZqGts9KM7YG/j9GfM7gzKjYrqvGL4a90N4+CQ9fBqqEMlNi9sicUWPF0WSjaNuw0j raghuram@raghuram-Latitude-7410

YuriiKantonistov

Hi,

many thanks for the quick response.

Sent you a mail with the information you requested.

Best regaards, Yurii

raghuramg

Hi @YuriiKantonistov ,
I have verified the setup and everything looks good to me. please ignore the below task which is failed and continue with the next steps.

aio1_utility_container-702cc7d3 : ok=57 changed=3 unreachable=0 failed=1 skipped=26 rescued=0 ignored=

YuriiKantonistov

@raghuramg said:
Hi @YuriiKantonistov ,
I have verified the setup and everything looks good to me. please ignore the below task which is failed and continue with the next steps.

aio1_utility_container-702cc7d3 : ok=57 changed=3 unreachable=0 failed=1 skipped=26 rescued=0 ignored=

Thank you for the comment, so you are sure openstack horizon/keystone services are ok?

According to the lab description this would be the OK status, but no such line in my log:

EXIT NOTICE [Playbook execution success] ********************************
grep -q'Playbook execution success'setup-openstack.out

I'm afraid if I just continue - will run again into the issue mentioned in https://forum.linuxfoundation.org/discussion/860326/error-on-running-onap-deploy-all-sh-script#latest

=> cannot login into Horizon UI, always get plaintext json as a response instead of login page.

raghuramg

Hi @YuriiKantonistov ,
The setup looks good, I have verified. Please continue with the next steps. I am able to access the "Horizon UI" of your setup.

YuriiKantonistov

@raghuramg said:
Hi @YuriiKantonistov ,
The setup looks good, I have verified. Please continue with the next steps. I am able to access the "Horizon UI" of your setup.

Thank you! Will continue then.

Could you share how did you verify Horizon UI corectness, somehow myself cannot get to it.

raghuramg

Hi @YuriiKantonistov ,
Please proceed with the next steps and you will see the steps to access "Horizon UI" on pages # 18 and 19.

YuriiKantonistov

This becomes interesting and intense...

Same story - when accessing Horizon GUI via browser only plain json response, see attached picture.
All prior steps were done as aarna user.

Apparently something wrong - could you take a look/give a hint?

$ gcloud compute ssh --zone us-central1-f openstack-01 -- -N -D localhost:5000
+
http://10.128.0.30:5000/v3
=>
{"version": {"id": "v3.14", "status": "stable", "updated": "2020-04-07T00:00:00Z", "links": [{"rel": "self", "href": "http://10.128.0.30:5000/v3/"}], "media-types": [{"base": "application/json", "type": "application/vnd.openstack.identity-v3+json"}]}}

(openstackcli) aarna@anod-master:~/openstack$ cat overcloudrc.v3

export OS_AUTH_URL=http://10.128.0.30:5000/v3

export OS_PROJECT_ID=a54078069a2f47108793f7a44bbe46e3
export OS_PROJECT_NAME="admin"
export OS_USER_DOMAIN_NAME="Default"
export OS_PROJECT_DOMAIN_ID="default"
export OS_USERNAME="admin"
export OS_PASSWORD=e3b0da3b681d940bb83714341bee16071ae518876
export OS_REGION_NAME="RegionOne"
export OS_INTERFACE=public
export OS_IDENTITY_API_VERSION=3

raghuramg

Hi @YuriiKantonistov ,

Please try accessing only with IP. like this; https://10.128.0.30. It works for me.

YuriiKantonistov

@raghuramg said:
Hi @YuriiKantonistov ,

Please try accessing only with IP. like this; https://10.128.0.30. It works for me.

Thank you for the quick response.
Could you share details what is your tunnel/SOCK5 setup when you got it working?

In my case https://10.128.0.30 just fails.

raghuramg

Hi @YuriiKantonistov ,

1. Run SSH Tunnel command:gcloud compute ssh --zone us-central1-f openstack-01 -- -N -D localhost:5000
2. I have used Firefox browser and here is my proxy settings: Browser Setting --> Network Settings

YuriiKantonistov

Lots of thanks for your patience and assistance. Got it working.

YuriiKantonistov

Hi,

still one question regarding this - is something wrong with my setup?:

Login as user admin and with the password in the above file(OS_PASSWORD).
Answer the following questions:
Is DHCP enabled for the external-subnet under the external network?
Is there a router between the OAM_NETWORK and external network?
. . .
How big is the root disk for the m1.medium flavor?
What is the size of the ’trusty’ image in MB?

But I do not see either of external-subnet or OAM_NETWORK or m1.medium flavor or trusty image, like below:

[aarna@aio1 ~]$ source /home/aarna/overcloudrc.v3
[aarna@aio1 ~]$ openstack image list
+--------------------------------------+--------+--------+
| ID | Name | Status |
+--------------------------------------+--------+--------+
| 7a9c9518-a0dd-4466-921b-baef21df4584 | cirros | active |
+--------------------------------------+--------+--------+
[aarna@aio1 ~]$ openstack flavor list
+-----+----------+-----+------+-----------+-------+-----------+
| ID | Name | RAM | Disk | Ephemeral | VCPUs | Is Public |
+-----+----------+-----+------+-----------+-------+-----------+
| 201 | tempest1 | 256 | 1 | 0 | 1 | True |
| 202 | tempest2 | 512 | 1 | 0 | 1 | True |
+-----+----------+-----+------+-----------+-------+-----------+

[aarna@aio1 ~]$ openstack subnet list
+--------------------------------------+----------------+--------------------------------------+-----------------+
| ID | Name | Network | Subnet |
+--------------------------------------+----------------+--------------------------------------+-----------------+
| 495d6dca-536c-41d5-aba9-24acde1f940c | public-subnet | 3a7b761e-2682-4e19-b544-cb612c9b6707 | 172.29.248.0/22 |
| ec82d5f0-9375-434f-bd83-21911a2a2f51 | private-subnet | a106d739-ba75-4b46-840c-a849d68afba6 | 192.168.74.0/28 |
+--------------------------------------+----------------+--------------------------------------+-----------------+

[aarna@aio1 ~]$ cat /home/aarna/overcloudrc.v3

export OS_AUTH_URL=http://10.128.0.30:5000/v3

export OS_PROJECT_ID=a54078069a2f47108793f7a44bbe46e3
export OS_PROJECT_NAME="admin"
export OS_USER_DOMAIN_NAME="Default"
export OS_PROJECT_DOMAIN_ID="default"
export OS_USERNAME="admin"
export OS_PASSWORD=e3b0da3b681d940bb83714341bee16071ae518876
export OS_REGION_NAME="RegionOne"
export OS_INTERFACE=public
export OS_IDENTITY_API_VERSION=3

raghuramg

Hi @YuriiKantonistov , You will all those only after you deploy vFW VNF on OpenStack.

YuriiKantonistov

@raghuramg said:
Hi @YuriiKantonistov , You will all those only after you deploy vFW VNF on OpenStack.

Not an easy step either. When ran these steps in LAB 4.4:

aarna@anod-master $ cd $HOME/onap-custom/util-scripts/
aarna@anod-master $ git pull -v
aarna@anod-master $ nohup ./onap_deploy_all.sh &

=> onap_deploy_all.sh does nothing.

When added "--all" option:

aarna@anod-master $ nohup ./onap_deploy_all.sh &

it did lots of things giving lots of errors about helm/tiller(???), see attached nohup_onap_deply.txt

Kubernetes cluster is up and running but ONAP deployment totally failed.

Would appreciate any hint how to troubleshoot/fix this without total rebuild.

If you need to access it - anod-master public IP is 35.239.221.208, you can get to it with a key I sent couple days ago by private email.

YuriiKantonistov

...second run was of course like this:

aarna@anod-master $ nohup ./onap_deploy_all.sh --all &

raghuramg

Hi @YuriiKantonistov ,

Tiller pod is UP. however, 3 nodes out of 5 worker nodes are not in a Ready state. Please fix this issue and re-run the script again.

aarna@anod-master:~$ kubectl get nodes
NAME STATUS ROLES AGE VERSION
anod-master Ready controlplane,etcd 12h v1.13.5
anod-worker-01 NotReady worker 12h v1.13.5
anod-worker-02 NotReady worker 12h v1.13.5
anod-worker-03 Ready worker 12h v1.13.5
anod-worker-04 NotReady worker 12h v1.13.5
anod-worker-05 Ready worker 12h v1.13.5
anod-worker-06 Ready worker 12h v1.13.5

aarna@anod-master:~$ kubectl get pods -A |grep tiller
kube-system tiller-deploy-7c654c6fbb-g6jsk 1/1 Running 7 9h